Re: [Openstack] [Horizon] [UX] phabriactor/pholio as a possible UX option

[Monty Taylor]

Hi!
Sorry - we've been a bit busy and this got put on the back burner. I
believe that ttx has done some work over the last couple of weeks ...
Theirry, any updates from your end?

Github as an option is problematic for several reasons. The ones that
come to mind are that it's not opensource, it doesn't integrate into any
of the rest of our tooling or SSO, and we're actually working to clarify
to people that we do not do our development on github, and adding the
usage of a github issue tracker somewhere would kindof undercut that.

So we're still trying to find an option that we can run that does work
for everyone. I will redouble those efforts.

We also have an effort underway to spin up an owncloud instance ... so
for filesharing needs that might be a choice.


On 07/12/2013 08:47 PM, Toshiyuki Hayashi wrote:
> Hi,
> 
>> So far, my understanding is that requirements are:
>> - discussion
>> - messages containing images
>> - possibly specific image annotation/commenting
> 
> Also if there is file sharing space, that would be great.
> e.g.)
> -  photoshop template for designing and prototyping
> - html template for designing and prototyping
> - wireframe data (it seems Jaromir has created good one already :-) )
> - some document regarding UI
> 
> Discourse seems good, but still Github is better for me.
> Why Github was evaluated as not suitable?
> 
> 
> On Tue, Jul 9, 2013 at 4:52 AM, Jaromir Coufal  wrote:
>>
>> On 2013/27/06 09:37, Thierry Carrez wrote:
>>
>> As a data point, Discourse could also be a solution:
>> http://www.discourse.org/
>>
>> It's clearly a discussion tool (including pretty advanced threading,
>> post likes, etc.), and messages can contain images.
>>
>> See a design discussion for example at:
>> http://test.ubuntu-discourse.org/t/a-ubuntu-ish-theme-for-the-site/177
>>
>>
>> Discourse actually looks pretty good. I was playing around that a little bit
>> and like it. We can consider labels as categories - not optimal, but can
>> work if we have only design discussions. Only problem is that we might want
>> to extend the tool for other discussions as well and then it will be less
>> optimal.
>>
>> Do you guys see any other possibilities apart from this one?
>>
>> -- Jarda
> 
> 
> 

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Horizon] [UX] phabriactor/pholio as a possible UX option

[Monty Taylor]

Hi!

Thanks for looking in to Phabricator for us! The feedback is helpful. I
think we've also got some concerns around elements of it as well.

However, I still don't feel like I fully understand what the
requirements list are here. github isn't a requirement, it's a suggested
solution, and one with already distressing massive negative
implications. so I'd like us to work on what requirements the tool needs
to have so that we can figure out solutions that will solve them.

So far, my understanding is that requirements are:
- discussion
- messages containing images
- possibly specific image annotation/commenting

Are there any others I've missed?

To summarize things we've learned so far about possible solutions:
- Launchpad Bugs don't work due to lack of images
- Phabricator is too image centric, and also confusing
- github issues is not open source, and also increases confusion about
OpenStack's use of github, and is not integrated with the rest of the
project
- mailing list is too text oriented and has a bad threading model

We've got folks working on the area - so let's figure out what we need
and then we can move forward.

thanks!
Monty

On 06/25/2013 12:16 AM, Jaromir Coufal wrote:
> Hey All,
> 
> I investigated and played with few tools for team collaboration, mainly
> focused on designs and discussions. They are mostly similar as
> Phabricator [1], what Monty suggested. You can see inVision [2] or
> GoVisually [3] for example. And of course there are more, however they
> are all somehow similar.
> 
> I have few notes which covers most of them (from my point of view):
> 
> * I can't help myself, but I have disorganized feeling.
> - Might be only my problem, but the whole system navigation is
> just... Strange. Maybe too much graphical :)
> * Main focus on pictures.
> - You can't start a thread without picture.
> - It's just a little bit weird, if everything is focused on the
> picture, which from my point of view shouldn't be the main point.
> Pictures and other documents should be supportive material - discussion
> matters here.
> - Due on pictures focus, discussions are just somehow neglected.
> * I love the inline comments for pictures, but...
>  - Having possibility to attach comment to any place of the picture
> is cool, but... still this tool will fail for example in sequence of
> screens, if you are presenting workflow.
> * Mainly - I don't see developers coming to this tool and actively ask &
> discuss.
> 
> Therefore, also thanks to comments from Toshi and Kyle, I tried to focus
> a little bit more on GitHub. I asked couple of colleagues and friends
> what would they prefer. From developers, the answer was obvious -
> GitHub. Designers said that they wouldn't mind GH, they are ok with it.
> Anyway, it's a normal discussion tool, nothing to be afraid of. The
> reasons why GitHub matters are already covered in my first e-mail and I
> still see it as the best possibility.
> 
> Another reason for GitHub occurred in last conversation on G+ community
> site. There started thread about design question, which got solved, but
> then followed implementation discussion how to implement such thing. And
> here you can see, that any tool focusing on designers in the first
> place, would fail.
> 
> I really don't want to discourage creative people from proposals and
> discussions - completely the opposite. I want them to connect to
> developers and vice versa.
> 
> That's why I believe that GitHub worths trying.
> 
> -- Jarda
> 
> [1] http://www.invisionapp.com/
> [2] http://www.invisionapp.com/
> [3] http://www.govisually.com/
> 
> 
> On 2013/19/06 03:49, Monty Taylor wrote:
>> Hey all!
>>
>> I spoke with Gabriel about this briefly on IRC, but there is an app for
>> Phabricator called Pholio which seems to do the things the UI folks are
>> looking for.
>>
>> To play with it further, I've spun up a phabricator here:
>>
>> http://phab.mnorp.com/
>>
>> You should have gotten account signup emails (if not, look in your spam
>> folder - it's a throwaway machine)
>>
>> Check out:
>>
>> http://phab.mnorp.com/pholio/
>>
>> I've put up one design review here:
>>
>> http://phab.mnorp.com/M1
>>
>> that Jim and I have discussed a little bit.
>>
>> We're not thrilled with Phabricator for things like bug tracking or code
>> review yet - but it's configurable enough that we could turn off
>> everything except design review and move forward with that.
>>
>> Then, if we get to a point where more of its features are useful to us,
>> then gr

Re: [Openstack] New code name for networks

[Monty Taylor]

On 05/13/2013 11:03 AM, Doug Hellmann wrote:
> 
> 
> 
> On Sat, May 11, 2013 at 5:48 PM, Anne Gentle  <mailto:a...@openstack.org>> wrote:
> 
> 
> 
> 
>     On Sat, May 11, 2013 at 3:12 PM, Monty Taylor  <mailto:mord...@inaugust.com>> wrote:
> 
> 
> 
> On 05/11/2013 04:07 PM, Asher Newcomer wrote:
> > Or even better, just continue to call it openstack networking.
> The code
> > names only serve to confuse the uninitiated. They needlessly
> steepen the
> > learning curve and slow uptake.
> 
> The problem with OpenStack Networking (or getting rid of
> codenames) is
> seen with pre-incubation->incubation->integrated cycle.
> 
> A project cannot call itself "OpenStack Foo" until it actually _is_
> openstack foo. So any new project by necessity has to start off as
> something else.
> 
> But - if we then require them to drop that name and become
> openstack foo
> when they become incubated or integrated, then we've got what's
> become a
> stable project with a decent amount of contributors renaming itself.
> 
> Every. Time.
> 
> The code names aren't just cute. I kind of wish they were,
> because it
> would make several things much easier if we could just ditch
> them and do
> a pure openstack. code namespace. But the reality is that it
> gets really
> really tricky to deal with a bunch of things if they go away.
> 
> 
> I told Monty and the TC this at the Summit (sorry I couldn't attend
> the session about code names). I find this trickiness a weak
> argument in the face of the invented names that are getting to be as
> bad as U.S. pharmaceuticals. Plus it forces us to put a "lookup"
> table in the docs forever. [1] Let's find a process for naming that
> meets a few reqs:
> - describes the service
> - goes through a legal review
> - enables new eyes to understanding it by reading the English word
> that the service represents (that can be translated into a
> meaningful word in other languages)
> 
> If we have to preface with Stackforge to indicate pre-incubation,
> that's fine. Use Incubating or some such to indicate incubation.
> Meaningful words have meaning. 
> 
> 
> +1
> 
> Use a namespace package "openstack" then each project has a unique
> package under that for their meaningfully named package (compute,
> networking, etc.).
> 
> We only have to rename projects if they start out with "bad" names to
> begin with. Projects that want to be integrated should be developing on
> stackforge and using the openstack namespace package.

It's just logically infeasible. If we were to make that choice, I'm
pretty sure we'd have to stop everything that everyone is doing and just
deal the fallout from doing it.

In any case - we did have a session on this at the summit, and we did
lay out a strategy for moving forward. The etherpad is here

https://etherpad.openstack.org/ProjectsReNaming

tl;dr - we are going to rename quantum to a new codename, and we are
going to do our best as a community to lessen the external usage of our
codenames.

> I acknowledge we still have to indicate what commands, daemon names,
> and so on are related to a particular service. That issue is also
> fixable with some resources and effort and pays off in easier
> adoption and understanding.
> 
> 
> The unified command line project will resolve the command issue because
> all commands will be "openstack $noun $verb" (end users shouldn't have
> to know which OpenStack component owns a resource in order to operate on
> it via the command line). Daemon startup scripts could use a similar
> framework, or just a naming convention ("openstack-compute-api",
> "openstack-network-api", etc.).

I agree - having the unified command line client will be very helpful to
lessening our externally-facing usages of code names.

> Doug
>  
> 
> 
> Anne
> 
> 1. 
> http://docs.openstack.org/grizzly/openstack-compute/install/yum/content/terminology-codenames.html
>  
>  
> 
> 
> > On May 11, 2013 3:59 PM, "Davanum Srinivas"  <mailto:dava...@gmail.com>
> > <mailto:dava...@gmail.com <mailto:dava...@gmail.com>>> wrote:
> >
> > Lattice
> >
> > -- dims
> >
> > On Sat, May 11, 2013 at 3

Re: [Openstack] New code name for networks

[Monty Taylor]

Don't get me wrong... I don't disagree with you. I think lawyers are super 
important. At the risk of "some of my best friends are..." I took the lsat a 
few years ago (did quite well ;) ) with a view towards law school and my 
girlfriend is in law school. TOTALLY respect for lawyers.

I don't want to keep lawyers away from the project any more than I want to keep 
business people away. What I do want to do is keep random lawyers away from 
implementation details of our source code repositories.

Here's the rub... The technical leadership of the project has no body from 
which it can receive legal counsel. The foundation has counsel, but that is 
different than the TC having actual counsel to whom we can give intent 
direction and receive advice. We have many interested and involved parties who 
are top lawyers, and I regularly talk with them. But they're all quite clear 
that they cannot directly advise us as the collected technical leadership.

So we are in a sticky situation of needing to make decisions that have actual 
technical ramifications, and that may have legal consequences, but no counsel 
to whom we can give priorities and direct to help us find solutions.

>From the foundation perspective, the risk/reward is clear, and from that 
>perspective discussions with counsel can proceed. But in addition to a board 
>who can have an executive session with counsel to which privilege attaches, 
>the foundation has executive director who can also have privileged 
>discussions. Those are from a particular POV, and it's a totally valid one.

The technical leadership, OTOH, does not have this ability, nor are we 
interested in doing things in private. We are tasked with technical leadership, 
which is what we intend to do.

So, with all due respect to the many awesome lawyers who are involved with the 
project, what I'm getting at is that we need to do the right thing as best we 
can based on unofficial advice from well meaning people who may or may not have 
legal backgrounds as well as input and help from Jonathan Bryce and the 
foundation board where appropriate.

I wish there was a more straightforward way to approach this, but it's not a 
straightforward situation. However, maybe if we play our cards right and are 
conscientious in our approach to the problems we can help forge ground for more 
people who want to run a thing like we are without getting into too much 
trouble along the way.

Brad Knowles  wrote:

>On May 12, 2013, at 9:52 AM, Monty Taylor  wrote:
>
>> I would really like to keep the marketing/business folks out of our
>> source code.
>> 
>> Most importantl, I would really like to keep the lawyers out of our
>> source code.
>
>My wife is a lawyer, so maybe I'm particularly sensitive to this issue.  
>However, I don't believe that the lawyers themselves are the problem.  I think 
>the problem is that maybe you haven't gotten the *RIGHT* lawyers involved..
>
>The RIGHT lawyers can help you spot potential problems while they're still 
>just mole hills in the distance, and can help you avoid having them get turned 
>into mountains.  But they can't just give you a set of rules and have you 
>follow them blindly -- the reason they are the RIGHT lawyers is because of all 
>their experiences and the lessons they've learned, and the places where 
>they've seen clients trip up in the past, and therefore they know what to look 
>for.  They can't necessarily tell you what to look for, they'll just know it 
>when they see it.  Sure, they have rules that will cover the easy 80%, but 
>it's the hard 20% that you have to really worry about.
>
>The RIGHT lawyers will know when they look at a contract what kinds of things 
>don't need to be written down, because they're covered by laws on the books, 
>or by existing case law.  And when they look at contracts in a foreign 
>country, they'll have some idea of what kinds of questions to ask -- and the 
>different kind of legal systems around the world, how they differ, etc.
>
>This is the kind of thing that got BazaarVoice in trouble -- they went out of 
>their way to structure the buyout of their major competitor in such a way that 
>it didn't need to be approved in advance by the regulators.  But then they got 
>hit with a lawsuit by the federal government, and they ended up having to 
>divest themselves of most of the assets they had bought.  Had they structured 
>the deal differently, they could have at least known in advance whether or not 
>the regulators would have approved it, and if approved would never had to 
>divest themselves of their acquisition.
>
>
>The RIGHT lawyers can help you see and avoid these problems before you ever 
>get there, but even they can't necessarily

Re: [Openstack] New code name for networks

[Monty Taylor]

On 05/11/2013 08:58 PM, Anne Gentle wrote:
> 
> 
> 
> On Sat, May 11, 2013 at 6:43 PM, Monty Taylor  <mailto:mord...@inaugust.com>> wrote:
> 
> 
> 
> On 05/11/2013 05:48 PM, Anne Gentle wrote:
> >
> >
> >
> > On Sat, May 11, 2013 at 3:12 PM, Monty Taylor  > <mailto:mord...@inaugust.com <mailto:mord...@inaugust.com>>> wrote:
> >
> >
> >
> > On 05/11/2013 04:07 PM, Asher Newcomer wrote:
> > > Or even better, just continue to call it openstack
> networking. The
> > code
> > > names only serve to confuse the uninitiated. They needlessly
> > steepen the
> > > learning curve and slow uptake.
> >
> > The problem with OpenStack Networking (or getting rid of
> codenames) is
> > seen with pre-incubation->incubation->integrated cycle.
> >
> > A project cannot call itself "OpenStack Foo" until it actually
> _is_
> > openstack foo. So any new project by necessity has to start off as
> > something else.
> >
> > But - if we then require them to drop that name and become
> openstack foo
> > when they become incubated or integrated, then we've got
> what's become a
> > stable project with a decent amount of contributors renaming
> itself.
> >
> > Every. Time.
> >
> > The code names aren't just cute. I kind of wish they were,
> because it
> > would make several things much easier if we could just ditch
> them and do
> > a pure openstack. code namespace. But the reality is that it
> gets really
> > really tricky to deal with a bunch of things if they go away.
> >
> >
> > I told Monty and the TC this at the Summit (sorry I couldn't
> attend the
> > session about code names).
> 
> I promise, it wasn't the world's most fun session. :)
> 
> 
> I'm sure. :) I think I don't have much regret but do feel sorry that I
> don't know more. 
>  
> 
> 
> > I find this trickiness a weak argument in the
> > face of the invented names that are getting to be as bad as U.S.
> > pharmaceuticals. Plus it forces us to put a "lookup" table in the docs
> > forever. [1] Let's find a process for naming that meets a few reqs:
> > - describes the service
> > - goes through a legal review
> > - enables new eyes to understanding it by reading the English word
> that
> > the service represents (that can be translated into a meaningful
> word in
> > other languages)
> 
> I don't think it's a weak argument at all. There are real technical
> issues.
> 
> 
> The technical issues, to me, and I may be missing something, are when
> the name is used as:
> - service/daemon name
> - command/CLI name

And the directories in the code where those things live, and the name of
the python package that gets installed, and the name of the client
library used to connect to it.

> You can use any pet name you want for your team and project while
> addressing technical issues some other way? 
> 
> Here's another way I'm looking at the naming autonomy/process. Why
> incubate? 
> - you get to pick your cool name
> OR
> - you get access to infrastructure, tools, events, community, and
> branding that is OpenStack
> 
> The naming can't be THAT crucial. I get that we want projects to be fun
> to work on. But it can't be just the naming that brings the fun.

I don't think "having a cool name" is interesting or important at all.
Not one little bit. If any part of this was about esprit de corps or
team bonding or identity I'd be 100% on board with the no-codenames
approach.

Also, to be clear, I don't think there are any problems with using
non-codename for identification. Already, as part of the upgrade of our
build stuff to PBR I've been setting the project short description to
the non-codename. So, nova's is "OpenStack Compute" I think that's a
great idea, and it's important.

Equally as important, although harder, is that we should all try our
best to use the non-codenames when we're talking about official
projects. It's not going to work or be 100% covered - but we should all
make a best-effort.

(these are all things that did come out of that session - perhaps one of
us should do a writeup on that?)

The thing that _I'm_ sticking on is the ab

Re: [Openstack] New code name for networks

[Monty Taylor]

On 05/11/2013 05:48 PM, Anne Gentle wrote:
> 
> 
> 
> On Sat, May 11, 2013 at 3:12 PM, Monty Taylor  <mailto:mord...@inaugust.com>> wrote:
> 
> 
> 
> On 05/11/2013 04:07 PM, Asher Newcomer wrote:
> > Or even better, just continue to call it openstack networking. The
> code
> > names only serve to confuse the uninitiated. They needlessly
> steepen the
> > learning curve and slow uptake.
> 
> The problem with OpenStack Networking (or getting rid of codenames) is
> seen with pre-incubation->incubation->integrated cycle.
> 
> A project cannot call itself "OpenStack Foo" until it actually _is_
> openstack foo. So any new project by necessity has to start off as
> something else.
> 
> But - if we then require them to drop that name and become openstack foo
> when they become incubated or integrated, then we've got what's become a
> stable project with a decent amount of contributors renaming itself.
> 
> Every. Time.
> 
> The code names aren't just cute. I kind of wish they were, because it
> would make several things much easier if we could just ditch them and do
> a pure openstack. code namespace. But the reality is that it gets really
> really tricky to deal with a bunch of things if they go away.
> 
> 
> I told Monty and the TC this at the Summit (sorry I couldn't attend the
> session about code names). 

I promise, it wasn't the world's most fun session. :)

> I find this trickiness a weak argument in the
> face of the invented names that are getting to be as bad as U.S.
> pharmaceuticals. Plus it forces us to put a "lookup" table in the docs
> forever. [1] Let's find a process for naming that meets a few reqs:
> - describes the service
> - goes through a legal review
> - enables new eyes to understanding it by reading the English word that
> the service represents (that can be translated into a meaningful word in
> other languages)

I don't think it's a weak argument at all. There are real technical issues.

That assumes that OpenStack is involved with the project pre-incubation.
That's was the case with Quantum and Ceilometer and Ironic. On the other
hand, the heat folks had heat-api.org and a heat github org and other
things before they started down the stackforge road. Looking right now
at non-incubated projects just off the top of my head:

Libra is an LBaaS solution that was started on stackforge and which it's
increasingly unlikely will go to incubation rather than just atttempt to
merge stuff in with Quantum LBaaS. Moniker is a DNSaaS that hasn't
applied for incubation, might do so, but as of right now has been around
for almost a year yet has no formal relationship with OpenStack.
healthnmon may or may not be an incubation candidate.

Before that, reddwarf was not-incubated for pretty much the entire time
OpenStack has existed until just now.

All of these things have had to have lifecycles during a period of time
before they have any interaction with us formally.

On the other hand, if we did checking pre-incubation, we'd be in a very
odd position of granting permission/blessing/tentative interest in
projects before they come close to sorting things out. Moniker is great,
but 6 months ago there were as many as 4 different DNSaaS possibilities.

In any case, I don't think there is any way that we can, become directly
involved in projects before they are incubated.

Stackforge helps already, in that moniker is stackforge/moniker rather
than openstack/moniker. But neither has any effect on the fact that
there is a directory named "moniker" in moniker repository. If we go
with 'descriptive' names, then there are two choices:

a) moniker starts life with a directory structure underneath
openstack/dns inside of its repository, even though it is not an
openstack project.

b) moniker starts life with a moniker directory, and then when
incubated, renames that directory from moniker to openstack/dns.

We can't stop anybody from doing (a) of course, but let me tell you -
you wanna talk about confusing and bad messaging - we had apt repos at
the beginning of the project with giant letters on them "FOR TESTING
ONLY" but because they had our name on them people assumed we supported
them.

(b) sound easy, until you reckon with things like line-wrapping changes,
sort order changes, and client library/API consumption changes. If
something is using python-monikerclient and thus the monikerclient
namespace of the API, that person would have to port their code to now
do something like openstack.dns.client or similar.

Even ignoring people who may have already been using the code (many of
these things start life as a service by one of our member companies and
then 

Re: [Openstack] Guest PXE Boot

[Monty Taylor]

Neat!

Have you seen any of the work around nova baremetal (which is
transitioning to be called ironic?) Related to that is a set of "virtual
power drivers" which allow for treating virtual machines like real
machines - so that you can use nova to pxe boot a kvm or a virtualbox or
a vmware instance.

I know it's not exactly the same thing, but I don't know what you're
trying to accomplish. Perhaps what you want is similar enough to work
together?

Monty

On 05/10/2013 12:55 PM, David Hill wrote:
> Hi guys,
> 
>  
> 
> I was trying to PXE boot a guest for quite some time now and I think
> I’ve found a solution that is kind of hackish but pretty simple.   I’m
> not quite sure it’s good to go in trunk but felt like I’d share it since
> I’ve been messing a while on this.  
> 
> If anybody have a better solution, I would really like to hear/see/try it …
> 
>  
> 
> Here is how I did it:
> 
>  
> 
> First, patch the libvirt/driver.py file:
> 
> --- /usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py.orig  
> 2013-05-10 16:25:17.787862177 +
> 
> +++ /usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py   
> 2013-05-10 16:26:39.442022870 +
> 
> @@ -87,6 +87,9 @@
> 
> LOG = logging.getLogger(__name__)
> 
>  
> 
> libvirt_opts = [
> 
> +cfg.StrOpt('default_guest_boot_dev',
> 
> +   default='hd',
> 
> +   help='Sets the default guest boot device'),
> 
>  cfg.StrOpt('rescue_image_id',
> 
> default=None,
> 
> help='Rescue ami image'),
> 
> @@ -1792,7 +1795,7 @@
> 
> instance['name'],
> 
> "ramdisk")
> 
>  else:
> 
> -guest.os_boot_dev = "hd"
> 
> +guest.os_boot_dev = FLAGS.default_guest_boot_dev
> 
>  
> 
>  if FLAGS.libvirt_type != "lxc" and FLAGS.libvirt_type != "uml":
> 
>  guest.acpi = True
> 
>  
> 
>  
> 
> And add to nova.conf:
> 
> default_guest_boot_dev=network
> 
>  
> 
> And finally add to /etc/dnsmasq.conf
> 
> dhcp-boot=boot\x86\pxelinux.com,host_name,host_ip
> 
> dhcp-no-override
> 
>  
> 
> And restart dnsmasq.conf
> 
>  
> 
> In my actual setup, the guest will PXE boot, show the menu 60 seconds
> and then boot from hard disk after the 60 seconds timeout.
> 
>  
> 
>  
> 
> Thank you very much,
> 
>  
> 
> Dave
> 
> 
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
> 

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] New code name for networks

[Monty Taylor]

Jeremy Stanly on IRC just suggested kumquat... but to that I respond:

qumkuat

Same benefits as qumutna - except it's more pronouncable.

On 05/11/2013 04:07 PM, Monty Taylor wrote:
> I have been arguing for:
> 
> mutnuaq
> 
> Granted, it takes a minute to learn how to type, but it's just a little
> snarky, and it takes up the exact same number of letter. However, it
> does screw with sorting. SO - what about:
> 
> qumutna
> 
> It's a little bit easier to wrap your head around, it's still clearly an
> homage, and it should be super easy to bulk cut/replace.
> 
> On 05/11/2013 03:58 PM, Davanum Srinivas wrote:
>> Lattice
>>
>> -- dims
>>
>> On Sat, May 11, 2013 at 3:52 PM, Mark Turner  wrote:
>>> Tubes
>>>
>>> ;-)
>>>
>>>
>>> On Sat, May 11, 2013 at 12:51 PM, Jason Smith 
>>> wrote:
>>>>
>>>> Hello,
>>>> I understand why we had to give up Quantum code name but rather than just
>>>> refer to it as networking let's come up with a new code name!
>>>>
>>>> Thoughts?
>>>>
>>>> Thanks,
>>>> -js
>>>> ___
>>>> Mailing list: https://launchpad.net/~openstack
>>>> Post to : openstack@lists.launchpad.net
>>>> Unsubscribe : https://launchpad.net/~openstack
>>>> More help : https://help.launchpad.net/ListHelp
>>>
>>>
>>>
>>> ___
>>> Mailing list: https://launchpad.net/~openstack
>>> Post to : openstack@lists.launchpad.net
>>> Unsubscribe : https://launchpad.net/~openstack
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>
>>
>>
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
> 

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] New code name for networks

[Monty Taylor]

On 05/11/2013 04:07 PM, Asher Newcomer wrote:
> Or even better, just continue to call it openstack networking. The code
> names only serve to confuse the uninitiated. They needlessly steepen the
> learning curve and slow uptake.

The problem with OpenStack Networking (or getting rid of codenames) is
seen with pre-incubation->incubation->integrated cycle.

A project cannot call itself "OpenStack Foo" until it actually _is_
openstack foo. So any new project by necessity has to start off as
something else.

But - if we then require them to drop that name and become openstack foo
when they become incubated or integrated, then we've got what's become a
stable project with a decent amount of contributors renaming itself.

Every. Time.

The code names aren't just cute. I kind of wish they were, because it
would make several things much easier if we could just ditch them and do
a pure openstack. code namespace. But the reality is that it gets really
really tricky to deal with a bunch of things if they go away.

> On May 11, 2013 3:59 PM, "Davanum Srinivas"  > wrote:
> 
> Lattice
> 
> -- dims
> 
> On Sat, May 11, 2013 at 3:52 PM, Mark Turner  > wrote:
> > Tubes
> >
> > ;-)
> >
> >
> > On Sat, May 11, 2013 at 12:51 PM, Jason Smith
> mailto:jason.sm...@rackspace.com>>
> > wrote:
> >>
> >> Hello,
> >> I understand why we had to give up Quantum code name but rather
> than just
> >> refer to it as networking let's come up with a new code name!
> >>
> >> Thoughts?
> >>
> >> Thanks,
> >> -js
> >> ___
> >> Mailing list: https://launchpad.net/~openstack
> >> Post to : openstack@lists.launchpad.net
> 
> >> Unsubscribe : https://launchpad.net/~openstack
> >> More help : https://help.launchpad.net/ListHelp
> >
> >
> >
> > ___
> > Mailing list: https://launchpad.net/~openstack
> > Post to : openstack@lists.launchpad.net
> 
> > Unsubscribe : https://launchpad.net/~openstack
> > More help   : https://help.launchpad.net/ListHelp
> >
> 
> 
> 
> --
> Davanum Srinivas :: http://davanum.wordpress.com
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> 
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
> 
> 
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
> 

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] New code name for networks

[Monty Taylor]

I have been arguing for:

mutnuaq

Granted, it takes a minute to learn how to type, but it's just a little
snarky, and it takes up the exact same number of letter. However, it
does screw with sorting. SO - what about:

qumutna

It's a little bit easier to wrap your head around, it's still clearly an
homage, and it should be super easy to bulk cut/replace.

On 05/11/2013 03:58 PM, Davanum Srinivas wrote:
> Lattice
> 
> -- dims
> 
> On Sat, May 11, 2013 at 3:52 PM, Mark Turner  wrote:
>> Tubes
>>
>> ;-)
>>
>>
>> On Sat, May 11, 2013 at 12:51 PM, Jason Smith 
>> wrote:
>>>
>>> Hello,
>>> I understand why we had to give up Quantum code name but rather than just
>>> refer to it as networking let's come up with a new code name!
>>>
>>> Thoughts?
>>>
>>> Thanks,
>>> -js
>>> ___
>>> Mailing list: https://launchpad.net/~openstack
>>> Post to : openstack@lists.launchpad.net
>>> Unsubscribe : https://launchpad.net/~openstack
>>> More help : https://help.launchpad.net/ListHelp
>>
>>
>>
>> ___
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
>>
> 
> 
> 

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] TC Election Results

[Monty Taylor]

The Sprint 2013 TC Election has concluded. The at-large members elected are:

vishy
ttx

For a term of one year.

mikal

For a term of six months.

Congratulations.

http://www.cs.cornell.edu/w8/~andru/cgi-perl/civs/results.pl?id=E_5af0b5341a01b892

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] REMINDER: TC election closes tomorrow

[Monty Taylor]

Hey all!

186 of 618 voters have cast ballots in the TC election. Tomorrow is the
last day ... go vote now!

Monty

PS. Look for an email titled "Poll: Spring 2013 OpenStack TC Election"
and click the link

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] TC candidacy

[Monty Taylor]

I confirm that Vishvananda is eligible to run for a seat on the TC.

On 03/15/2013 05:41 PM, Vishvananda Ishaya wrote:
> Hello all,
> 
> I would like to run for a seat on The Technical Comittee. I have been working 
> on Nova since it was a project as Nasa and I have been heavily involved in 
> openstack since it was founded. I was elected to the precursor to TC (the 
> Project Oversight Committee, later named the Project Policy Board) when it 
> was first created.
> 
> I was also elected as the first PTL for Nova and have been filling that role 
> for the last two years. I am the top contributor to Nova over the lifetime of 
> the project, and the third most frequent contributor over the past 12 months. 
> I helped to create Devstack, Keystone, and Cinder. In addition, I have 
> contributed to Oslo and I am a member of the stable-maintenance team.
> 
> Despite passing on the mantle of Nova PTL, I am still deeply involved with 
> OpenStack and I want to make sure that it continues to be a huge success. As 
> OpenStack grows, one of the most important challenges we face is integration. 
> It is vital that we have technical leaders that are focused cross-project and 
> dedicated to making OpenStack as a whole successful.
> 
> I currently work as the Director of Open Source at Nebula, Inc. Previously I 
> was a principal engineer on the private cloud team at Rackspace, and before 
> that I was a senior developer on the Nebula project at NASA where Nova was 
> created.
> 
> Thanks,
> Vish
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
> 

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] TC candidacy

[Monty Taylor]

I confirm that Gary is eligible to run for a seat on the TC.

On 03/15/2013 02:13 PM, Gary Kotton wrote:
> Hi,
> 
> I'd like to run for the Technical Committee in the up and coming elections.
> 
> I am a Principle Software Engineer at Red Hat. I have been actively
> developing OpenStack since the Essex release. I am currently a Quantum
> core developer. In addition to this I am also core on the Stable
> Maintenance team. I also have contribute to Nova, OSLO, documentation
> and devstack. In Nova the work was mainly focused on the Quantum
> integrations. My latest contribution was the "VM ensembles", part of
> which was added in Grizzly release and hopefully will be completed in
> the up and coming Havana release. I spend most of my days reviewing,
> testing, debugging, documenting and developing with the goal of making
> OpenStack better. I am thankful to my employer Red Hat to have the
> opportunity and time to work on such and amazing project.
> 
> I have close to 18 years of experience in the industry. Over that course
> of time I have strived to produce quality, usable, robust and optimal
> solutions. I would like to bring all that experience to the table to
> ensure that we have a better product. We are working in a very healthy,
> dynamic and vibrant community. A few things that I would like to improve
> are the following:
> - cross project interaction
> - growth of the community
> - sharing of ideas and information
> 
> In my spare time I run.
> 
> Thanks
> Gary
> 
> 
> 
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
> 

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] TC Candidacy

[Monty Taylor]

I confirm that Chuck is eligible to run for a seat on the TC.

On 03/16/2013 12:59 PM, Chuck Thier wrote:
> Hello all,
> 
> I would like to run for a seat on the TC.   I am one of the original
> developers of Rackspace Cloud Files which became Openstack Swift, and
> was deeply involved with the creation of Openstack.  I also lead the
> team at Rackspace to create Cloud Block Storage which built off the
> foundation of Openstack Cinder, and during that time contributed
> directly and indirectly to Nova and Cinder.  I have the unique
> experience of not only developing across several Openstack projects, but
> also being responsible for deploying the projects at a very large scale.
>  I have a track record for fighting for reasonable APIs, upgradeability,
> and maintainability across projects.  I have also fought to ensure that
> we have equal and fair representation from all projects in the Openstack
> community. 
> 
> The purpose of the TC is not to "legislate from the bench", but when
> questions and issues are brought to the TC I will continue to support
> these ideals.  I deeply care for Openstack and its future success, so
> please consider me for this position.
> 
> Thanks,
> 
> --
> Chuck Thier
> @creiht
> 
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
> 

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] TC candidacy

[Monty Taylor]

I confirm that Michael is eligible to run for a seat on the TC.

On 03/15/2013 11:27 AM, Michael Still wrote:
> Hi. I'd like to run for the TC Spring 2013 election. I am a senior
> software engineer at Rackspace in their OpenStack group, and have
> worked in a variety of "cloud devops" roles for the last seven years.
> I think my operations experience gives me an interesting perspective
> into where OpenStack should be going in the next few years.
> 
> My basic platform is the same as for the Nova PTL election [1] -- I
> think we need to get better at closing bugs, and selecting defaults
> which work "out of the box" for OpenStack deployers. We are blessed
> with a very engaged user community, and we need to focus us much as
> possible on giving new users a good experience.
> 
> I am an active Nova and Oslo core reviewer and frequently appear in
> the top ten contributors for Nova in a given month. I am a very active
> code reviewer, especially for Nova. I am also serve on the OpenStack
> Vulnerability Management Team. I strongly believe in the future of
> OpenStack and want to be part of that success in any way I can. For
> the last two years my non-OpenStack open source contributions have
> mainly been as Director for linux.conf.au 2013, which was the largest
> OpenStack event to be run in Australia so far.
> 
> Thanks,
> Michael
> 
> 1: http://lists.openstack.org/pipermail/openstack-dev/2013-March/006417.html
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
> 

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] TC election open

[Monty Taylor]

If you are an ATC, you should by this point have received a link that
will let you vote in the OpenStack TC election. You should vote.

If you read it, you will note that it says February 28 is the end date.
That is an error - March 28 is the end date.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] TC candidacy

[Monty Taylor]

I confirm that Eric is eligible to run for a seat on the TC.

On 03/18/2013 03:23 PM, Eric Windisch wrote:
> Hello,
> 
> I'd like to run for a seat on the Technical Committee. I am a Principal 
> Engineer at Cloudscaling, but I am running as an individual.
> 
> For over two years, beginning with Bexar, I have been working to bring 
> working OpenStack solutions to customers. My first code contributions landed 
> in Cactus, but my contributions have not been limited to code alone. I 
> propose and drive summit and mailing list discussions, contribute to those 
> discussions led by others, have begun contributing to packaging efforts, and 
> soon intend to land documentation changes.
> 
> The code I do write and review tends to be in Oslo. Working with Oslo, and 
> with OpenStack deployments, I have the perspective to work across projects.
> 
> Before my involvement with OpenStack, in my former role as the owner and 
> technical director of a VPS hosting company, I was already building open 
> source cloud infrastructure as early as 2007.  For years preceding the 
> existence of OpenStack, I've worked on many of the technical problems and 
> challenges facing a cloud, not only those related to virtualization, 
> networking, and storage, but also billing, metering, and user interfaces.
> 
> The number of companies and contributors involved in OpenStack is exploding. 
> Each conference has been shockingly larger than the last. We have new 
> projects being added in each 6-month release. These are problems that the TC 
> must be prepared to deal with. We need technical leadership that understands 
> the problems of these projects, can help them succeed, and especially for 
> those individually elected seats -- can bring them to together.
> 
> I am deeply committed to the success of OpenStack and to open source cloud. I 
> thank you for your consideration and ask that you please vote for me. 
> 
> Regards,
> Eric Windisch
> 
> 
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
> 

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] TC candidacy

[Monty Taylor]

I confirm that Chris is eligible to run for a seat on the TC.

On 03/18/2013 06:11 PM, Chris Behrens wrote:
> 
> Hi all,
> 
> I'd like to announce my candidacy for a seat on the OpenStack
> Technical Committee.
> 
> - General background -
> 
> I have over 15 years of experience designing and building distributed
> systems.  I am currently a Senior Software Developer at Rackspace,
> where I have been for a 2 and a half years now.  Most of my time
> at Rackspace has been spent working on OpenStack as both a developer and
> a technical leader.  My first week at Rackspace was spent at the very first
> OpenStack Design Summit in Austin where the project was announced.
> 
> Prior to working at Rackspace, I held various roles over 14 years
> at Concentric Network Corporation/XO Communications including Senior
> Software Architect and eventually Director of Engineering.  My main
> focus there was on an award winning web/email hosting platform which
> we'd built to be extremely scalable and fault tolerant.  While my
> name is not on this patent, I was heavily involved with the development
> and design that led to US6611861.
> 
> - Why am I interested? -
> 
> I have strong feelings for OpenStack and I want to help take it to
> the next level.  I have a lot of technical knowledge and experience
> building scalable distributed systems.
> 
> During most of my past experience, I haven't had the luxury of having
> access to a lot extremely fast hardware, so it's been important to
> make software as performant as possible.  I've also had to put lots of
> effort into having 0 downtime, meaning code can be updated seamlessly
> without dropping clients.  I've also been one to lead host and software
> security efforts so I have a lot of strong feelings in this area.
> 
> I am extremely interested in using this experience to make OpenStack
> perform well, be secure, be more easily pluggable, and easy to use!
> 
> - OpenStack contributions -
> 
> As I mentioned above, I was at the very first design summit, so
> I've been involved with the project from the beginning.  I started
> the initial work for nova-scheduler shortly after the project was
> opened.  I also implemented the RPC support for kombu, making sure
> to properly support reconnecting and so forth which didn't work
> quite so well with the carrot code.  I've contributed a number of
> improvements designed to make nova-api more performant.  I've worked on
> the filter scheduler as well as designing and implementing the
> first version of the Zones replacement that we named 'Cells'.
> 
> I'm currently looking forward to restructuring our use of DB API to better
> support upgrades w/ schema changes as well as committing an alternative
> DB backend implementation for mysql that significantly reduces how long
> we block on DB API calls compared to sqlalchemy.
> 
> - Summary -
> 
> I feel my years of experience contributing to and leading large scale
> technical projects along with my knowledge of the OpenStack projects
> will provide a good foundation for technical leadership.
> 
> Thanks,
> 
> - Chris
> 
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
> 

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] TC candidacy

[Monty Taylor]

I confirm that Carl is eligible to run for a seat on the TC.

On 03/15/2013 07:08 PM, Carl Perry wrote:
> Greetings -
> 
> I would like to run for a TC seat as well. My platform is a focus on
> deployment and operations for OpenStack. I'm not going to mince words:
> deploying OpenStack is hard. Maintaining it is even harder. I don't
> think it needs to be this way. I spent two years designing and deploying
> a cloud based on OpenStack and it really seemed much harder to do than
> it should have been, especially when you factor in configuration
> management tools.
> 
> I would like to promote a more operational approach to the decisions
> made inside the OpenStack community. Some of it is easy, some of it is
> hard. I don't expect things to change overnight, but I do feel that a
> course correction is needed, that it's going to need to come from a
> whole project approach as well as code contribution, and that now is the
> time to make it.
> 
> As I said, I spent the last two years as an implementor of OpenStack
> with the DreamCompute project at DreamHost and have now moved on to a
> new position as a solutions architect at Midokura. These positions have
> given (and continue to give) me a customer perspective of what it takes
> to deploy, maintain, and upgrade OpenStack in production. While my
> company supports me and my desire to run for the TC, I'm running
> independent of my company in order to keep a vendor neutral approach.
> 
> Thanks for your time!
>   -Carl
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
> 

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] TC candidacy

[Monty Taylor]

I confirm that Thierry is eligible to run for a seat on the TC.

On 03/15/2013 11:32 AM, Thierry Carrez wrote:
> Hi everyone,
> 
> I'd like to run for reelection to one of the Technical Committee
> directly-elected seats.
> 
> For those who don't know me, I've been handling release management
> duties for OpenStack since November 2010, a work currently sponsored by
> the OpenStack Foundation. My involvement is mostly around project
> coordination, keeping a global view and trying to anticipate issues as
> our development community grows even larger. I'm also heading the
> Vulnerability Management team, which handles incoming security issues
> reports. On the development side, I authored the rootwrap framework
> which is being used by a few of our projects, and whenever I find some
> free time, I'm working on improving it.
> 
> I've been regularly elected by our community to the Project Policy Board
> and Technical Committee directly-elected seats for the last two years. I
> was heavily involved in the transition to our new governance, authored
> the Technical Committee charter, and have been chosen to chair it for
> the past 6 months.
> 
> I think it's important that the Technical Committee contains
> representation from the horizontal functions within the project (Docs,
> QA, Infrastructure, Vulnerability management, Release management...),
> since each project is already represented by the seats granted for all PTLs.
> 
> Over the last 30 months we grew from 2 projects to 10 projects, and I'm
> proud to be part of this community which successfully managed to handle
> growth and adoption while preserving our ideals of open design and open
> development.
> 
> The challenges ahead of us include accommodating further growth, resist
> fragmentation, and maintaining efficiency and coherence as we grow well
> past Dunbar's number. I hope that you place me in a position where I can
> help us through those challenges.
> 
> Thanks,
> 

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] Technical Committee Nominations are Open

[Monty Taylor]

Now that the TC elections have ended, we now have three at-large seats open.

https://wiki.openstack.org/wiki/TC_Elections_Spring_2013

Mar 15 - 21: Open candidacy to directly-elected TC positions
Mar 22 - 28: TC elections

The persons ranking 1st and 2nd will get one-year seats on the TC, and
the person ranking 3rd will be elected for a 6-month seat as a
replacement for Russell Bryant who is now granted a PTL seat.

The electorate for the TC direct seats election are the Foundation
individual members that are also committers for an official OpenStack
project, over the Folsom-Grizzly timeframe, up to 23:59 PST on February
28, 2013.

Any member of the election electorate can propose his candidacy for this
election. No nomination is required. They do so by sending an email to
the openstack@lists.launchpad.net mailing-list, which the subject: "TC
candidacy". The email can include a description of the candidate
platform. The candidacy is then confirmed by one of the election
officials, after verification of the electorate status of the candidate.

Note: PTLs that just got elected in the previous election are
automatically granted a 6-month term seat on the TC, and therefore
cannot run for this election.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] PTL Election Results

[Monty Taylor]

The PTL elections for the Havana cycle have completed. The new PTL's are:

Nova: Russell Bryant
Ceilometer: Julien Danjou
Keystone: Dolph Matthews

Congratulations!

As a side note, we had over 50% participation in each of the three
elections, which I have been told is actually a really good turnout.

Monty

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] PTL Elections are Open!

[Monty Taylor]

If you are an ATC for a keystone, nova or ceilometer, you should have
now received in the mail your link to vote in the PTL election for that
project be sure to vote! The elections end March 14.

For the other projects, there was only one person standing for election,
so congratulations guys, you win!

Havana PTLs so far:

Glance: Mark Washenberger
Heat: Steven Dake
Oslo: Mark McLoughlin
Quantum: Marck McClain
Swift: John Dickenson
Horizon: Gabriel Hurley
Cinder: John Griffith

Monty

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] orchestration engine wrt to openstack api

[Monty Taylor]

On 03/04/2013 10:02 AM, Nirlay Kundu wrote:
> Which tool would you use for configuration management geared towards
> Openstack api : Chef, Puppet, Saltstack ? If anybody has experience with
> Saltstack, please let me know the advantages , shortcomings.


I would use Heat to orchestrate thing WRT the openstack APIs. On a
per-machine basis, chef, puppet and salt should all be fine for doing
configuration management and can read the metadata that heat provides
onto the machine.

Monty

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] PTL and Technical Committee Election Time

[Monty Taylor]

Hi everyone!

Spring is upon us, which means it's time to overthrow our leadership and
replace it with new (or the same) leadership!

First we elect PTL's, then we elect TC at-large members. So first things
first:

** PTL Candidacy Nomination Period is open from now until March 7 **

If you would like to announce that you would like to run for PTL for a
project, please send an email to openstack-...@lists.openstack.org and
tell us. You should probably tell us WHY you're a good choice, but I'll
leave that decision up to you.

The nomination period will end at 23:59 PST March 7.

Each PTL is elected by the Active Project Contributors for the project -
which means you have to have landed a change to the project in question
in year preceding the election. We apparently still haven't amended our
process to define a person who has only reviewed code as an Active
Contributor - so if all you've done over the past year is code review -
thank you! - but you don't get a vote.

Also, to be an APC, the Foundation bylaws also mandate that you be an
individual member of the Foundation. That means that *in order to vote*
(or propose your candidacy), you additionally have to *join the
Foundation* if you haven't done so yet. The email you specify there will
be the one used in the election process. The cut-off date for joining
the Foundation to be able to participate to this election is set to the
end of day (23:59 PST), Thursday March 7. You can do so at:

http://openstack.org/join

For those of you following at home, this means we are about to have 10
(ten) different simultaneous elections. If you aren't following at home,
the following projects are electing PTLs right now:

Ceilometer
Cinder
Glance
Heat
Horizon
Keystone
Nova
Oslo
Quantum
Swift

After that, the following will happen:

March 8-14: Vote for PTL
March 15-21: Nominations for direct seats
March 22-28: Vote for direct seats

Thanks everybody!

PS. Yes. I know, we're not supposed to cross-post - but this is
important and I'm pretty sure that there are people out there who do not
read both lists. It's also a subject I'm not expecting tons of chatter
in response to.

PPS. If any of you are wondering why this email sounds less official
than usual, ttx is eligible for a seat this term, wheras I'm not since
my term is still active - so you're stuck with me running this thing.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Name it Hood!

[Monty Taylor]

On 01/25/2013 12:54 PM, Thomas Goirand wrote:
> On Fri Jan 25 2013 06:29:32 AM CST, Monty Taylor  wrote:
>>> f) Hood is only 4 letters. Think about that when you think about typing
>>> hatfield a lot. Also, if we name it hatfield, we're going to have to
>>> have the M summit somewhere that has a town called McCoy.
> 
> Oh! I didn't realized that was the reason why the
> next summit will be in Gortland ;)

Oh dear god. PLEASE let the next summit be in a place called Gortland.

>>> g) I'll buy you a beer at the summit if you vote for Hood.
> 
> And will you also sign my PGP key? These go 2gether!

Only if you can prove to my satisfaction that you are, actually, Thomas
Goirand. :)

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Name it Hood!

[Monty Taylor]

I vote for hoodies.

On 01/25/2013 09:28 AM, Doug Hellmann wrote:
> Will we have hoodies instead of t-shirts for ODS attendees?
> 
> Doug
> 
> On Thu, Jan 24, 2013 at 2:50 PM, Monty Taylor  <mailto:mord...@inaugust.com>> wrote:
> 
> Hey all!
> 
> Here's my pitch for Hood:
> 
> a) It's the tallest mountain in Oregon, and honestly, it's a pretty
> kick-ass mountain in general
> b) Being in the pacific northwest, the mountain itself is quite
> regularly in the clouds. That's gotta count for something.
> c) It's actually a volcano.
> d) Mount Hood is CLEARLY an Oregon thing. Havana is clearly a town in
> Cuba. (We should have a design summit in cuba!!!)
> e) Harbor is super-problematic because of the US/UK clash in spelling.
> Half of us will spell it wrong no matter what.
> f) Hood is only 4 letters. Think about that when you think about typing
> hatfield a lot. Also, if we name it hatfield, we're going to have to
> have the M summit somewhere that has a town called McCoy.
> g) I'll buy you a beer at the summit if you vote for Hood.
> 
> Monty
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> <mailto:openstack@lists.launchpad.net>
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
> 
> 

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Name it Hood!

[Monty Taylor]

On 01/25/2013 07:08 AM, Matt Joyce wrote:
> I think we all know M is for Manhattan.

I will vote for whatever thing gets me a summit in NYC - although I'd
love to not have to wait until fall 2015

> On Thu, Jan 24, 2013 at 12:07 PM, Sean Dague  <mailto:sda...@linux.vnet.ibm.com>> wrote:
> 
>     On 01/24/2013 02:50 PM, Monty Taylor wrote:
> 
> Hey all!
> 
> Here's my pitch for Hood:
> 
> a) It's the tallest mountain in Oregon, and honestly, it's a pretty
> kick-ass mountain in general
> b) Being in the pacific northwest, the mountain itself is quite
> regularly in the clouds. That's gotta count for something.
> c) It's actually a volcano.
> d) Mount Hood is CLEARLY an Oregon thing. Havana is clearly a
> town in
> Cuba. (We should have a design summit in cuba!!!)
> e) Harbor is super-problematic because of the US/UK clash in
> spelling.
> Half of us will spell it wrong no matter what.
> f) Hood is only 4 letters. Think about that when you think about
> typing
> hatfield a lot. Also, if we name it hatfield, we're going to have to
> have the M summit somewhere that has a town called McCoy.
> 
> 
> Yes, but I'm totally cool with that. +1 for Hatfield.
> 
> Just means that we have to go to Florida for the M summit -
> http://en.wikipedia.org/wiki/__Fort_McCoy,_Florida
> <http://en.wikipedia.org/wiki/Fort_McCoy,_Florida>
> 
> 
> g) I'll buy you a beer at the summit if you vote for Hood.
> 
> Monty
> 
> _
> Mailing list: https://launchpad.net/~__openstack
> <https://launchpad.net/~openstack>
> Post to : openstack@lists.launchpad.net
> <mailto:openstack@lists.launchpad.net>
> Unsubscribe : https://launchpad.net/~__openstack
> <https://launchpad.net/~openstack>
> More help   : https://help.launchpad.net/__ListHelp
> <https://help.launchpad.net/ListHelp>
> 
> 
> 
> -- 
> Sean Dague
> IBM Linux Technology Center
> email: sda...@linux.vnet.ibm.com <mailto:sda...@linux.vnet.ibm.com>
> alt-email: slda...@us.ibm.com <mailto:slda...@us.ibm.com>
> 
> 
> 
> _
> Mailing list: https://launchpad.net/~__openstack
> <https://launchpad.net/~openstack>
> Post to : openstack@lists.launchpad.net
> <mailto:openstack@lists.launchpad.net>
> Unsubscribe : https://launchpad.net/~__openstack
> <https://launchpad.net/~openstack>
> More help   : https://help.launchpad.net/__ListHelp
> <https://help.launchpad.net/ListHelp>
> 
> 

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] Name it Hood!

[Monty Taylor]

Hey all!

Here's my pitch for Hood:

a) It's the tallest mountain in Oregon, and honestly, it's a pretty
kick-ass mountain in general
b) Being in the pacific northwest, the mountain itself is quite
regularly in the clouds. That's gotta count for something.
c) It's actually a volcano.
d) Mount Hood is CLEARLY an Oregon thing. Havana is clearly a town in
Cuba. (We should have a design summit in cuba!!!)
e) Harbor is super-problematic because of the US/UK clash in spelling.
Half of us will spell it wrong no matter what.
f) Hood is only 4 letters. Think about that when you think about typing
hatfield a lot. Also, if we name it hatfield, we're going to have to
have the M summit somewhere that has a town called McCoy.
g) I'll buy you a beer at the summit if you vote for Hood.

Monty

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] TC Candidacy

[Monty Taylor]

Hi everybody!

I'd like to run for a seat on the Technical Committee.

- OpenStack Experience -

I run the CI and Developer Automation team for OpenStack. There hasn't
always been a team, but as long as there has been, I've been doing it.
Before there was a proper team, there was Soren and I, and before that
there was just me. I set up the original Tarmac-based trunk gate that we
used before Gerrit, and I'm the original owner on many of the Launchpad
resources (because it turns out some human must be the ultimate owner of
things there) So it's pretty easy to say without boasting that there is
very little about the tooling that runs the project that I don't know
about, both in its current form and the history that got us to its
current form.

My commits and reviews can be seen here:
https://review.openstack.org/#/q/reviewer:mordred%2540inaugust.com+OR+owner:mordred%2540inaugust.com,n,z

- Other Experience -

I work at HP where I lead a team of people who staff the rest of the CI
team. I do what I can as part of my job there to ensure that the
OpenStack Dev systems always have the resources they need to operate
effectively. Before HP I worked at Rackspace doing the same thing.

Before OpenStack at Rackspace I was a core developer on Drizzle, having
moved with the rest of the Drizzle team to Rackspace when Sun got bought
by Oracle. I was lucky enough to get involved in Drizzle hacking from
the beginning of that project while I was a Senior Consultant for MySQL
Inc (and then for Sun when we got bought) Although it doesn't come up in
this context very often, I'm pretty stinking good at MySQL Scaling and
High Availability. I've been a Python hacker since I wrote the SNPP
protocol library for Python in 2000, and have experience as both a
developer and a *nix sysadmin stretching back to 1994.

Recently I became a member of the Python Software Foundation, and I sit
on the OpenStack Foundation Board.

- Why I should be on the Technical Committee -

The TC provides direction to the project as a whole and acts as a
collaboration point and focus for consensus between the projects. Often
times things that are decided by the TC have a direct effect on the
automation systems that we use to run things ... so having someone from
the CI and Automation teams represented seems like a really good idea.
Additionally, since I tend to be cross-project focused rather than
involved in any one specific project, I'd like to think that I have a
decent unbiased perspective on issues that are related to project
agreement and consistency.

Thanks for your consideration!
Monty

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Bug 1030646] [NEW] Devstack vs. Horizon client versions

[Monty Taylor]

On 07/29/2012 07:47 PM, Monty Taylor wrote:
> Hey!
> 
> This might be fairly tricky to fix properly given the design of the
> devstack gate. I could be wrong, it might not be terrible now that we
> can release new client lib versions to PyPI more quickly. The devstack
> gate explicitly tests proposed change to trunk of one project against
> tip of trunk across all the other projects. That's great for the dev
> purpose, and can solidify through sequencing during the
> milestone-proposed period, but if we have people with standalone horizon
> installations that are tracking trunk more closely, we might need to do
> some more thinking.

Ok. jeblair and I chatted about this some last night in prep, and it
might not be as tricky as I was thinking. In fact, we're already doing
some testing that you could take advantage of to prevent the kind of
breakage you're talking about, and a few things that are new enough that
we haven't really taken advantage of them yet. Let me 'splain:

The devstack gate works as I mentioned above, and will test you against
trunk of other branches. That's important, because that's how we ensure
that everything works together as we're developing on it.

However, we also do unittest runs, which will test the project in
question using the specific dependency versions the project declares.
Those tests for horizon will use the released versions of all of the
client libs, and would be the automated trap against patches to horizon
that depend on changes that have not yet been released to pypi from
client libs.

Of course, horizon is tricky, being a django app, and unittests
themselves don't always tell the full story. We have just in the last
week added selenium test running (which you know, but for completeness
I'm including here) Those tests also will run with released versions of
libs from pypi.

Also important to note here is that we have just recently gotten
everything hooked up properly so that the client libs can release to
pypi whenever they need to - so while we haven't started taking
advantage of this fully yet, I think as people add functionality to the
client libs, we may not have to wait as long for it to land.

Anyhow - you can also totally just not accept patches to horizon that
use things that aren't in released versions via code review - but if
it's helpful, we are running a set of tests against horizon and the
declared/released depends, so if you can figure out some sensible tests
to add to unit or selenium tests that would trip this, they will
definitely get run.

We'll still be there at the CI meeting of course, but I wanted to set
the stage a little bit so that we didn't have to cover all that in IRC.


> Could I request that you drop by the CI meeting on Tuesday so we can
> chat about it interactively? I think this is important and that we
> should do everything we can to get this right.
> 
> Monty
> 
> On 07/29/2012 06:34 PM, Gabriel Hurley wrote:
>> Public bug reported:
>>
>> We patched this bug: https://bugs.launchpad.net/horizon/+bug/1027210
>>
>> However, that fix worked for devstack and broke stand-alone
>> installations of horizon because devstack pulled the clients from Github
>> while Horizon's requirements file pulls them from PyPI. This is no good.
>>
>> Now anytime you try to list out images from glance (except when using
>> devstack) you get this error:
>>
>> list() got an unexpected keyword argument 'page_size'
>>
>> We need to reconcile devstack with Horizon, and in the future Horizon
>> will *only* accept changes that work with *released* client versions.
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
> 

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Bug 1030646] [NEW] Devstack vs. Horizon client versions

[Monty Taylor]

Hey!

This might be fairly tricky to fix properly given the design of the
devstack gate. I could be wrong, it might not be terrible now that we
can release new client lib versions to PyPI more quickly. The devstack
gate explicitly tests proposed change to trunk of one project against
tip of trunk across all the other projects. That's great for the dev
purpose, and can solidify through sequencing during the
milestone-proposed period, but if we have people with standalone horizon
installations that are tracking trunk more closely, we might need to do
some more thinking.

Could I request that you drop by the CI meeting on Tuesday so we can
chat about it interactively? I think this is important and that we
should do everything we can to get this right.

Monty

On 07/29/2012 06:34 PM, Gabriel Hurley wrote:
> Public bug reported:
> 
> We patched this bug: https://bugs.launchpad.net/horizon/+bug/1027210
> 
> However, that fix worked for devstack and broke stand-alone
> installations of horizon because devstack pulled the clients from Github
> while Horizon's requirements file pulls them from PyPI. This is no good.
> 
> Now anytime you try to list out images from glance (except when using
> devstack) you get this error:
> 
> list() got an unexpected keyword argument 'page_size'
> 
> We need to reconcile devstack with Horizon, and in the future Horizon
> will *only* accept changes that work with *released* client versions.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Openstack JAVA SDK

[Monty Taylor]

Hi!

You probably want to check out jclouds as well. It's extremely mature,
is in production all over the world and has support for openstack
compute and storage.

http://www.jclouds.org/

Monty

On 07/18/2012 08:50 AM, Jyothsna Padavala wrote:
> Thanks Vincent for your reply.
> 
> I lean towards using java cloud-files too since my primary work would be 
> involving swift.
> How did you build the java cloud-files code? My existing Java application is 
> done on Netbeans IDE and i'm bound to use that. Do we need any extra build 
> tool like Ant / Maven etc?
> 
> Thanks again,
> Jyothsna.
> 
> - Original Message -
> From: "Sheng Bo Hou" 
> To: "Jyothsna Padavala" 
> Cc: openstack@lists.launchpad.net
> Sent: Wednesday, July 18, 2012 2:28:30 AM (GMT-0800) America/Los_Angeles
> Subject: Re: [Openstack] Openstack JAVA SDK
> 
> Hi Jyothsna, 
> 
> I think they are different java codes for the client sides. 
> I was using java cloud-files for accessing the swift. 
> Open-java-sdk can be used as the client side of nova. 
> 
> And I used the source code directly. 
> 
> Best wishes. 
> Vincent Hou (侯胜博) 
> 
> Software Engineer, Standards Growth Team, Emerging Technology Institute, IBM 
> China Software Development Lab 
> 
> Tel: 86-10- 82450778 Fax: 86-10- 82453660 
> Notes ID: Sheng Bo Hou/China/IBM@IBMCN E-mail: sb...@cn.ibm.com 
> Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West 
> Road, Haidian District, Beijing, P.R.C. 100193 
> 
> 
>   Jyothsna Padavala < jyothsna.padav...@strongauth.com > 
> Sent by: openstack-bounces+sbhou= cn.ibm@lists.launchpad.net 
> 
> 2012-07-18 06:09  
> Toopenstack < openstack@lists.launchpad.net > 
>   
> cc
>   
> Subject   [Openstack] Openstack JAVA SDK 
>   
> 
> 
> 
> Hello all, 
> 
> I've the openstack (only keystone and swift)setup ready. Now, i want to talk 
> to this setup from my java application. When i tried to google for java sdk 
> for openstack, i got two options. 
> 
> 1) "java cloud-files" from rackspace ( 
> https://github.com/rackspace/java-cloudfiles ) 
> 2) Java SDK from ( https://github.com/woorea/openstack-java-sdk ) 
> 
> My question is which one is reliable and easy to use. How do i go about 
> installing them? 
> 
> Thanks much, 
> Jyothsna 
> 
> ___ 
> Mailing list: https://launchpad.net/~openstack 
> Post to : openstack@lists.launchpad.net 
> Unsubscribe : https://launchpad.net/~openstack 
> More help : https://help.launchpad.net/ListHelp 
> 
> 
> 
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
> 


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Release Upgrades (was [nova] [cinder] Nova-volume vs. Cinder in Folsom)

[Monty Taylor]

On 07/12/2012 04:36 PM, Vishvananda Ishaya wrote:
> 
> On Jul 12, 2012, at 2:22 PM, Narayan Desai wrote:
> 
>> I think that the long term maintenance or removal of nova-volume in
>> its existing form is orthogonal to the actual issue of continuity from
>> one release to the next.
> 
> Agreed. Discussion the volume/cinder strategy is a separate topic. I've
> taken the liberty of updating the subject to keep the discussions on point
> 
>>
>> At this point, we've now run cactus, diablo and are in testing with
>> essex. Each of these has effectively been a flag day for us; we build
>> the new system, migrate users, images, etc, and let users do a bunch
>> of manual migration of volume data, etc, while running both systems in
>> parallel. This hasn't been as painful as it sounds because our
>> understanding of best practices for running openstack is moving pretty
>> quickly and each system has been quite different from the previous.
> 
> Upgrading has been painful and we are striving to improve this process
> as much as possible.
> 
>>
>> The lack of an effective process to move from one major release to the
>> next is the major issue here in my mind. It would be fantastic if
>> (some day, ha ha ha) you could apt-get upgrade from folsom to grizzly,
>> but i think that is likely to be more trouble than it is worth. A
>> reasonable compromise would be a well documented process as well as
>> tools to aid in the process. Each real deployment will have a
>> substantial set of local customizations, particularly if they are
>> running at any sort of scale. While it won't be feasible to support
>> any upgrade with these customizations, tools for the process (which
>> may only be used a straw man in complex cases) would go a long way.
> 
> I would like to take this a bit further. Documentation is a great first step,
> but I would actually like to have an actual Jenkins test that does the upgrade
> from essex to Folsom with live resources created. I think this the only way
> that we can be sure that the upgrade is working properly.

++

> The first version of this doesn't even have to be on a full cluster. I'm 
> thinking
> something as simple as:
> 
> * configure devstack to checkout stable/essex from all of the projects
> * run the system, launch some instances and volumes
> * terminate the workers
> * upgrade all of the code to folsom
> * do any manual upgrade steps (nova-manage db sync, cinder migrate, etc.)
> * run all the workers
> * make sure the existing data still works and new api commands run
> 
> The manual upgrade steps should be contained in a single script so that the
> distress can use it to help make their package upgrades and deployers can
> use it for reference when upgrading their clusters.

Yes - especially if it's a self contained thing like devstack is currently.

For the "upgrade all the code to folsom" step - let's chat about making
sure that we get the right hooks/env vars in there so that we can make
that "upgrade to tip of trunk in most projects + proposed patch in one
of them"  - same as we do for devstack runs today.

> This is something we can start working on today and we can run after each
> commit. Then we will immediately know if we do something that breaks
> upgradability, and we will have a testable documented process for upgrading.

The creation of the self-contained script devstack was a HUGE step
forward for us for integration testing. I think a similar thing for
upgradability would similarly be huge.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [nova] mysql connection optimization

[Monty Taylor]

On 07/11/2012 12:33 PM, Devananda van der Veen wrote:
> Hi all,
> 
> I've been taking a look at the way Nova uses its MySQL database. Having
> done MySQL performance audits for years as a consultant, a few things
> jumped out right away at me. First is the way that SQLAlchemy is
> wrapping nearly every query in an unnecessary "ping check" and rollback,
> eg.:
> 
> select 1;
> select ... from ... where ...;
> rollback;
> select 1;
> update ... where ...;
> commit;
> rollback;
> 
> 
> You can find a complete sample here: http://paste.openstack.org/show/18731/ 
> 
> I think I understand the reason for both the "select 1" and the
> "rollback" statements. However, in the interest of performance for
> large-scale deployments, I feel pretty strongly that they need to go away.
> 
> As I see it, there are three factors here.
> 
> (I) Most of the code in db/sqlalchemy/api.py abstracts a "unit of work"
> to a very low level, generally a single database read or write, and does
> not currently support transactions spanning multiple consistent writes.
> This is why "select 1" and "rollback" appear around almost every query
> -- most functions in api.py is checking out a session object, doing one
> or two queries, and then releasing the session. This actually creates a
> much larger issue about transactional atomicity for larger operations,
> such as the open bug about network creation here, and is probably better
> for another discussion. 
> https://bugs.launchpad.net/nova/+bug/755138.
> 
> (II) Connections are tested by the MySQLPingListener function, using
> "SELECT 1" statements, every time a connection / session is checked out
> of the pool. Since connections are usually OK, this adds overhead
> unnecessarily. It would be more efficient to handle the errors when
> connections aren't OK. I've opened a bug with a description of one
> possible way to fix this issue. There are probably other viable
> solutions as well.
> https://bugs..launchpad.net/nova/+bug/1007027
> 
> 
> My understanding is that this was implemented to prevent "Database has
> gone away" errors from occurring every time that the database is
> restarted, or when connections are closed by the database after being
> idle for long periods. In my opinion, a better solution to these
> problems is to:
> * wrap queries in retry logic, which will catch disconnect errors and
> attempt to reconnect to the database. I have a patch for this, if folks
> are interested.
> * set Nova's sql_idle_timeout to less than MySQL's wait_timeout, and set
> them both to a reasonably short interval (eg, 1 minute), rather than the
> default (which I think is 8 hours).
> 
> (III) Transaction state is reset when connections are returned to the
> pool, even if no transaction was in progress, or the
> transaction-in-progress already committed. This is completely wasteful,
> and easy to disable.
> https://bugs.launchpad.net/nova/+bug/1007038
> 
> Caveat here is that this reset-on-return functionality *is* useful when
> code doesn't properly clean up its own transaction state. When I turned
> it off, it exposed a few bugs, which I haven't tracked down yet.
> Lowering the sql_idle_timeout will provide the same "solution" as the
> current reset-on-return behavior in that it will prevent long-running
> idle transactions that tie up resources unnecessarily.
> 
> 
> In summary, I'd like to see Nova stop spamming the database with "SELECT
> 1" and  "ROLLBACK", and think this should be pretty easy to do. Testing
> these two changes in my devstack seems to work fine, but I'd like to
> know what others think before I propose a changeset with this kind of
> potential impact.

I agree and welcome work in that direction. Should we be looking at
patches to sqlalchemy to increase its support for dealing appropriately
with mysql-gone-away errors?

Monty

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [CHEF] Clarification on osops/chef-repo/roles/nova-compute.rb

[Monty Taylor]

On 07/11/2012 10:04 AM, Matt Ray wrote:
> I know Rackspace gates their commits with foodcritic
> (https://rubygems.org/gems/foodcritic), which is becoming the standard
> we're recommending to people for cookbooks style. For doing Travis CI
> testing automatically, I plan on adding testing outlined in this post:
> http://nathenharvey.com/blog/2012/05/29/mvt-foodcritic-and-travis-ci/

Great. I imagine running foodcritic on a chef repo owned by openstack
would not be hard to do if that's something we want to have/own/do. (The
Travis CI stuff is obviously not very interesting to me, but I suppose
other people read this list who are not me. weird. :) )

> Opscode also has a "test-kitchen" project in progress that will be
> running CI with Jenkins on a number of platforms automatically for
> community cookbooks, we'll be pushing the OpenStack cookbooks on that
> as well as soon as it's made public and open sourced.

Sweet.

Let me know if there are any things that people are wanting related to
any of these projects from the OpenStack CI infrastructure.
Foodcritic/jsonlint seem pretty easy - deployments on to bare nodes
using the chef stuff similar to our devstack-based installs might take a
little more work and would need to be planned for. :)

> Thanks,
> Matt Ray
> Senior Technical Evangelist | Opscode Inc.
> m...@opscode.com | (512) 731-2218
> Twitter, IRC, GitHub: mattray
> 
> 
> On Wed, Jul 11, 2012 at 9:39 AM, Sullivan, Jon Paul
>  wrote:
>>> -Original Message-
>>> From: openstack-bounces+jonpaul.sullivan=hp@lists.launchpad.net
>>> [mailto:openstack-bounces+jonpaul.sullivan=hp@lists.launchpad.net]
>>> On Behalf Of Monty Taylor
>>> Sent: 11 July 2012 14:37
>>> To: Matt Ray
>>> Cc: Nati Ueno; openstack@lists.launchpad.net
>>> Subject: Re: [Openstack] [CHEF] Clarification on osops/chef-
>>> repo/roles/nova-compute.rb
>>>
>>>
>>>
>>> On 07/10/2012 04:23 PM, Matt Ray wrote:
>>>> Bluntness appreciated, this process is already in motion.
>>>> http://opscode.com/openstack was launched 2 weeks ago and I promptly
>>>> left for conferences and vacation. I am consolidating GitHub repos
>>>> here:
>>>
>>> Awesome.
>>>
>>>> https://github.com/opscode/openstack-chef-repo
>>>> https://github.com/opscode-cookbooks/nova
>>>> https://github.com/opscode-cookbooks/glance
>>>> https://github.com/opscode-cookbooks/horizon
>>>> https://github.com/opscode-cookbooks/keystone
>>>> https://github.com/opscode-cookbooks/swift
>>>>
>>>> Work is being done in my own repos until it's ready for an initial
>>>> release, which will include a Getting Started with Chef and OpenStack
>>>> document.
>>>> https://github.com/mattray/
>>>>
>>>> I'm working with quite a few folks already, Rackspace, Dell, DreamHost
>>>> and others and Intel is sponsoring this work.
>>>>
>>>> Jay and I chatted a bit in IRC, we're quite aligned in how we plan on
>>>> working this and the goal will be to get
>>>> github.com/openstack/chef-repo gated with Gerrit and CI and pulling
>>>> from Opscode's repos soon.
>>>
>>> Only really replying because I saw the word gated. :) I'd love to be
>>> part of any conversations that are being had on this subject, sooner
>>> rather than later.
>>
>> Our standard gating tests for chef code are to run "jsonlint" on all json 
>> files, "knife cookbook test" on all cookbooks, and then running all chefspec 
>> tests for every cookbook via rspec.
>>
>> Suggestions of extra tests that would be worthwhile gratefully received ;-)
>>
>>>
>>>> Please feel free to join the discussion on our new mailing list
>>>> focused on this effort here:
>>>> http://groups.google.com/group/opscode-chef-openstack
>>>>
>>>> And an IRC channel:
>>>> #openstack-chef on irc.freenode.net
>>>>
>>>> Thanks,
>>>> Matt Ray
>>>> Senior Technical Evangelist | Opscode Inc.
>>>> m...@opscode.com | (512) 731-2218
>>>> Twitter, IRC, GitHub: mattray
>>>>
>>>>
>>>> On Tue, Jul 10, 2012 at 11:25 AM, Jay Pipes 
>>> wrote:
>>>>> Apologies in advance for my blunt and somewhat dour response, Matt.
>>>>> I'm not singling you out at all, and I know you've tried your best to
>>>>> get the various C

Re: [Openstack] [CHEF] Clarification on osops/chef-repo/roles/nova-compute.rb

[Monty Taylor]

On 07/10/2012 04:23 PM, Matt Ray wrote:
> Bluntness appreciated, this process is already in motion.
> http://opscode.com/openstack was launched 2 weeks ago and I promptly
> left for conferences and vacation. I am consolidating GitHub repos
> here:

Awesome.

> https://github.com/opscode/openstack-chef-repo
> https://github.com/opscode-cookbooks/nova
> https://github.com/opscode-cookbooks/glance
> https://github.com/opscode-cookbooks/horizon
> https://github.com/opscode-cookbooks/keystone
> https://github.com/opscode-cookbooks/swift
> 
> Work is being done in my own repos until it's ready for an initial
> release, which will include a Getting Started with Chef and OpenStack
> document.
> https://github.com/mattray/
> 
> I'm working with quite a few folks already, Rackspace, Dell, DreamHost
> and others and Intel is sponsoring this work.
> 
> Jay and I chatted a bit in IRC, we're quite aligned in how we plan on
> working this and the goal will be to get
> github.com/openstack/chef-repo gated with Gerrit and CI and pulling
> from Opscode's repos soon.

Only really replying because I saw the word gated. :) I'd love to be
part of any conversations that are being had on this subject, sooner
rather than later.

> Please feel free to join the discussion on
> our new mailing list focused on this effort here:
> http://groups.google.com/group/opscode-chef-openstack
>
> And an IRC channel:
> #openstack-chef on irc.freenode.net
> 
> Thanks,
> Matt Ray
> Senior Technical Evangelist | Opscode Inc.
> m...@opscode.com | (512) 731-2218
> Twitter, IRC, GitHub: mattray
> 
> 
> On Tue, Jul 10, 2012 at 11:25 AM, Jay Pipes  wrote:
>> Apologies in advance for my blunt and somewhat dour response, Matt. I'm
>> not singling you out at all, and I know you've tried your best to get
>> the various Chef stakeholders to work together. Also apologies for
>> top-posting, but there's not a whole lot of use inline posting this.
>>
>> tl;dr
>> -
>>
>> We need to stop the needless fracturing of the operational knowledge of
>> the Chef community and try working as a team towards some common goals
>> instead of creating fork after fork of repos of Chef cookbooks.
>>
>> There is a ton of wasted effort in this area.
>>
>> Proposal:
>>
>> * Get our act together and treat Chef repos (and puppet/juju/whatever)
>> as we do other OpenStack core and supporting projects -- use Gerrit, use
>> a CI gating system, do real code reviews on it, and in general treat
>> them as a supporting OpenStack project
>> * Mark ALL Chef repos that are not currently maintained with an OBSELETE
>> marker and/or DELETE the repo on Github
>> * Consolidate all *cookbooks* into a repository in
>> github.com/openstack/chef-repo
>> * Use git submodules to manage cookbooks that are upstreamed in
>> github.com/opscode/ that have little to no changes in them
>> * Actually fix the documentation of the dang cookbooks -- right now,
>> half of them include the documentation from the memcache cookbook, as
>> they were lazily copy-pasted around, or the standard example doc that is
>> created when using something like knife.
>> * Put as much variation in deployment philosophy into *roles* and
>> attribute overrides/defaults
>>
>> More/Rant/Details
>> -
>>
>> Maybe it's just the open source developer in me, but I don't understand
>> why there is such an aversion to coordination in the deployment/ops
>> community around the scripts and deployment
>> cookbooks/modules/charms/whatever.
>>
>> Is it that everyone has a different idea of what is best? Is it because
>> deployers/ops folks think that coordinating with other contributors is
>> too time-consuming? Is it because the chef repos are not controlled in
>> the same way as, say, devstack or the core projects, with an automated
>> patch queue manager and code review system that actually encourages
>> debate over patches? A combination of all of the above?
>>
>> Over the last 2 years, I've worked at 3 companies in the OpenStack
>> ecosystem. All three companies had their own repos of Chef cookbooks
>> (still do to this day). 50-60% of the content of these cookbooks is
>> identical. 10-20% of the content of these cookbooks is different -- but
>> only slightly or cosmetically. And a good portion of the remaining
>> 20-40% are differences that are incorrectly (IMHO) placed in the
>> cookbooks and recipes instead of where they should be: in roles and
>> environments, with cookbooks created that deal with variations in
>> deployments with attributes and the occasional if/else block.
>>
>> In trying to determine the appropriate Chef repo to use for the TryStack
>> project, we found the following repo:
>>
>> https://github.com/osops/chef-repo
>>
>> to have the most up-to-date. I've since been told this repo is no longer
>> maintained. This is very frustrating, not because of this particular
>> repo, but because this is just one in a long line of neglected and
>> forgotten forks of chef cookbook repositories. The fact that the d

Re: [Openstack] OpenStack "G" naming poll

[Monty Taylor]

On 07/03/2012 07:33 PM, James E. Blair wrote:
> Brian Waldon  writes:
> 
>> TL;DR - Screw the rules, let's call the next release 'Grizzly'
>>
>> As California is rather lacking in the 'municipality names starting
>> with a G that we should use for an OpenStack release' department, I
>> have had to look *slightly* outside the ruleset to find a suitable 'G'
>> release name - that name being 'Grizzly'. The rules clearly state that
>> a release name must represent a city or county near the corresponding
>> design summit and be comprised of a single word of ten characters or
>> less - the problem here being that 'Grizzly' is actually 'Grizzly
>> Flats.' Having already polled a small subset of the community, I feel
>> like there would be enough support for 'Grizzly' to win if it were on
>> the ballot. As I'm more interested in selecting a suitable name than
>> accurately representing some arbitrary territory, I'd love to either
>> permanently amend the rules to make this acceptable or grant an
>> exception in this one case. As Thierry said, if this reaches critical
>> mass, we will figure out what to do. Otherwise, I'll shut up and deal
>> with 'Gazelle'.
> 
> I will join your Bear Flag Revolt. 

Best. Picture. Ever. We have a G winner... and honestly, if that's not
the t-shirt then I'm going to have my own bear flag revolt.

Grizzly FTW

> 
> We could amend the rules to add official symbols of the territory in
> question.  Despite being one of the most recognized symbols of
> California, named the state animal, and appearing on the state flag, the
> Grizzly bear (Ursus californicus) has been extinct here since 1922.
> 
> [1] http://en.wikipedia.org/wiki/California_Republic#Bear_Flag_Revolt
> 
> -Jim
> Though as a Firesign Theatre fan, I like Goshen too.
> 
> 
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
> 


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] best practices for merging common into specific projects

[Monty Taylor]

Interestingly enough - gerrit supports submodules pretty well... and it
does exactly what Eric said below ... if both the project and
superproject are in gerrit, and a change is made to the project, gerrit
can automatically update the superproject reference.

Here's the thing though (and one of the reasons we haven't actually
earnestly suggested using this for anything yet) ... testing

If you propose a change to openstack-common and we were going to use the
auto-update feature, we'd need to test that that change doesn't break
ALL of the projects that use it. Now, what if nova is going to need a
patch to use the new version of the code. Oops. Complexity. Overload.
Everyone dies.

However, with a versioned library model, the projects can consume things
pinned to specific versions, and then they can submit a change that
updates the version depend and the code which needs to be updated to
support the version change, and that change can be atomic.

So honestly, I'd say the real key is getting us closer to the point
where openstack-common is a proper library, because all of the rest of
the complexity is stuff we're inventing to make life harder on
ourselves, when the standard library with api-contract and a version
model of the world works pretty fine without needing coordinated changes
across multiple repositories.

On 07/03/2012 06:54 PM, Gabriel Hurley wrote:
> I’m pretty -1 on triggering changes in other projects from common.
> That’s gonna result in a broken code (whether subtle or obvious) no
> matter how good your gates are.
> 
>  
> 
> At least as an external library you can freeze a version requirement
> until such time as you see fit to properly updated that code and
> **ensure** compatibility in your project.
> 
>  
> 
> Or if your project likes ridin’ trunk, then don’t pin a version and
> you’ve got the same effect as an automatic trigger.
> 
>  
> 
> -  Gabriel
> 
>  
> 
> *From:*openstack-bounces+gabriel.hurley=nebula@lists.launchpad.net
> [mailto:openstack-bounces+gabriel.hurley=nebula@lists.launchpad.net]
> *On Behalf Of *Andrew Bogott
> *Sent:* Tuesday, July 03, 2012 3:54 PM
> *To:* Eric Windisch; openstack@lists.launchpad.net
> *Subject:* Re: [Openstack] best practices for merging common into
> specific projects
> 
>  
> 
> On 7/3/12 5:47 PM, Eric Windisch wrote:
> 
> git submodules don't have to be linked to the head of a branch. Instead
> of double-commiting (for every commit), we can do a single commit in
> each project to change the git reference of the submodule. This would
> not be too far from the existing behavior, except that it would minimize
> the double commits.
> 
>  
> 
> Oh, I guess I left out an important part of my vision, which is that
> there would be a commit hook in common which moves the submodule
> reference in the parent projects anytime a patch is merged in common. 
> So, in short: once a patch passed review for inclusion in common, that
> patch would automatically go live in all other project heads simultaneously.
> 
> 
> -- 
> Eric Windisch
> 
>  
> 
> On Tuesday, July 3, 2012 at 15:47 PM, Andrew Bogott wrote:
> 
> On 7/3/12 1:59 PM, Gabriel Hurley wrote:
> 
> The notion that copying code is any protection against APIs that
> may change is a red herring. It's the exact same effect as
> pegging a version of a dependency (whether it's a commit hash or
> a real version number), except now you have code duplication. An
> unstable upgrade path is an unstable upgrade path, and copying
> the code into the project doesn't alleviate the pain for the
> project if the upstream library decides to change its APIs.
> 
>  
> 
> Also, we're really calling something used by more or less all
> the core projects "incubated"? ;-) Seems like it's past the
> proof-of-concept phase now, at least for many parts of common.
> Questions of API stability are an issue unto themselves.
> 
>  
> 
> Anyhow, I'm +1 on turning it into a real library of its own, as
> a couple people suggested already.
> 
>  
> 
> - Gabriel
> 
>  
> 
> I feel like I should speak up since I started this fight in the first
> 
> place :)
> 
>  
> 
> Like most people in this thread, I too long for an end to the weird
> 
> double-commit process that we're using now. So I'm happy to set aside
> 
> my original Best Practices proposal until there's some consensus
> 
> regarding how much longer we're going to use that process. Presumably
> 
> opinions about how to handle merge-from-common commits will vary in the
> 
> meantime, but that's something we can live with.
> 
>  
> 
> In terms of promoting common into a real project, though, I want to
> 
> raise another option that's guaranteed to be unpopular: We make
> 
> openstack-common a git-submodule that is automatically checked out
> 
> within the directory tree of

Re: [Openstack] Single global dependency list

[Monty Taylor]

On 07/03/2012 04:23 PM, Gabriel Hurley wrote:
> Agreed on all points, and I know you're not evil, Monty. ;-)
> (mostly)

Dammit. I'm going to HAVE to try harder... see my other post about
Bulgarian bars with freezers...

> You're totally right that this particular case won't stymie new
> contributors, but as we've seen for changes to common--and sometimes
> even to the client libraries or devstack--reviewers are in short
> supply and getting the change you need in one of the "gate" projects
> merged can often add days of impedance to otherwise fruitful work.
> It's bitten me plenty of times.

Yes. It's super hard and frustrating- especially as someone who rarely
makes patches to the projects themselves UNLESS it's something happening
through openstack-common ... or something that just broke between
devstack and our infrastructure.

> So the need for balance is critical. Being able to vet the impact of
> a change on every project consuming it is difficult for either
> automated systems or human reviewers, so we do our best.
> 
> Perhaps the simplest answer for now is devising a reasonable set of
> automated gate tests for this "os-requires " module that humans can
> trust, and working to expand the circle of reviewers on these
> centralized projects that have the power to block everyone yet are so
> easy to ignore...

TOTALLY agree - especially on expanding the circle of reviewers.

> All the best,
> 
> - Gabriel
> 
>> -Original Message- From: Monty Taylor
>> [mailto:mord...@inaugust.com] Sent: Tuesday, July 03, 2012 12:49
>> PM To: Gabriel Hurley Cc: Eric Windisch;
>> openstack@lists.launchpad.net Subject: Re: [Openstack] Single
>> global dependency list
>> 
>> It's a good and valid question and I don't really know. In this
>> case, I'm merely the pack-horse who was told "global synchronized
>> dependencies lists!" (not that I'm not the evil person cooking up
>> schemes)
>> 
>> That said - most patches from new contributors don't actually come
>> with new library dependencies. If they are adding a new depend, I
>> think it's reasonable to expect it to be slightly harder to get
>> that landed.
>> 
>> I do think that we need an answer to "who approves changes to this
>> list". Getting stuff merged to openstack-common is often hard
>> because it's a smaller list of people who work on it. I'd hate to
>> see this be only PTLs. However, things like "let's upgrade webob"
>> seem to _actually_ need more eyes than it seems like at the time.
>> 
>> meh.
>> 
>> On 07/03/2012 03:12 PM, Gabriel Hurley wrote:
>>> So, I understand the rationales, and I think of those three
>>> options the one
>> chosen is the most reasonable. I'm gonna just come out and say I
>> hate this whole idea, but let's set this aside for a minute 'cuz I
>> have a genuine question:
>>> 
>>> What will the process be for merging changes to this requirements
>>> list?
>> Having yet another roadblock to getting your contribution merged is
>> a huge developer disincentive. We're really making it exceptionally
>> hard for new contributors, and frustrating even for the old hands.
>>> 
>>> So, with the goal of making the coordinated management of the
>>> projects
>> possible, what can we do to respect developers?
>>> 
>>> - Gabriel
>>> 
>>>> -Original Message- From: openstack-
>> bounces+gabriel.hurley=nebula@lists.launchpad.net
>>>> [mailto:openstack- 
>>>> bounces+gabriel.hurley=nebula@lists.launchpad.net] On
>>>> Behalf Of Monty Taylor Sent: Tuesday, July 03, 2012 7:54 AM To:
>>>> Eric Windisch Cc: openstack@lists.launchpad.net Subject: Re:
>>>> [Openstack] Single global dependency list
>>>> 
>>>> 
>>>> 
>>>> On 07/03/2012 10:09 AM, Eric Windisch wrote:
>>>>> I have to agree with others that copying files around is not
>>>>> ideal, and I can see the maintenance of this getting more
>>>>> involved as Nova becomes more coupled with common.
>>>>> 
>>>>>>> Additionally, we'd make the copy only copy in the
>>>>>>> versions from openstack-common for package that were
>>>>>>> already listed in the target project, so that we wouldn't
>>>>>>> add django to python-swiftclient, for instance.
>>>>> 
>>>>> This seems to be a reasonab

Re: [Openstack] OpenStack "G" naming poll

[Monty Taylor]

On 07/03/2012 07:29 PM, Brian Waldon wrote:
> 
> On Jul 3, 2012, at 5:21 PM, Monty Taylor wrote:
> 
>> tl;dr - Screw the rules, I agree
>>
>> Let's at least add it to the poll.
>>
>> Also - I think we should further amend the rules such that we select the
>> NEXT release by the summit for the current release. That means two things:
>>
>> At the g summit, we'd tell everyone where the next summit is:
>> At the g summit, we'd vote and announce the name of h
>> We wouldn't have to spend half the cycle saying "h, or whatever" when we
>> mean "we're going to defer that crazy idea until next time"
>> I wouldn't have had to use the letter g by itself twice just above here.
> 
> Fantastic idea. 
> 
> I haven't been involved in choosing the next location, so I'm not sure how 
> hard it would be to choose it that far in advance. Maybe somebody can comment 
> on how doable this is?

I actually think it's HARDER to not choose it that far in advance,
because the longer you wait, the more places are booked already.
Although it's not exactly the same - linux conf australia always
announces the location of the next conference at the current conference
- and those are yearly. Also, they have a rotating set of host teams...
so basically a team from a town puts in a proposal to linux.au saying
"we'd like to host the next one, here's what we're going to do, here's
where we're going to have it, blah blah blah" - and then one of them
gets selected, and then they're the poor sods that have to actually run
the darned thing.

So to stick my nose WAY in where it doesn't belong, once we have the
foundation - what if we move to a model of having folks propose that
they'd like to host the design summit? We can make a CFP-style deadline,
and people can all pitch their location, and one can get chosen and
announced by Thierry at a closing session or whatnot. That way if I
wanted to get together with some other folks and say "hey guys, I've got
10 rooms that NYU has donated, and I'll provide these facilities" - then
great, or if mercado libre wanted to say "zomg - we totally going to
bring you all to the Shearton WTC in Sao Paulo" - or NTT was all "dude,
we've got a thousand rooms in the middle of Tokyo" or Rackspace went "I
don't know if anybody noticed, but we bought a shopping mall a few years
ago and it's got conference rooms" ... the folks can sit in a room, look
at the proposals, say things like "wow, I really don't want to sit in
the castle all week when I sit there all week anyway - but drinking with
Monty in the Lower East Side at a bar that has freezer they'll lock you
in with a bottle of vodka sounds like a great way to plan the Houston
release" (that's how-ston for all you Texans) -- it would almost be like
distributed development with code review.

Of course, if that flys I guess I'm going to have to figure out how to
take everyone to Mehanata...

Monty

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] coding standards (was: review for implement dhcp agent for quantum)

[Monty Taylor]

On 07/03/2012 05:07 PM, Duncan McGreggor wrote:
> On Tue, Jul 3, 2012 at 5:39 PM, Dan Wendlandt  wrote:
>> Lately, Quantum reviewers have been doing their best to enforce python style
>> guidelines above and beyond the programmatically enforced pep8 checks.  This
>> has happened for many recent reviews, so Mark isn't being singled out here,
> 
> My objection isn't to Mark being singled-out -- my objection is to
> *anyone* engaging in this level of nit-pickery. This is death to
> projects.
> 
> This is coming from a guy who's incredibly anal about his code and
> coding standards, too. I've been coding in Python for over a decade,
> adhering to PEP8 for a considerable period of that time, am a member
> of the notoriously picky Twisted project, and even I was surprised by
> the flood of review comments -- a high number of which contributed
> nothing to the improved readability, maintaiability, or functionality
> of this code under review.
> 
> There were definitely some good points/comments. But there was a lot
> in there that you had to wade through the rest, before you saw them.

I actually am going to need to side with Duncan here, although also I'm
going to slightly disagree- but hopefully we're all used to that by now.

Duncan is right - nitpickery can be quite deadly, but I think what's
worse is when it's vague, not codified, and not checkable.

With pep8, there is a clear document, and there is a tool that a dev can
use to simply check his code. It's not like pylint, where it's literally
impossible to write code which satisfies all of the warnings - it is
completely possible to write code which is pep8 clean (as we all know,
since we are all required to do so)

But the best part about having a tool (other than my single-minded
devotion to automated gating) isn't that we can use it to gate - it's
that a dev can use it locally to verify things before sending them in
for review... and that's great. The death cycle is really about the lag
time. If you write some stuff, then run pep8 - or even nova's hacking.py
- and it tells you things like "Hey Duncan, I don't like it when you
write methods that have the word "is" in the name" - you may think it's
ridiculous, but the feedback cycle is quick and deterministic and it's
not nearly as frustrating.

I think this is why the extra pedanticness in nova has worked out ok
without killing people. The rules are in HACKING and are clear, but
they're also in tools/hacking.py - and we use them as part of the pep8
gate. Because the code is clean to begin with, they're not very onerous
to deal with... they're also simple and deterministic enough, because
someone had to code a flipping check for them.

Once there is a predictable and quick feedback cycle that can be locally
tested, a developer can train himself to write the code that way in the
first place - and they also don't feel like they're being picked on.

SO - I'd recommend a middle ground here - if you want to add additional
strictness in style checking, do what nova did with hacking.py ... we'll
happily add it to the gate if you like. However... just remember that
we're not here to write python style guidelines, or to write python
programs enforcing those guidelines (not even those of us on the CI
team) ... so if you find yourself spending weeks on a new version of
hacking.py, something has probably gone wrong.

>> though admittedly there's a lot of code previously accepted to the codebase
>> that wasn't held to such a high bar.  This attention to style guidelines is
>> generally a good thing,
> 
> I *strongly* disagree.
> 
> /Attention/ to style guidelines is a huge boon to open source
> projects. But /this/ attention seems beyond the pale, like a good idea
> was taken too far and the intent of the guidelines has been lost.
> 
>> though I understand that it can be frustrating,
>> especially for new developers unfamiliar with the rules (I personally like
>> garyk's comment about how he felt dealing with PEP-257, see:
>> http://www.youtube.com/watch?v=lYU-SeVofHs)
> 
> But that's just it: I'm *not* a new developer! I'm a seasoned Python
> hacker, PSF member, obsessive-compulsive neat-freak with code. etc.,
> etc. I haven't ever seen this level of zealous syntax pursuit in any
> well-functioning open source project.
> 
>> As long as reviewer comments are inline with items covered in
>> https://github.com/openstack/quantum/blob/master/HACKING.rst,
> 
> I may have missed something, but a lot of the comments I saw did not
> reference something particular in the HACKING file, nor were many of
> these marked as CONSIDER ...
> 
>> then I
>> consider them fair game for reviews.  If they go beyond that, they should be
>> generally be expressed as a "CONSIDER".
>>
>> If we're unhappy with what is or is not enforced,
> 
> I'm definitely unhappy with what is being enforced and how.
> 
> But even more: if reviews devolve to this level of non-code minutiae,
> how long do you think you will have the hearts and minds of
> enthu

Re: [Openstack] OpenStack "G" naming poll

[Monty Taylor]

tl;dr - Screw the rules, I agree

Let's at least add it to the poll.

Also - I think we should further amend the rules such that we select the
NEXT release by the summit for the current release. That means two things:

At the g summit, we'd tell everyone where the next summit is:
At the g summit, we'd vote and announce the name of h
We wouldn't have to spend half the cycle saying "h, or whatever" when we
mean "we're going to defer that crazy idea until next time"
I wouldn't have had to use the letter g by itself twice just above here.

On 07/03/2012 06:50 PM, Brian Waldon wrote:
> TL;DR - Screw the rules, let's call the next release 'Grizzly'
> 
> As California is rather lacking in the 'municipality names starting with
> a G that we should use for an OpenStack release' department, I have had
> to look *slightly* outside the ruleset to find a suitable 'G' release
> name - that name being 'Grizzly'. The rules clearly state that a release
> name must represent a city or county near the corresponding design
> summit and be comprised of a single word of ten characters or less - the
> problem here being that 'Grizzly' is actually 'Grizzly Flats.' Having
> already polled a small subset of the community, I feel like there would
> be enough support for 'Grizzly' to win if it were on the ballot. As I'm
> more interested in selecting a suitable name than accurately
> representing some arbitrary territory, I'd love to either permanently
> amend the rules to make this acceptable or grant an exception in this
> one case. As Thierry said, if this reaches critical mass, we will figure
> out what to do. Otherwise, I'll shut up and deal with '/Gazelle/'.
> 
> Brian
> 
> 
> On Jul 3, 2012, at 3:20 PM, Thierry Carrez wrote:
> 
>> Yes, it's that time of the year again... time for us to choose the name
>> of the next OpenStack release !
>>
>> This time, cities and counties in California (San Diego, CA being the
>> location of the G design summit)
>>
>> I set up a poll with the available options (based on our current rules
>> of naming) at:
>>
>> https://launchpad.net/~openstack/+poll/g-release-naming
>>
>> Poll is accessible to all members of ~openstack group in Launchpad, and
>> ends next Tuesday, 21:30 UTC. Please cast your vote!
>>
>> I'm aware that a subversive movement wants to try to amend the rules so
>> that another name option becomes available. Since we can't stop (or
>> modify) the poll now that it's been launched, if that movement reaches
>> critical mass, we may organize a second round of polling :)
>>
>> -- 
>> Thierry Carrez (ttx)
>> Release Manager, OpenStack
>>
>>
>> ___
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
> 
> 
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
> 


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Single global dependency list

[Monty Taylor]

It's a good and valid question and I don't really know. In this case,
I'm merely the pack-horse who was told "global synchronized dependencies
lists!" (not that I'm not the evil person cooking up schemes)

That said - most patches from new contributors don't actually come with
new library dependencies. If they are adding a new depend, I think it's
reasonable to expect it to be slightly harder to get that landed.

I do think that we need an answer to "who approves changes to this
list". Getting stuff merged to openstack-common is often hard because
it's a smaller list of people who work on it. I'd hate to see this be
only PTLs. However, things like "let's upgrade webob" seem to _actually_
need more eyes than it seems like at the time.

meh.

On 07/03/2012 03:12 PM, Gabriel Hurley wrote:
> So, I understand the rationales, and I think of those three options the one 
> chosen is the most reasonable. I'm gonna just come out and say I hate this 
> whole idea, but let's set this aside for a minute 'cuz I have a genuine 
> question:
> 
> What will the process be for merging changes to this requirements list? 
> Having yet another roadblock to getting your contribution merged is a huge 
> developer disincentive. We're really making it exceptionally hard for new 
> contributors, and frustrating even for the old hands.
> 
> So, with the goal of making the coordinated management of the projects 
> possible, what can we do to respect developers?
> 
> - Gabriel
> 
>> -Original Message-
>> From: openstack-bounces+gabriel.hurley=nebula@lists.launchpad.net
>> [mailto:openstack-
>> bounces+gabriel.hurley=nebula@lists.launchpad.net] On Behalf Of
>> Monty Taylor
>> Sent: Tuesday, July 03, 2012 7:54 AM
>> To: Eric Windisch
>> Cc: openstack@lists.launchpad.net
>> Subject: Re: [Openstack] Single global dependency list
>>
>>
>>
>> On 07/03/2012 10:09 AM, Eric Windisch wrote:
>>> I have to agree with others that copying files around is not ideal,
>>> and I can see the maintenance of this getting more involved as Nova
>>> becomes more coupled with common.
>>>
>>>>> Additionally, we'd make the copy only copy in the versions from
>>>>> openstack-common for package that were already listed in the target
>>>>> project, so that we wouldn't add django to python-swiftclient, for
>>>>> instance.
>>>
>>> This seems to be a reasonable argument against using git submodules,
>>> but I'm afraid we might be losing more than we're gaining here.
>>>
>>> Just because python-swiftclient depends on openstack-common, and
>>> django-using code exists there, doesn't mean that django needs to be
>>> installed for python-swiftclient. We might do better to use git
>>> submodules and solve the dependency problem, than continuing down
>> this
>>> copy-everything path.
>>
>> We're explicitly NOT doing a copy-everything path. That's the whole point..
>> We're only copying the needed depends from the master list.
>>
>> git submodules actually make the problem worse, not better.
>>
>>> Alternatively, speed up the movement from incubation to library.
>>
>> Yeah - that's kind of the reason that bcwaldon was saying this shouldn't be 
>> in
>> openstack-common. openstack-common wants to be a library, and then
>> we're back at not having an appropriate place for the master list.
>>
>> Monty
>>
>> ___
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
> 
> 
> 


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Single global dependency list

[Monty Taylor]

On 07/03/2012 10:09 AM, Eric Windisch wrote:
> I have to agree with others that copying files around is not ideal, and
> I can see the maintenance of this getting more involved as Nova becomes
> more coupled with common.
> 
>>> Additionally, we'd make the copy only copy in the versions from
>>> openstack-common for package that were already listed in the target
>>> project, so that we wouldn't add django to python-swiftclient, for
>>> instance.
>  
> This seems to be a reasonable argument against using git submodules, but
> I'm afraid we might be losing more than we're gaining here.
> 
> Just because python-swiftclient depends on openstack-common, and
> django-using code exists there, doesn't mean that django needs to be
> installed for python-swiftclient. We might do better to use git
> submodules and solve the dependency problem, than continuing down this
> copy-everything path.

We're explicitly NOT doing a copy-everything path. That's the whole
point. We're only copying the needed depends from the master list.

git submodules actually make the problem worse, not better.

> Alternatively, speed up the movement from incubation to library.

Yeah - that's kind of the reason that bcwaldon was saying this shouldn't
be in openstack-common. openstack-common wants to be a library, and then
we're back at not having an appropriate place for the master list.

Monty

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] PyPI uploads for client libs live

[Monty Taylor]

On 07/03/2012 08:47 AM, Doug Hellmann wrote:
> 
> On Jul 3, 2012, at 5:57 AM, Thierry Carrez 
> wrote:
> 
>> Monty Taylor wrote:
>>> At the moment, the only people with permission to upload tags is
>>> the openstack-release team. However, since we're letting client
>>> libs manage their own versions, I kinda think we should give PTLs
>>> the right on their own project - so Vish would get tag access to
>>> python-novaclient, Brian to python-glanceclient, etc.
>> 
>> Ideally it would be a two-side approval process (PTL + 
>> openstack-release), because openstack-release shouldn't be able to
>> tag without PTL approval, and openstack-release should still be
>> kept in the loop before a tag is pushed by PTLs (there are multiple
>> reasons why a few hours delay before tagging would be a good idea,
>> and the openstack-release people actually keep track of those).
>> 
>> That said, we don't have that approval mechanism available yet
>> (same issue with the core projects release tagging) so in the mean
>> time we should probably let the small set of individuals with an
>> understanding of the issues (PTLs + openstack-release) have the
>> power to do it. Within that group, we can have a soft two-side
>> approval process (based on IRC pings) to check "everything is OK"
>> before triggering a release.
> 
> Could we use gerrit's 2-step approval like we do for other projects
> and combine it with a fancy tag detector like was just added for
> DocImpact?

I have an outstanding bug/feature request for gerrit to allow for the
review of and approval or disapproval of tags.

That being said - if we wanted to go the route you're talking about in
the mean time, instead of actually using the git tag route we could have
an additional commit with a magical text in the commit message which, on
merging, would cause a tag to be created... We've got guys on the team
who hack gerrit now though - lemme get some feedback on how much it
would take to actually get proper tag reviewing.

Monty

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Single global dependency list

[Monty Taylor]

On 07/02/2012 08:43 PM, Joshua Harlow wrote:
> Ack, please don’t keep on adding on to the copy around stuff scheme.
> Plese :-)

You know - I was a huge opponent to the copy around stuff scheme when it
started. It raised all of my hackles and I got all upset about things
not being done the "right" way.

Turns out that we have a complex project with some interesting
requirements, and sometimes having an automated way to inject filtered
sets of code from a common source isn't as terrible a thing as you might
think (especially considering how half-baked ALL of pythons library and
dependency management stuff is. oy - I'm amazed I have hair left...)

> Is the devstack dependency list complete, when I created the anvil one,
> I found more than one hole...

Hi! This has nothing to do with the devstack dependency list. This list
is two lists, actually - one is the the superset of all of the
pip-requires files in all of the projects, the other is the superset of
all of the test-requires files in all of the projects.

> Also the devstack one is in a micro-format, maybe we can move to say, YAML?

The purpose of this is to drive pip installations of packages for
virtualenvs. That already has a required format that we have to use.
(again, this has nothing to do with devstack)

> How about hosting these requires on some website (with versions there)?

The apprpriate contents still need to exist in each projects'
pip-requires and test-requires files for unit testing, so you'll still
have to copy code into the projects.

> Github already provides this for the anvil dependency list, maybe that
>  (or something) similar is sufficent?

It's not even close.

It's possible that I was not clear on the purpose of what this is trying
to accomplish.

The underlying problem is that each project declares a set of
python-level dependencies. This is important so that the base unit of
release, the source tarball, is itself both complete, and also testable,
since we have to be able to have python install the necessary depends to
run unittests. I know there are people who would prefer we did that from
OS packages, but that cat is out of the bag and _way_ out of scope for
this particular discussion. Suffice it to say, each project needs a list
of depends that it can hand to pip (or in the case of the client libs,
additionally express in setup.py so that when the client lib is
installed via pip as a dependency of something else, all of its depends
come as well)

So thing number one is that there will be a python requirements.txt
format file in each project. The problem at the point is keeping them in
sync so that we don't have version drift, which is a thing we decided at
ODS that we wanted to do.

When I mentioned devstack in my earlier email, what I was referring to
is the fact that, since devstack installs each of the projects in
sequence, although we do not at the moment have a de jure global
dependency list, in practice we run all of our integration testing on
installs on a single machine - which means there _is_ a single set of
packages and versions that everything is required to run from - we just
don't declare it.

Back to the specific implementation. There are three main ways we've
come up with to solve having this list. The first had to do with git
submodules. We discarded this because most people are still against the
use of submodules in the project, and also because it makes testing
coordinated changes quite difficult here. (if you want to increase the
version of webob that the project uses, do you need to land that change
in the depend list and support it in multiple projects that are
consuming that list) The other problem is that if we just made the tools
dir a submodule (or something similar) there would be no way to exclude
certain depends from certain projects. The total list is quite large,
and for many of our projects (swift and all of the client libs) it would
introduce a massive additional unnecessary installation burden for unit
testing.

The second and third both involve copying, although the mechanism could
be different. In one version, we would do as you suggest above and post
the global depends file up on a website somewhere. Then, at runtime, the
project could do a wget of the file, perform the merge into the local
requirements list, and then go on about its business. It means that the
merge code would run hundreds of time more, the projects would lose the
ability to control when they sync up with the global depends, and it
would make it much more brittle to use any of our projects. So we
thought that since we've already gone down the hell-hole of copying code
from openstack-common, copying requirements file contents at the same
time won't kill us.

Is that more helpful?

Thanks!
Monty

> On 7/2/12 3:40 PM, "Monty Taylor"  > wrote:
> 
> Hey all!
> 
> One of the tasks from

Re: [Openstack] Single global dependency list

[Monty Taylor]

On 07/03/2012 08:43 AM, Doug Hellmann wrote:
> 
> 
> On Jul 2, 2012, at 6:40 PM, Monty Taylor 
> wrote:
> 
>> Hey all!
>> 
>> One of the tasks from the last ODS was to implement a single
>> global dependency list. Turns out the more you think about it, the
>> more important it is... because of the way we use devstack as part
>> of the gate, we actually _currently_ have a de facto global
>> dependency list, it's just not declared anywhere. (oops)
>> 
>> Anyway - the original thought was to put the depends in 
>> openstack-common. We'd use update.py to copy the depends in to the 
>> project, so that projects could align on their own timeframe. 
>> Additionally, we'd make the copy only copy in the versions from 
>> openstack-common for package that were already listed in the
>> target project, so that we wouldn't add django to
>> python-swiftclient, for instance.
>> 
>> The mechanics of that all work and are ready - but then bcwaldon
>> pointed out that it didn't make a ton of sense for them to go in 
>> openstack-common, since that has its own lifecycle and is a place
>> for common code to go - not just a catch all place.
>> 
>> To that end, I took the code we had written for the update logic
>> and put it, along with the depends lists, into its own repo. I
>> think we're ready to start actually moving forward with it - but
>> we've run up against the hardest problem we every have:
>> 
>> naming
>> 
>> openstack-depends already got vetoed on IRC because it makes
>> people think of adult diapers. I'm proposing openstack-requires,
>> since the files we're talking about are actually python
>> requirements files.
>> 
>> Any objections?
> 
> +0 on the name
> 
> As an alternative, how about combining the requirements file with the
> other packaging related stuff from openstack-common and calling the
> result openstack-packaging?

Interesting. Which other packaging stuff? Do you mean the stuff in
openstack/common/setup.py?


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] new locations for some things

[Monty Taylor]

Hey all!

(It must be a busy day - I'm writing you all so many emails...)

A little while ago, after chatting with Anne Gentle, we started
publishing the sphinx documentation to
docs.openstack.org/developer/$project  ... instead of to
$project.openstack.org. That went really well and we're happy about it
... but we didn't put in any redirects from the old locations because we
still had tarballs to content with (which we uploaded to
$project.openstack.org/tarballs)

That's fixed now - we now have tarballs going to
tarballs.openstack.org/$project, docs going to
docs.openstack.org/developer/$project, and redirects from the old
locations to the new locations.

While doing this, we added a couple of features.

First of all - client libs get their own dir, so there should be no
confusion there.

Secondly, in addition to the normal per-commit tarballs, we're now
publishing tarballs of the form "$project-$branch.tar.gz" which will get
overwritten with each commit - that way, if you need to track trunk from
a pip-requires file, (such as ceilometer, which needs to track nova
trunk) you can simply plop in something like
http://tarballs.openstack.org/nova/nova-master.tar.gz  - and it'll work
for both pip installs AND easy_install/distutils based installs. Yay!

Finally, we've got code landing that will properly publish documentation
for tagged revisions to a subdir ... so when nova gets tagged 2012.2 -
there will be a docs.openstack.org/developer/nova/2012.2 dir with those
docs in it and they will not change over time.

Just wanted to let everyone know.

Monty

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] PyPI uploads for client libs live

[Monty Taylor]

At long last, we are automatically uploading client lib packages to PyPI.

It works like this:

someone tags the repo
someone pushes that tag to gerrit
jenkins notice the tag
jenkins builds an sdist tarball and uploads it

Simple.

One of the best parts about it is that we can stop using github zipballs
in our requires files (which is one of the more common network hiccups
in our build failures) Normal version pinning works... so managing
associating with a version should work like any other library. Whee!

At the moment, the only people with permission to upload tags is the
openstack-release team. However, since we're letting client libs manage
their own versions, I kinda think we should give PTLs the right on their
own project - so Vish would get tag access to python-novaclient, Brian
to python-glanceclient, etc.

Thoughts?

Monty

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] Single global dependency list

[Monty Taylor]

Hey all!

One of the tasks from the last ODS was to implement a single global
dependency list. Turns out the more you think about it, the more
important it is... because of the way we use devstack as part of the
gate, we actually _currently_ have a de facto global dependency list,
it's just not declared anywhere. (oops)

Anyway - the original thought was to put the depends in
openstack-common. We'd use update.py to copy the depends in to the
project, so that projects could align on their own timeframe.
Additionally, we'd make the copy only copy in the versions from
openstack-common for package that were already listed in the target
project, so that we wouldn't add django to python-swiftclient, for instance.

The mechanics of that all work and are ready - but then bcwaldon pointed
out that it didn't make a ton of sense for them to go in
openstack-common, since that has its own lifecycle and is a place for
common code to go - not just a catch all place.

To that end, I took the code we had written for the update logic and put
it, along with the depends lists, into its own repo. I think we're ready
to start actually moving forward with it - but we've run up against the
hardest problem we every have:

naming

openstack-depends already got vetoed on IRC because it makes people
think of adult diapers. I'm proposing openstack-requires, since the
files we're talking about are actually python requirements files.

Any objections?

We've also been discussing an idea that we could write some gating tests
that are only run against milestone-proposed branches that would verify
that the requires files in a given project match the versions in
openstack-requires - that way there is some lee-way throughout the
cycle, but the expectation is that by release time the requires files
will be brought in line with the rest of the project. (it's an option if
people find that useful - certainly not required)

Finally, as an added bonus to this approach, once we have the list and
we're even mostly aligned on it, since a library version would need to
be in openstack-requires before it hits a project, we can use it as the
main basis for our pypi mirror and turn off the upstream pypi source
from our jenkins slaves. This would remove the last of the ephemeral
build errors that are due to upstream network timeouts. I'm sure
everyone will be pleased about that.

Anyway - I think general buy in on at _least_ the name
openstack-requires will let us move forward with the next step. After
that point, it'll all be normal code reviews.

Monty

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [OpenStack][Nova] Issues with "run_tests.sh", no tests are run when "import libvirt is present"

[Monty Taylor]

Run:

  sudo pip install tox

And you will get the tox command.

Does ./run_tests.sh -N nova.tests.test_libvirt work fine when you
_don't_ have libvirt? It needs to skip the test if you don't have
libvirt installed, and it needs to run it and pass if you do.

Jenkins is going to run "tox -v -epy27" and then eventually also "tox -v
-efull" - so make sure both of those work and you'll be set.

Monty

On 07/02/2012 10:30 AM, Leander Bessa Beernaert wrote:
> Running with " ./run_tests.sh -N nova.tests.test_libvirt" works just
> fine, however i don't know if this is enough to get it past jenkins :/
> 
> On Mon, Jul 2, 2012 at 3:26 PM, Daniel P. Berrange  > wrote:
> 
> On Mon, Jul 02, 2012 at 01:43:31PM +0100, Leander Bessa Beernaert wrote:
> > So, if no system packages can be imported, how do you test the
> "connection"
> > class for the libvirt driver?
> >
> > How does that particular test case wrap around the fact that it
> requires
> > the libvirt module? The only thing i could find are these lines of
> code in
> > the driver's __init__ method. Do these somehow detect if this is a
> unit
> > test environment and import the fakelibvirt driver instead? I'm no
> expert
> > in python so i'm not sure what's happening there :s
> >
> > > global libvirt
> > > if libvirt is None:
> > > libvirt = __import__('libvirt')
> 
> If you have installed all the neccessary python packages on your
> local host, then it is entirely possible to run the Nova test
> suites without using virtualenv. You just need to pass the '-N'
> arg to the run_tests.sh script, eg on my Fedora 17 host, I can
> run
> 
>./run_tests.sh -N nova.tests.test_libvirt
> 
> Regards,
> Daniel
> --
> |: http://berrange.com  -o-  
>  http://www.flickr.com/photos/dberrange/ :|
> |: http://libvirt.org  -o-
> http://virt-manager.org :|
> |: http://autobuild.org   -o-
> http://search.cpan.org/~danberr/ :|
> |: http://entangle-photo.org   -o-  
> http://live.gnome.org/gtk-vnc :|
> 
> 


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] PEP8 checks

[Monty Taylor]

Awesome.

On 07/02/2012 08:46 AM, John Garbutt wrote:
> I had pep8 v1.2 in my virtual env from this:
> https://github.com/openstack/nova/commit/2fa2cd2254a4044aaa584c4fcf5d6c3e1ec60d1f#tools/test-requires
> 
> Rebased with trunk and re-creating my virtualenv (i.e. move back to pep8 
> v1..1) seemed to fix things.
> 
> Thanks for your help,
> John
> 
>> -----Original Message-
>> From: Monty Taylor [mailto:mord...@inaugust.com]
>> Sent: 02 July 2012 1:28
>> To: John Garbutt
>> Cc: openstack@lists.launchpad.net
>> Subject: Re: [Openstack] PEP8 checks
>>
>>
>>
>> On 07/02/2012 06:46 AM, John Garbutt wrote:
>>> Hi,
>>>
>>> I noticed I can now run the pep8 tests like this (taken from Jenkins job):
>>> tox -v -epep8
>>> ...
>>> pep8: commands succeeded
>>> congratulations :)
>>>
>>> But the old way to run tests seems to fail:
>>> ./run-tests.sh -p
>>> ...
>>> File
>> "/home/johngar/openstack/nova/.venv/local/lib/python2.7/site-
>> packages/pep8.py", line 1220, in check_logical
>>> for result in self.run_check(check, argument_names):
>>> TypeError: 'NoneType' object is not iterable
>>>
>>> Is this expected?
>>> Did I just miss an email about this change?
>>
>> I cannot reproduce this on my system. Can you run "bash -x run_tests.sh -p"
>> and pastebin the output? Also, tools/with_venv.sh pep8 --version just to be
>> sure.
> 


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [OpenStack][Nova] Issues with "run_tests.sh", no tests are run when "import libvirt is present"

[Monty Taylor]

On 07/02/2012 08:43 AM, Leander Bessa Beernaert wrote:
> So, if no system packages can be imported, how do you test the
> "connection" class for the libvirt driver? 

We're working on that - but as I said, please try running tox -efull
which _should_ run tests with libvirt support enabled.

> How does that particular test case wrap around the fact that it requires
> the libvirt module? The only thing i could find are these lines of code
> in the driver's __init__ method. Do these somehow detect if this is a
> unit test environment and import the fakelibvirt driver instead? I'm no
> expert in python so i'm not sure what's happening there :s
> 
> global libvirt
> if libvirt is None:
> libvirt = __import__('libvirt')
> 
> 
> Regards,
> Leander 
> 
> On Mon, Jul 2, 2012 at 1:27 PM, Monty Taylor  <mailto:mord...@inaugust.com>> wrote:
> 
> 
> 
> On 07/02/2012 06:02 AM, Leander Bessa Beernaert wrote:
> > Thanks, that let me see the real problem now:
> >
> > ./tools/with_venv.sh nosetests -svx nova
> > nose.config: INFO: Set working dir to /home/gsd/nova/nova/tests
> > nose.config: INFO: Working directory /home/gsd/nova/nova/tests
> is a
> > package; adding to sys.path
> > nose.config: INFO: Ignoring files matching ['^\\.', '^_',
> > '^setup\\.py$']
> > nose.plugins.cover: INFO: Coverage report will include only
> > packages: ['nova']
> > nose.selector: INFO: /home/gsd/nova/nova/auth/opendj.sh is
> > executable; skipped
> > nose.selector: INFO: /home/gsd/nova/nova/auth/slap.sh is
> executable;
> > skipped
> > nose.selector: INFO:
> > /home/gsd/nova/nova/cloudpipe/bootscript.template is
> executable; skipped
> > Failure: ImportError (No module named libvirt) ... ERROR
> >
> ==
> > ERROR: Failure: ImportError (No module named libvirt)
> >
> --
> > Traceback (most recent call last):
> >   File
> >
> "/home/gsd/nova/.venv/local/lib/python2.7/site-packages/nose/loader.py",
> > line 390, in loadTestsFromName
> > addr.filename, addr.module)
> >   File
> >
> "/home/gsd/nova/.venv/local/lib/python2.7/site-packages/nose/importer.py",
> > line 39, in importFromPath
> > return self.importFromDir(dir_path, fqname)
> >   File
> >
> "/home/gsd/nova/.venv/local/lib/python2.7/site-packages/nose/importer.py",
> > line 86, in importFromDir
> > mod = load_module(part_fqname, fh, filename, desc)
> >   File "/home/gsd/nova/nova/test.py", line 41, in 
> > from nova.tests import fake_flags
> >   File "/home/gsd/nova/nova/tests/fake_flags.py", line 24, in
> 
> > flags.DECLARE('compute_scheduler_driver',
> 'nova.scheduler.multi')
> >   File "/home/gsd/nova/nova/flags.py", line 52, in DECLARE
> > __import__(module_string, globals(), locals())
> >   File "/home/gsd/nova/nova/scheduler/multi.py", line 27, in
> 
> > from nova.scheduler import driver
> >   File "/home/gsd/nova/nova/scheduler/driver.py", line 53, in
> 
> > flags..DECLARE('libvirt_type', 'nova.virt.libvirt.connection')
> >   File "/home/gsd/nova/nova/flags.py", line 52, in DECLARE
> > __import__(module_string, globals(), locals())
> >   File "/home/gsd/nova/nova/virt/libvirt/connection.py", line
> 73, in
> > 
> > from nova.virt.libvirt import diagnostics as
> libvirt_diagnostics
> >   File "/home/gsd/nova/nova/virt/libvirt/diagnostics.py", line 75,
> > in 
> > import libvirt as virt
> > ImportError: No module named libvirt
> >
> --
> > Ran 1 test in 0.002s
> > FAILED (errors=1)
> >
> >
> > Libvirt is present on my system, i can import it through the python
> > interpreter from any location. Yet, it still says it is not 

Re: [Openstack] PEP8 checks

[Monty Taylor]

On 07/02/2012 06:46 AM, John Garbutt wrote:
> Hi,
> 
> I noticed I can now run the pep8 tests like this (taken from Jenkins job):
>   tox -v -epep8
>   ...
>   pep8: commands succeeded
>   congratulations :)
> 
> But the old way to run tests seems to fail:
>   ./run-tests.sh -p
>   ...
>   File 
> "/home/johngar/openstack/nova/.venv/local/lib/python2.7/site-packages/pep8.py",
>  line 1220, in check_logical
>   for result in self.run_check(check, argument_names):
>   TypeError: 'NoneType' object is not iterable
> 
> Is this expected?
> Did I just miss an email about this change?

I cannot reproduce this on my system. Can you run
"bash -x run_tests.sh -p" and pastebin the output? Also,
tools/with_venv.sh pep8 --version just to be sure.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [OpenStack][Nova] Issues with "run_tests.sh", no tests are run when "import libvirt is present"

[Monty Taylor]

On 07/02/2012 06:02 AM, Leander Bessa Beernaert wrote:
> Thanks, that let me see the real problem now:
> 
> ./tools/with_venv.sh nosetests -svx nova
> nose.config: INFO: Set working dir to /home/gsd/nova/nova/tests
> nose.config: INFO: Working directory /home/gsd/nova/nova/tests is a
> package; adding to sys.path
> nose.config: INFO: Ignoring files matching ['^\\.', '^_',
> '^setup\\.py$']
> nose.plugins.cover: INFO: Coverage report will include only
> packages: ['nova']
> nose.selector: INFO: /home/gsd/nova/nova/auth/opendj.sh is
> executable; skipped
> nose.selector: INFO: /home/gsd/nova/nova/auth/slap.sh is executable;
> skipped
> nose.selector: INFO:
> /home/gsd/nova/nova/cloudpipe/bootscript.template is executable; skipped
> Failure: ImportError (No module named libvirt) ... ERROR
> ==
> ERROR: Failure: ImportError (No module named libvirt)
> --
> Traceback (most recent call last):
>   File
> "/home/gsd/nova/.venv/local/lib/python2.7/site-packages/nose/loader.py",
> line 390, in loadTestsFromName
> addr.filename, addr.module)
>   File
> "/home/gsd/nova/.venv/local/lib/python2.7/site-packages/nose/importer.py",
> line 39, in importFromPath
> return self.importFromDir(dir_path, fqname)
>   File
> "/home/gsd/nova/.venv/local/lib/python2.7/site-packages/nose/importer.py",
> line 86, in importFromDir
> mod = load_module(part_fqname, fh, filename, desc)
>   File "/home/gsd/nova/nova/test.py", line 41, in 
> from nova.tests import fake_flags
>   File "/home/gsd/nova/nova/tests/fake_flags.py", line 24, in 
> flags.DECLARE('compute_scheduler_driver', 'nova.scheduler.multi')
>   File "/home/gsd/nova/nova/flags.py", line 52, in DECLARE
> __import__(module_string, globals(), locals())
>   File "/home/gsd/nova/nova/scheduler/multi.py", line 27, in 
> from nova.scheduler import driver
>   File "/home/gsd/nova/nova/scheduler/driver.py", line 53, in 
> flags..DECLARE('libvirt_type', 'nova.virt.libvirt.connection')
>   File "/home/gsd/nova/nova/flags.py", line 52, in DECLARE
> __import__(module_string, globals(), locals())
>   File "/home/gsd/nova/nova/virt/libvirt/connection.py", line 73, in
> 
> from nova.virt.libvirt import diagnostics as libvirt_diagnostics
>   File "/home/gsd/nova/nova/virt/libvirt/diagnostics.py", line 75,
> in 
> import libvirt as virt
> ImportError: No module named libvirt
> --
> Ran 1 test in 0.002s
> FAILED (errors=1)
> 
> 
> Libvirt is present on my system, i can import it through the python
> interpreter from any location. Yet, it still says it is not present.
> Could this have to do with the virtual_env or fakelibvirt?

Yes. Our virtualenvs are currently configured to not allow system
packages to be used (so that they only use python libs installed into
the virtualenv. However, libvirt is not installable via pip, which is a
problem.

We've recently added a new tox environment to help with this, so try
running:

tox -v -efull

Which is set up to allow system site packages.

However, if you're adding unittests, you should look at the other ones
which test for libvirt existence and skip the tests if they can't find it.

> On Fri, Jun 29, 2012 at 6:54 PM, Jay Pipes  > wrote:
> 
> Hi Leander,
> 
> I've noticed some weirdness with the openstack.nose_plugin (which is
> used by default for the Nova test runner) sometimes either throwing
> errors (particularly errors raised by nosetests) away and/or coming
> up with a different set of skip tests than when running just with
> nosetests.
> 
> So, I'd recommend running just this:
> 
> ./tools/with_venv.sh nosetests -svx nova
> 
> That will stop at the first error and probably will give you better
> insight into the actual errors that are likely being suppressed by
> the openstack nose plugin.
> 
> best,
> -jay
> 
> 
> On 06/29/2012 09:02 AM, Leander Bessa Beernaert wrote:
> 
> Hello,
> 
> I'm sorry to restart the topic
> (https://lists.launchpad.net/__openstack/msg13621.html
> ), but
> i accidentally deleted the message in my inbox :S.
> 
> I'm still having the same problem, each time i add "import
> libvirt" to
> the file "diangostics.py"
> (https://review.openstack.org/__#/c/8839/
> ) the
> entire test suit won't run.
> 
> *With the import* present i get the following output:
> 
> 
>  
> 
> --

Re: [Openstack] RFC: Thoughts on improving OpenStack GIT commit practice/history

[Monty Taylor]

On 07/02/2012 05:14 AM, Daniel P. Berrange wrote:

> I must say that this has been driving me mad this last week. IIUC, only
> members of the core review team have permission to retrigger Jenkins,
> but I feel it is putting too much burden on them to have to track every
> patch with a bogus Jenkins failure. If we can't get Jenkins to be more
> reliable, then can we see about letting patch submitters retrigger
> Jenkins builds for their own patches.

There's a whole other thread about this at the moment, which outlines
the problems and what we're doing about it. One of the main issues
should be already solved. The primary goal is to get jenkins to be more
reliable so that hiccups don't happen in the first place.

In any case, if you want all the details, please see James Blair's
recent email to the Jenkins and Transient Failures thread.

Monty

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] PEP8 checks

[Monty Taylor]

On 07/02/2012 06:46 AM, John Garbutt wrote:
> Hi,
> 
> I noticed I can now run the pep8 tests like this (taken from Jenkins job):
>   tox -v -epep8
>   ...
>   pep8: commands succeeded
>   congratulations :)
> 
> But the old way to run tests seems to fail:
>   ./run-tests.sh -p
>   ...
>   File 
> "/home/johngar/openstack/nova/.venv/local/lib/python2.7/site-packages/pep8.py",
>  line 1220, in check_logical
>   for result in self.run_check(check, argument_names):
>   TypeError: 'NoneType' object is not iterable
> 
> Is this expected?
> Did I just miss an email about this change?

It's not really expected, and I honestly don't understand why
run_tests.sh -p would have problems running pep8. Although we do not use
run_tests.sh for anything in jenkins, we have not done anything to
disable or change what it's doing.

I'm looking at it right now, lemme see what's up.

Monty

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] RFC: Thoughts on improving OpenStack GIT commit practice/history

[Monty Taylor]

Hey! I agree with most of this in general. A few comments about a
section I just happened to read:

On 06/27/2012 06:52 AM, Daniel P. Berrange wrote:

snip

> Including external references
> -
> 
> The commit message is primarily targetted towards human interpretation,
> but there is always some metadata provided for machine use. In the case
> of OpenStack this includes at least the 'Change-id', but also optional
> "bug" ID references and "blueprint" name references. Although GIT records
> the author & committer of a patch, it is common practice across many
> open source projects to include a "Signed-off-by" tag. Though OpenStack
> does not mandate its use, the latter is still useful to include if a patch
> is a combination of work by many different developers, since GIT only
> records a single author. All machine targetted metadata, however, is
> of secondary consequence to humans and thus it should all be grouped
> together at the end of the commit message. For example:
> 
> 
> Switch libvirt get_cpu_info method over to use config APIs
> 
> The get_cpu_info method in the libvirt driver currently uses
> XPath queries to extract information from the capabilities
> XML document. Switch this over to use the new config class
> LibvirtConfigCaps. Also provide a test case to validate
> the data being returned
> 
> Fixes: bug #1003373
> Implements: blueprint libvirt-xml-cpu-model

These two are fine in general - although we actually parse and do bug
and blueprint linking a bit more permissively. I believe our current
regexes should still apply to those though.

> Change-Id: I4946a16d27f712ae2adf8441ce78e6c0bb0bb657
> Signed-off-by: Daniel P. Berrange 
> 
> As well as the 'Signed-off-by' tag, there are various other ad-hoc
> tags that can be used to credit other people involved in a patch
> who aren't the author.
> 
>  - 'Reviewed-by: ...some name.. <...email...>'
> 
>Although Gerrit tracks formal review by project members, some
>patches have been reviewed by people outside the community
>prior to submission

I'm not in support of this one. Our review system has no restrictions on
who can review things - and I value the actual content of their reviews,
which itself is stored in git notes along with the change. So, in
general, you can tell me that someone else reviewed it somewhere else,
but I don't really care. They should review it in the same place
everyone else does.

>  - 'Suggested-by: ...some name.. <...email...>'
> 
>If a person other than the patch author suggested the code
>enhancement / influnced the design

Great idea.

>  - 'Reported-by:  ...some name.. <...email...>'
> 
>If a person reported the bug / problem being fixed but did
>not otherwise file a launchpad bug report.

I think this should be strongly discouraged. If there is a bug or
problem, it should be filed as a bug report. I don't really know of a
valid use case where someone can't do this, but where referencing them
here is valid. We have systems, and just as you're suggesting overall
that we be a bit stricter in how we do things, I think that being more
lax in how we record some of this metadata is counter productive.


However - strongly in favor of better commit message practice!

Monty

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] need help about jenkins

[Monty Taylor]

Hi!

We were doing some maintenance this afternoon on jenkins and gerrit -
you were unlucky enough to push right in the middle of that.

Try amending your commit (git commit --amend) and change the commit
message a little (put a space after the comma here:
lp:1019348,update and then run git review again. This will cause a new
changeset to be uploaded and will re-trigger tests.

Monty


On 07/01/2012 01:30 PM, heut2008 wrote:
> Hi,all
> 
> recently,I made a commit push to gerrit ,see
> https://review.openstack.org/#/c/9204/. jenkins always show failed
> ,how can I figure out what's wrong with my commit,or some thing wrong
> with jenkins? thanks in advance:)
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
> 


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] setuptools-git

[Monty Taylor]

On 06/29/2012 10:48 AM, Alan Pevec wrote:
> On Fri, Jun 29, 2012 at 7:19 PM, Monty Taylor  wrote:
>> We use pip-requires as part of building the virtualenvs. Once we start
>> using it, setuptools-git is pretty much required for running setup.py,
>> so many common actions in our workflow will require it.
>>
>> It's a good question though - and I consider the current existence of it
>> in pip-requires purely a convenience hack. We don't have a strong place
>> in our infrastructure to indicate setup_requires entries. I'll work on
>> that next.
> 
> test-requires is also used when building venv and until there are
> separate build-requires, seems to be a better place for deps such as
> this
> 
> Trouble is that openstack.common.setup.parse_requirements() parses
> pip-requires and puts everything there into egg-info, which should be
> only run-time deps.

I just put up another possibility for review before I read this - check
it out and see what you think - we can also go test-requires if you like
that better.

Monty

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] setuptools-git

[Monty Taylor]

On 06/29/2012 10:48 AM, Alan Pevec wrote:
> On Fri, Jun 29, 2012 at 7:19 PM, Monty Taylor  wrote:
>> We use pip-requires as part of building the virtualenvs. Once we start
>> using it, setuptools-git is pretty much required for running setup.py,
>> so many common actions in our workflow will require it.
>>
>> It's a good question though - and I consider the current existence of it
>> in pip-requires purely a convenience hack. We don't have a strong place
>> in our infrastructure to indicate setup_requires entries. I'll work on
>> that next.
> 
> test-requires is also used when building venv and until there are
> separate build-requires, seems to be a better place for deps such as
> this
> 
> Trouble is that openstack.common.setup.parse_requirements() parses
> pip-requires and puts everything there into egg-info, which should be
> only run-time deps.

Yeah - good call.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] setuptools-git

[Monty Taylor]

On 06/29/2012 10:30 AM, Joshua Harlow wrote:
> Should there be a separation of build-time setup.py and run-time setup.py??
> 
> I’m not sure if something like that is possible (maybe with a setuptools
> variant, distribute or something similar??)

Well, we use distribute already - but the problem isn't really
build-time vs. run-time as much as it has to do with our use of virtualenv.

HOWEVER - I think I just had an idea of how to make this slighly
cleaner. Let me poke at it.

> On 6/29/12 4:06 AM, "Alan Pevec"  wrote:
> 
>     On Thu, Jun 28, 2012 at 9:56 PM, Monty Taylor 
> wrote:
> > https://review.openstack..org/9109 <https://review.openstack.org/9109>
> 
> Why is setuptools_git added in pip-requires, I thought that's for
> run-time, not build-time dependencies?
> 
> Cheers,
> Alan
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
> 


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] setuptools-git

[Monty Taylor]

On 06/29/2012 04:06 AM, Alan Pevec wrote:
> On Thu, Jun 28, 2012 at 9:56 PM, Monty Taylor  wrote:
>> https://review.openstack.org/9109
> 
> Why is setuptools_git added in pip-requires, I thought that's for
> run-time, not build-time dependencies?

Hey Alan!

We use pip-requires as part of building the virtualenvs. Once we start
using it, setuptools-git is pretty much required for running setup.py,
so many common actions in our workflow will require it.

It's a good question though - and I consider the current existence of it
in pip-requires purely a convenience hack. We don't have a strong place
in our infrastructure to indicate setup_requires entries. I'll work on
that next.

Monty

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Jenkins vs SmokeStack tests & Gerrit merge blockers

[Monty Taylor]

On 06/28/2012 01:49 PM, Dan Prince wrote:
> 
> 
> - Original Message -
>> From: "Monty Taylor"  To:
>> openstack@lists.launchpad.net Sent: Thursday, June 28, 2012
>> 11:13:28 AM Subject: Re: [Openstack] Jenkins vs SmokeStack tests &
>> Gerrit merge blockers
>> 
>> 
>> 
>> On 06/28/2012 07:32 AM, Daniel P. Berrange wrote:
>>> Today we face a situation where Nova GIT master fails to pass
>>> all the libvirt test cases. This regression was accidentally
>>> introduced by the following changeset
>>> 
>>> https://review.openstack.org/#/c/8778/
>>> 
>>> If you look at the history of that, the first SmokeStack test
>>> run fails with some (presumably) transient errors, and added
>>> negative karma to the change against patchset 2. If it were not
>>> for this transient failure, it should have shown the regression
>>> in the libvirt test case. The libvirt test case in question was
>>> one that is skipped, unless libvirt is actually present on the
>>> host running the tests. SmokeStack had made sure the tests would
>>> run on such a host.
>>> 
>>> There were then further patchsets uploaded, and patchset 4 was 
>>> approved for merge. Jenkins ran its gate jobs and these all
>>> passed successfully. I am told that Jenkins will actually run
>>> the unittests that are included in Nova, so I would have expected
>>> it to see the flawed libvirt test case, but it didn't. I presume
>>> therefore, that Jenkins is not running on a libvirt enabled
>>> host.
>> 
>> Kind of - it's sadly more complex than that ...
>> 
>>> The end result was that the broken changeset was merged to
>>> master, which in turns means any other developers submitting
>>> changes touching the libvirt area will get broken tests reported
>>> that were not actually their own fault.
>>> 
>>> This leaves me with the following questions...
>>> 
>>> 1. Why was the recorded failure from SmokeStack not considered to
>>> be a blocker for the merge of the commit by Gerrit or Jenkins or
>>> any of the reviewers ?
>>> 
>>> 2. Why did SmokeStack not get re-triggered for the later patch 
>>> set revisions, before it was merged ?
>> 
>> The answer to 1 and 2 is largely the same - SmokeStack is a
>> community contributed resources and is not managed by the CI team.
>> Dan Prince does a great job with it, but it's not a resource that
>> we have the ability to fix should it start messing up, so we have
>> not granted it the permissions to file blocking votes.
> 
> I would add that if anyone else is interested in collaborating on
> making SmokeStack better I'm more than happy to give access. Its all
> open source and has been since Cactus.
> 
> As is now SmokeStack can can cast a -1 vote and hopefully this is
> proving to be useful. I'm open to suggestions.

I think it's stellar!

>> 
>> The tests that smokestack runs could all be written such that they 
>> are run by jenkins.
> 
> I actually put in quite a bit of work to maintain an openstack_vpc
> job on Jenkins post-Cactus. When we talked about gating on this job
> at the Diablo conference the idea didn't seem to get very far... I
> kind of saw that as the end of the line for maintaining an
> openstack_vpc job and eventually it went away. Not sure who deleted
> it, but anyway.
>
> The way I see it there is value in both testing systems. Rather than
> complaining about why one system exists and/or doesn't port its tests
> to the other why don't we build on each others strengths. Seeing
> a green "verified +1" from both Jenkins and SmokeStack on a review
> should be very encouraging... and if one of the two systems fails it
> might require further investigation.

I completely agree with that. I'm still hoping we'll see more systems
from more people so that the set of combinations get larger.

I think also there's clearly value in running tests, like how SmokeStack
is doing right now, that aren't necessarily part of the gate, but which
pro-actively provide useful information to the reviewers.

>> The repos that run the jenkins tests are all in git and managed by
>> openstack's gerrit. If there are testing profiles that it runs that
>> we as a community value and want to see part of the gate, anyone is
>> welcome to port them.
>> 
>>> 3. Why did Jenkins not ensure that the tests were run on a
>>> libvirt enabled host ?
>> 
>> This is a different, and slight

[Openstack] setuptools-git

[Monty Taylor]

Hey all!

Using quantum as a little bit of a guinea pig, we've been poking at
setuptools-git, which is a setuptools plugin which adds git vcs support
to setuptools. Why would we care? Well, setuptools itself has baked in
support for cvs and svn (yay! such future thought!) One of the nice bits
is that if you use those, you don't have to list all of your extra files
in MANIFEST.in - it puts everything that's in your VCS into your tarball.

That's pretty much always what we want. So- I added it in a change to
quantum after we had an issue with files missing from the tarball, in
case anyone wants a look-see:

https://review.openstack.org/9109

I'd like to transition all of the projects to doing this, because, well,
let's be honest, it's less work long term and will be more correct.

Thing is- it requires a setup_requires= stanza in setup.py, which is a
facility we have not used in the project before, and is not covered in
any sensible way by either tox or install_venv. It'll still WORK, but if
you don't run off and pip install setuptools-git you're going to wind up
with an egg file in your source tree.

Anyway - we use git for everything and it doesn't hurt - so I recommend
adding setuptools-git to the list of crap you pip install outside of the
virtualenv. I'm going to try to get patches done for tox and
install_venv to slightly change how stuff gets injected so that it's not
wonky...

That is, unless there are massive objections or anything.

Monty

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Jenkins vs SmokeStack tests & Gerrit merge blockers

[Monty Taylor]

On 06/28/2012 12:05 PM, Vishvananda Ishaya wrote:
> 
> On Jun 28, 2012, at 8:13 AM, Monty Taylor wrote:
> 
>> Which adds an additional testing environment that has system software
>> enabled and also installs additional "optional" things. With that
>> environment, we should be able to run a jenkins gate that tests things
>> with full libvirt, and also tests the mysql upgrade paths, without
>> screwing our fine friends who run OSX.
> 
> Just fyi, libvirt is installable on OSX with brew and the tests
> currently run and pass.

Great!

As a follow up - the "full" env got merged today. I'll add jenkins jobs
to start running it. If you want to give it a spin locally, try:

tox -v -efull

And it should both install MySQL-python and set up the virtualenv with
system-site-packages turned to true, so libvirt tests should run and not
skip.

In theory. In practice, it's virtualenv, so all bets are off. ;)

Monty

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Adding docs gating jobs?

[Monty Taylor]

Yeah - just to make sure that docs are actually produced (basically,
anything that would cause a doc job failure)

I don't think any of us could deal with a spell checker. :)

On 06/28/2012 09:19 AM, Chmouel Boudjnah wrote:
> Just to make sure this gating test will just run python setup.py
> build_sphinx, right?
> 
> (if it was a gating spellcheck I'll be in big trouble :))
> 
> On Tue, Jun 26, 2012 at 4:02 PM, Monty Taylor  wrote:
>> Hey guys!
>>
>> We have all of the projects properly and consistently building and
>> uploading sphinx docs from in tree. This is pretty exciting, because it
>> means one more resource we can expect to work.
>>
>> So related to that, we were talking about putting in a gating job for
>> each project to prevent changes from breaking the docs. I don't really
>> expect these jobs to fail builds very often, as the jobs themselves are
>> pretty stable - but obviously it's the kind of thing people might have
>> an opinion on.
>>
>> Thoughts?
>> Monty
>>
>> ___
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
> 


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Jenkins vs SmokeStack tests & Gerrit merge blockers

[Monty Taylor]

On 06/28/2012 07:32 AM, Daniel P. Berrange wrote:
> Today we face a situation where Nova GIT master fails to pass all
> the libvirt test cases. This regression was accidentally introduced
> by the following changeset
> 
>https://review.openstack.org/#/c/8778/
> 
> If you look at the history of that, the first SmokeStack test run
> fails with some (presumably) transient errors, and added negative
> karma to the change against patchset 2. If it were not for this
> transient failure, it should have shown the regression in the
> libvirt test case. The libvirt test case in question was one that
> is skipped, unless libvirt is actually present on the host running
> the tests. SmokeStack had made sure the tests would run on such a
> host.
> 
> There were then further patchsets uploaded, and patchset 4 was
> approved for merge. Jenkins ran its gate jobs and these all passed
> successfully. I am told that Jenkins will actually run the unittests
> that are included in Nova, so I would have expected it to see the
> flawed libvirt test case, but it didn't. I presume therefore, that
> Jenkins is not running on a libvirt enabled host.

Kind of - it's sadly more complex than that ...

> The end result was that the broken changeset was merged to master,
> which in turns means any other developers submitting changes
> touching the libvirt area will get broken tests reported that
> were not actually their own fault.
> 
> This leaves me with the following questions...
> 
>  1. Why was the recorded failure from SmokeStack not considered
> to be a blocker for the merge of the commit by Gerrit or
> Jenkins or any of the reviewers ?
>
>  2. Why did SmokeStack not get re-triggered for the later patch
> set revisions, before it was merged ?

The answer to 1 and 2 is largely the same - SmokeStack is a community
contributed resources and is not managed by the CI team. Dan Prince does
a great job with it, but it's not a resource that we have the ability to
fix should it start messing up, so we have not granted it the
permissions to file blocking votes.

The tests that smokestack runs could all be written such that they are
run by jenkins. The repos that run the jenkins tests are all in git and
managed by openstack's gerrit. If there are testing profiles that it
runs that we as a community value and want to see part of the gate,
anyone is welcome to port them.

>  3. Why did Jenkins not ensure that the tests were run on a libvirt
> enabled host ?

This is a different, and slightly more complex. We run tests in
virtualenvs so that the process used to test the code can be
consistently duplicated by all of the developers in the project. This is
the reason that we no longer do ubuntu package creation as part of the
gate - turns out that's really hard for a developer running on OSX to do
locally on their laptop - and if Jenkins reports an blocking error in a
patch, we want a developer to be able to reproduce the problem locally
so that they can have a chance at fixing it.

Problem arise in paradise though. libvirt being one of them. It's not
possible to install libvirt into a virtualenv, because it's a swig-based
module built as part of the libvirt source itself. One of the solutions
to this is to allow the testing virtual environments to use packages
installed at the system level. We suggested this a little while ago, but
this was rejected by the nova team who valued the benefit of having a
restricted test run so that we know we've got all of the depends
properly specified.

To that end, after chatting with Brian Waldon, I put this up as a
possible next try:

https://review.openstack.org/#/c/8949/

Which adds an additional testing environment that has system software
enabled and also installs additional "optional" things. With that
environment, we should be able to run a jenkins gate that tests things
with full libvirt, and also tests the mysql upgrade paths, without
screwing our fine friends who run OSX.

Fundamentally though - we're at a point of trying to have our cake and
eat it too. Either we want comprehensive testing of all of the unit
tests, or we want to be careful about not making the test environment to
hard for a developer to exactly mimic.

I'm obviously on the side of having us have gating tests that some devs
might not be able to do on their laptops - such as  running the libvirt
tests properly. We're working on cloud software - worst case scenario if
there's an intractable problem, as dev can always spin up an ubuntu
image somewhere.

> Obviously this was all made worse by the transient problems we've had
> with the tests suite infrastructure these past 2 days, but regardless
> it seems like we have a gap in our merge approval procedures here.
> 
> IMHO, either SmokeStack needs to be made compulsory, or Jenkins needs
> to ensure tests are run on suitable hosts like SmokeStack does, or
> both.

The second is much more possible and as I've pointed out is in work -
but I do think we should develop a clea

[Openstack] Adding docs gating jobs?

[Monty Taylor]

Hey guys!

We have all of the projects properly and consistently building and
uploading sphinx docs from in tree. This is pretty exciting, because it
means one more resource we can expect to work.

So related to that, we were talking about putting in a gating job for
each project to prevent changes from breaking the docs. I don't really
expect these jobs to fail builds very often, as the jobs themselves are
pretty stable - but obviously it's the kind of thing people might have
an opinion on.

Thoughts?
Monty

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Thoughts on client library releasing

[Monty Taylor]

I'm going to top-post, because there is a whole other thing which is not
a response to points below. Basically, this is yet-another-instance of
two competing and partially contradictory sets of use cases and usage
patterns that we're trying to find the right compromise for.

Tying client libs to server releases is really handy for the distros. It
is terrible for the public cloud implementations who are doing rolling
releases - not in as much as it effects the public cloud's ability to
use the client libs - they can obviously pull trunk client lib from git
and use that for intra-server communication just fine as part of their
deployment. Rather, it is a terrible experience for the end-users of
OpenStack public clouds.

I'm going to use myself as an example, because it's specific.

A few months ago, HP Cloud added keystone-based auth to their
diablo-based public cloud. This was before essex released. This meant
that to use HP Cloud's public service, there were two choices:

- trunk python-novaclient
- crazy library released directly from HP

The fact that there was a stable diablo python-novaclient in existence
was useless - it did not work with this cloud that I had an account to
that was branded OpenStack (and mind you, not branded OpenStack Diablo
or OpenStack Essex - just branded OpenStack, because that's how it works)

So as an end user, if we had the 6-month client lib release model, I'm
hosed, and all of a sudden I'm installing libraries from git, or I'm
starting to use a non-standard "hpcloud" library which is then tying my
use of APIs to a non-standard thing, which is bad for all of us.

What version of a library ubuntu or fedora shipped could not be less
interesting to me - using my cloud accounts is what is important.

Thing is - trunk python-novaclient at that point in time worked just
fine with vanilla diablo clouds (as well as anything did) So there was
no reason to not do an actual pypi release of python-novaclient.

This is the model and the use case that I'm talking about here. The best
part is - there is absolutely nothing in the more-frequent client
release model which subverts the abilities of the distros to do the
thing they do - which is release software on schedules and then maintain
a version. If you're interested in Ubuntu Enterprise Cloud, and you
install UEC Essex on your local machines, then you will also install the
version of the libs that your distro released at the same time, and the
fact that there are newer libs on pypi is irrelevant to you. The distro
release model FOR CLIENT LIBS on the other hand, makes it hard for end
users to consume public cloud resources - which is why I'm suggesting
aligning this release model more closely with the needs of public
rolling releases. It seems like the right side of the line on which to
put the compromise this time. Some of the rolling-release installations
would likely prefer if we did more frequent releases for the server
projects as well... but that _does_ impact the ability of the distros to
do their thing, so we're in more alignment with the distros on that one.

So while I hear all of your points (and I promise, Thierry makes them
too) - we have many more end users and use cases that the distros or the
distro model, and I think that's most evident in the case of the client
libraries... especially in so much as I imagine the predominant channel
for the client libs with be pypi and not apt, yum or zypper.

I hope this helps understand where this approach is coming from.


As for CI - we do not strictly need to release to PyPI for CI purposes -
github zipball support allows us to meet our needs from a
pip-requires/test-requires perspective. It will be easier/make more
sense/more cacheable if it's on pypi because then our pypi mirror can
handle things, which means we aren't exposed to github breakages, but
it's certainly not needed for it. Red herring.

Monty

PS. Doing separate channels in pypi is a complete non-starter. It's a
terrible piece of software which is horribly designed, but it's the
standard mechanism for distributing python libraries, so we have to live
with it.

On 06/19/2012 08:06 AM, Mark McLoughlin wrote:
> On Tue, 2012-06-19 at 16:48 +0200, Thierry Carrez wrote:
>> Mark McLoughlin wrote:
> [..]
 However they also inherited the release scheme of the server project
 (new version every 6 months), which was (or was not) synced to PyPI
 depending of who was owning the PyPI project. More confusion as PyPI
 rarely contained the just-released version, since releasing to PyPI was
 not handled by OpenStack release management.
>>>
>>> I would have thought that only OpenStack release management should be
>>> publishing those libs to pypi now.
>>
>> PyPI doesn't support complex versioning, nor does it support "channels"
>> (think separate repositories for the same library from which you can get
>> only daily/milestones/stable-releases).
> 
> Nice way of putting it - "channels". I think that helps clarify the

Re: [Openstack] Common openstack client library

[Monty Taylor]

Hi!

On 06/19/2012 09:43 AM, Alexey Ababilov wrote:
> Hi!
> 
> Unfortunately, nova, keystone, and glance clients are very inconsistent.
> A lot of code is copied between all these clients instead of moving it
> to a common library. The code was edited without synchronization between
> clients, so, they have different behaviour:
> 
>   * all client constructors use different parameters (api_key in nova or
> password in keystone and so on);
>   * keystoneclient authenticates immediately in __init__, while
> novaclient does in lazily during first method call;
>   * {keystone,nova}client can manage service catalogs and accept
> keystone's auth URI while glanceclient allows endpoints only;
>   * keystoneclient can support authorization with an unscoped token but
> novaclient doesn't;
>   * novaclient uses class composition while keystoneclient uses inheritance.
> 
> I have developed a library to unify current clients. The library can be
> used as-is, but it would be better if openstack clients dropped their
> common code (base.py, exceptions.py and so on) and just began to import
> common code.

There are two projects already in work focused on various aspects of
this. openstack-common is the place that we put code that should be
shared between the clients. python-openstackclient is a project that
aims at a single consistent interface.

I'm thrilled that you have done some work in this area, but it would be
great if you could do this in the context of the two fairly official
projects that already exist.

Thanks!
Monty


> Here is an example of using unified clients.
> 
> from openstackclient_base import patch_clients
> from openstackclient_base.client import HttpClient
> http_client = HttpClient(username="...", password="...", tenant_name="...", 
> auth_uri="...")
> 
> from openstackclient_base.nova.client import ComputeClient
> print ComputeClient(http_client).servers.list()
> 
> from openstackclient_base.keystone.client import IdentityPublicClient
> print IdentityPublicClient(http_client).tenants.list()
> 
> 
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Openstack-poc] Thoughts on client library releasing

[Monty Taylor]

On 06/18/2012 02:26 PM, Joe Heck wrote:
> Monty -
> 
> Thierry stated it as an assumption last PPB meeting, but I'd like it
> to be explicit that we have at least a tag on each client library
> release that we make so that it's possible to distribute a version of
> the clients.

+1000

I didn't want to get too far into implementation details, but the way
I'd really like to see this work for the client libs is that releases
are actually trigger via jenkins by tags on the repo - so there would
literally be no way to release something _without_ a tag.

I've got a POC patch to do the tag-based-versioning here:

https://review.openstack.org/#/c/8427/

We need to get (aiui) one thing landed to zuul so that we can
appropriately trigger on tag events... but that's the plan in my brain hole.

> On Jun 18, 2012, at 2:11 PM, Monty Taylor wrote:
>> We're trying to figure out how we release client libraries. We're
>> really close - but there are some sticking points.
>> 
>> First of all, things that don't really have dissent (with
>> reasoning)
>> 
>> - We should release client libs to PyPI
>> 
>> Client libs are for use in other python things, so they should be
>> able to be listed as dependencies. Additionally, proper releases to
>> PyPI will make our cross project depends work more sensibly
>> 
>> - They should not necessarily be tied to server releases
>> 
>> There could be a whole version of the server which sees no needed 
>> changes in the client. Alternately, there could be new upcoming
>> server features which need to go into a released version of the
>> library even before the server is released.
>> 
>> - They should not be versioned with the server
>> 
>> See above.
>> 
>> - Releases of client libs should support all published versions of 
>> server APIs
>> 
>> An end user wants to talk to his openstack cloud - not necessarily
>> to his Essex cloud or his Folsom cloud. That user may also have
>> accounts on multiple providers, and would like to be able to write
>> one program to interact with all of them - if the user needed the
>> folsom version of the client lib to talk to the folsom cloud and
>> the essex version to talk to the essex cloud, his life is very
>> hard. However, if he can grab the latest client lib and it will
>> talk to both folsom and essex, then he will be happy.
>> 
>> There are three major points where there is a lack of clear
>> agreement. Here they are, along with suggestions for what we do
>> about them.
>> 
>> - need for "official" stable branches
>> 
>> I would like to defer on this until such a time as we actually need
>> it, rather than doing the engineering for in case we need it. But
>> first, I'd like to define we, and that is that "we" are OpenStack
>> as an upstream. As a project, we are at the moment probably the
>> single friendliest project for the distros in the history of
>> software. But that's not really our job. Most people out there
>> writing libraries do not have multiple parallel releases of those
>> libraries - they have the stable library, and then they release a
>> new one, and people either upgrade their apps to use the new lib or
>> they don't.
>> 
>> One of the reasons this has been brought up as a need is to allow
>> for drastic re-writes of a library. I'll talk about that in a
>> second, but I think that is a thing that needs to have allowances
>> for happening.
>> 
>> So the model that keystone-lite used - create an experimental
>> branch for the new work, eventually propose that it becomes the new
>> master - seems like a better fit for the "drastic rewrite" scenario
>> than copying the stable/* model from the server projects, because I
>> think the most common thing will be that library changes are
>> evolutionary, and having two mildly different branches that both
>> represent something that's actually pretty much stable will just be
>> more confusing than helpful.
>> 
>> That being said - at such a time that there is actually a
>> pain-point or a specific need for a stable branch, creating
>> branches is fairly easy ... but I think once we have an actual
>> burning need for such a thing, it will make it easier for us to
>> look at models of how we'll use it.
>> 
>> - API or major-rewrite-driven versioning scheme
>> 
>> I was wondering why bcwaldon and I were missing each other so
>> strongly in the channel the other day when we were discussing this,
>> and then I realiz

Re: [Openstack] Thoughts on client library releasing

[Monty Taylor]

On 06/18/2012 02:25 PM, Doug Hellmann wrote:
> How do these plans fit with the idea of creating a unified client
> library (either as one package or several, based on a common core)?

They are kind of orthogonal. At the point where python-openstackclient
is ready for release, we'd likely want to manage it the same way.

> On Mon, Jun 18, 2012 at 5:11 PM, Monty Taylor  <mailto:mord...@inaugust.com>> wrote:
> 
> We're trying to figure out how we release client libraries. We're really
> close - but there are some sticking points.
> 
> First of all, things that don't really have dissent (with reasoning)
> 
> - We should release client libs to PyPI
> 
> Client libs are for use in other python things, so they should be able
> to be listed as dependencies. Additionally, proper releases to PyPI will
> make our cross project depends work more sensibly
> 
> - They should not necessarily be tied to server releases
> 
> There could be a whole version of the server which sees no needed
> changes in the client. Alternately, there could be new upcoming server
> features which need to go into a released version of the library even
> before the server is released.
> 
> - They should not be versioned with the server
> 
> See above.
> 
> - Releases of client libs should support all published versions of
> server APIs
> 
> An end user wants to talk to his openstack cloud - not necessarily to
> his Essex cloud or his Folsom cloud. That user may also have accounts on
> multiple providers, and would like to be able to write one program to
> interact with all of them - if the user needed the folsom version of the
> client lib to talk to the folsom cloud and the essex version to talk to
> the essex cloud, his life is very hard. However, if he can grab the
> latest client lib and it will talk to both folsom and essex, then he
> will be happy.
> 
> There are three major points where there is a lack of clear agreement.
> Here they are, along with suggestions for what we do about them.
> 
> - need for "official" stable branches
> 
> I would like to defer on this until such a time as we actually need it,
> rather than doing the engineering for in case we need it. But first, I'd
> like to define we, and that is that "we" are OpenStack as an upstream.
> As a project, we are at the moment probably the single friendliest
> project for the distros in the history of software. But that's not
> really our job. Most people out there writing libraries do not have
> multiple parallel releases of those libraries - they have the stable
> library, and then they release a new one, and people either upgrade
> their apps to use the new lib or they don't.
> 
> One of the reasons this has been brought up as a need is to allow for
> drastic re-writes of a library. I'll talk about that in a second, but I
> think that is a thing that needs to have allowances for happening.
> 
> So the model that keystone-lite used - create an experimental branch for
> the new work, eventually propose that it becomes the new master - seems
> like a better fit for the "drastic rewrite" scenario than copying the
> stable/* model from the server projects, because I think the most common
> thing will be that library changes are evolutionary, and having two
> mildly different branches that both represent something that's actually
> pretty much stable will just be more confusing than helpful.
> 
> That being said - at such a time that there is actually a pain-point or
> a specific need for a stable branch, creating branches is fairly easy
> ... but I think once we have an actual burning need for such a thing, it
> will make it easier for us to look at models of how we'll use it.
> 
>  - API or major-rewrite-driven versioning scheme
> 
> I was wondering why bcwaldon and I were missing each other so strongly
> in the channel the other day when we were discussing this, and then I
> realized that it's because we have one word "API" that's getting
> overloaded for a couple of different meanings - and also that I was
> being vague in my usage of the word. So to clarify, a client library
> has:
> 
>  * programming level code APIs
>  * supported server REST APIs
> 
> So I back off everything I said about tying client libs version to
> server REST API support. Brian was right, I was wrong. The thing that's
> more important here is that the version should indicate programmer
>   

[Openstack] Thoughts on client library releasing

[Monty Taylor]

We're trying to figure out how we release client libraries. We're really
close - but there are some sticking points.

First of all, things that don't really have dissent (with reasoning)

- We should release client libs to PyPI

Client libs are for use in other python things, so they should be able
to be listed as dependencies. Additionally, proper releases to PyPI will
make our cross project depends work more sensibly

- They should not necessarily be tied to server releases

There could be a whole version of the server which sees no needed
changes in the client. Alternately, there could be new upcoming server
features which need to go into a released version of the library even
before the server is released.

- They should not be versioned with the server

See above.

- Releases of client libs should support all published versions of
server APIs

An end user wants to talk to his openstack cloud - not necessarily to
his Essex cloud or his Folsom cloud. That user may also have accounts on
multiple providers, and would like to be able to write one program to
interact with all of them - if the user needed the folsom version of the
client lib to talk to the folsom cloud and the essex version to talk to
the essex cloud, his life is very hard. However, if he can grab the
latest client lib and it will talk to both folsom and essex, then he
will be happy.

There are three major points where there is a lack of clear agreement.
Here they are, along with suggestions for what we do about them.

- need for "official" stable branches

I would like to defer on this until such a time as we actually need it,
rather than doing the engineering for in case we need it. But first, I'd
like to define we, and that is that "we" are OpenStack as an upstream.
As a project, we are at the moment probably the single friendliest
project for the distros in the history of software. But that's not
really our job. Most people out there writing libraries do not have
multiple parallel releases of those libraries - they have the stable
library, and then they release a new one, and people either upgrade
their apps to use the new lib or they don't.

One of the reasons this has been brought up as a need is to allow for
drastic re-writes of a library. I'll talk about that in a second, but I
think that is a thing that needs to have allowances for happening.

So the model that keystone-lite used - create an experimental branch for
the new work, eventually propose that it becomes the new master - seems
like a better fit for the "drastic rewrite" scenario than copying the
stable/* model from the server projects, because I think the most common
thing will be that library changes are evolutionary, and having two
mildly different branches that both represent something that's actually
pretty much stable will just be more confusing than helpful.

That being said - at such a time that there is actually a pain-point or
a specific need for a stable branch, creating branches is fairly easy
... but I think once we have an actual burning need for such a thing, it
will make it easier for us to look at models of how we'll use it.

 - API or major-rewrite-driven versioning scheme

I was wondering why bcwaldon and I were missing each other so strongly
in the channel the other day when we were discussing this, and then I
realized that it's because we have one word "API" that's getting
overloaded for a couple of different meanings - and also that I was
being vague in my usage of the word. So to clarify, a client library has:

 * programming level code APIs
 * supported server REST APIs

So I back off everything I said about tying client libs version to
server REST API support. Brian was right, I was wrong. The thing that's
more important here is that the version should indicate programmer
contract, and if it that is changed in a breaking manner, the major
number should bump.

If we combine that with the point from above that our libraries should
always support the existing server REST APIs, then I think we can just
purely have statements like "support for compute v3 can be found in
2.7.8 and later" and people will likely be fine, because it will map
easily to the idea "just grab the latest lib and you should be able to
talk to the latest server" Yea?

So in that case, the client libs versions are purely whatever they are
right now, and we'll increase them moving forward using normal library
version thoughts.

 - room for deprecating old APIs

The above then leads us to wanting to know what we do about supported
server REST APIs over time, especially since I keep making sweeping
statements about "should support all available server versions" ... How
about this as a straw man: Since we're planning on beginning to run
tests of the client libs against previous versions (so we'll test trunk
novaclient against essex nova in addition to trunk nova) ... we need
support for past versions of servers as long as our automation can
sensibly spin up a past version. (Since the supp

Re: [Openstack] [Swift] swift.common.client and swift CLI has moved to its own project python-swiftclient

[Monty Taylor]

On 06/13/2012 10:58 AM, Juan J. Martinez wrote:
> On 13/06/12 15:42, Dan Prince wrote:
>> Okay. It looks like Swift also still depends on swiftclient. Long term it 
>> would be nice if we could build and unit test swift without relying on the 
>> swiftclient package. Could we:
>>
> 
> I can't see the reason for that, swiftlclient is a dependency of swift
> (as webob, greenlet, pastedeploy, etc).

I agree. At the moment it's slightly more ugly because we haven't
finalized the pypi release process for our client libs... but we should
have that in the next week or so, and then it should be perfectly
possible to treat it just like anything else.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Glance/Swift integration broken (kills devstack)

[Monty Taylor]

On 06/11/2012 07:06 PM, Gabriel Hurley wrote:
> I just thought I'd call out that Glance's swift storage module is
> currently broken, and apparently this escaped the devstack gate even
> though devstack actually fails to complete if swift is enabled. Are
> we not testing with swift in the ENABLED_SERVICES list?
>
> Bug report here: https://bugs.launchpad.net/devstack/+bug/1011885

We do not run swift through the gate:

https://github.com/openstack-ci/devstack-gate/blob/master/devstack-vm-gate.sh#L28

When we first started doing the devstack gate, I do not believe that
swift support in devstack worked fully, and since then, no one has
turned it on in the gate.

Monty

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Errors running individual tests that call into the database

[Monty Taylor]

On 06/11/2012 12:04 PM, John Garbutt wrote:
> Hi,
> 
> I am trying to run tests like "test_xenapi" and "test_libvirt" by themselves 
> do things like:
>nosetests test_xenapi
> But it does work, I get DB errors relating to missing tables. However, I can 
> successfully run all the tests.
> 
> The way I understand it:
>  - nova.tests.__init__.py setup() does the database setup
>  - nova.test.py TestCase.setUp() does the resetting of the db
>  It is almost like doing "nosetests test_asdf" skips the database setup in 
> nova.tests.__init__.py, is that correct?

I believe this is because "nosetests test_asdf" is taking advantage of
the "where=nova/tests" in setup.cfg and actually just starts the test
from there. So whereas run_tests.py aliased "run_tests.sh test_asdf" to
"run_tests nova.tests.test_asdf", I believe nosetests is now actually
changing directory into nose/tests and then running, so is not actually
loading nova/tests/__init__.py because it's not resolving the library
path that way.

> Any ideas on how to run tests individually, but still get the database 
> correclty initialized? Am I just calling the tests incorrectly?

I like the patch that was submitted for this, actually. Part of the
intent behind the recent patches it to get our test runner behaving a
little bit more normal and to put extra things we need inside of test
fixtures and the like so that as we continue to grow we can take
advantage of the work of other people on things like automatic test run
parallelization and what not. So actually, doing database setup in the
test fixture is the more correct way to do things - whereas just relying
on __init__ import semantics to do it is a bit of a hack.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] busy day for the CI team

[Monty Taylor]

Hey guys!

In a fit of things coming together... today has been an unnaturally good
and productive day for us, so I thought I'd mention a bunch of stuff
(most things long-in-work and just sort of came together nicely today)

- builders are now updated to run precise - except for the python2.6
builders, which are still on oneiric since there's no 2.6 on oneiric

- local pypi mirror is in use and effective for unittests and devstack

- gerrit 2.4 rolled out (see separate email)

- fix the horizon build problem, which was actually pip related

- rolled out xunit output for unittests along with processing in jenkins
for all projects

- landed changes in glance and nova that use nosetests directly and
remove run_tests.py. Additionally, logging output is done in such a way
that nosetests processes it, so the above jenkins output should show
tracelog info automatically along with failed tests. We'll carry this
over to everyone else soon.

Monty


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] new version of gerrit - with new features!

[Monty Taylor]

Hey guys!

We just upgraded to a new version of gerrit. This is based on the new
upstream version 2.4, but in addition we've landed two additional
features on top of that - so there's tons of new toys to play with.

First of all, in 2.4 upstream has added a new button "Rebase Change" ...
which you can use to rebase your change on top of the current tip of the
target branch from within gerrit itself. Also, upstream has been working
on adding a proper REST interface, instead of json-rpc which is what
they have been using. I'm not sure how far that's gotten, but I believe
it can do a decent amount of stuff for those of you who like, you know,
REST-based scripting.

In addition to that, we've got two main features we've landed as well.

David Shrewsbury wrote a long-requested feature: a Work In Progress
state. Changes will now have a Work In Progress button on them that can
be used to mark the change as something that the dev is working on
again, so that other folks know they don't need to review it. The button
is available to the author of the patch, as well any group that we
assign to have access. In our case, we'll be granting $project-core Work
In Progress permission. Putting something back into the "ready for
review" state can be done one of two ways - either by just uploading a
new patch to the change, or by clicking the "Ready for Review" button.

Hand in hand with that change, Clark Boylan has written an "Important
Changes" view - which shows all on one page the changes that a developer
should be looking at. On the page are changes that were uploaded by the
developer (same as the "My Changes" page), then a section for changes
that the developer should be reviewing, which are changes that the dev
has either watched, starred, or that reviews have been requested, and
that are no in work in progress state and additionally that the dev has
not already reviewed. Finally, there is a section for changes that the
developer has already reviewed, in case they need to be further tracked.

We're hoping that some of these things help to reduce a little bit of
the burden in terms of tracking which things should be watched. We'll be
working on getting our patches upstreamed in the near future... but
since they are new workflow possibilities, we're happy to get feedback
on ways in which they could be more useful to folks.

Also - we have an open question - which is, if the pre-approval check
jobs fail in Jenkins, should the patch be automatically marked Work In
Progress. We're not going to do that right out of the gate, but would
love feedback on what people think.

Thanks everybody!
Monty

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] depend discrepancies

[Monty Taylor]

Hey guys!

One of the things that came out of ODS is the idea of having a single
global dependency list. There are two bits to that - the mechanics of
managing the list (which I think we may have sorted) and then, you know,
making the list. In compiling the list of what the current global list
is, most of the things have clear and obvious answers. However, there
are three problematic deps: kombu, PasteDeploy and xattr. Here's what we
have for them:

./melange/tools/pip-requires:kombu==1.5.1
./glance/tools/pip-requires:kombu
./nova/tools/pip-requires:kombu==1.0.4
./cinder/tools/pip-requires:kombu==1.0.4

./keystone/tools/pip-requires:PasteDeploy
./melange/tools/pip-requires:PasteDeploy
./quantum/tools/pip-requires:PasteDeploy==1.5.0
./glance/tools/pip-requires:PasteDeploy
./nova/tools/pip-requires:PasteDeploy==1.5.0
./swift/tools/pip-requires:pastedeploy==1.3.3
./cinder/tools/pip-requires:PasteDeploy==1.5.0

./glance/tools/pip-requires:xattr>=0.6.0
./swift/tools/pip-requires:xattr==0.4

I have no personal emotions towards any of these versions ... but if
folks who matter could make some call on what they _should_ be, we can
move forward with the first step.

I should point out that at the moment we're looking at using update.py
from openstack-common to copy in the relevant depends from the master
list - so a project will only get the ones it needs. It also means that
a decision on a version here does not mean that everyone needs to move
to that version immediately ... just that we should be moving towards
supporting those versions before folsom, really.

Anywhoo... thoughts?

Monty

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [metering] Changes to the ceilometer Jenkins job

[Monty Taylor]

On 06/05/2012 05:24 AM, Julien Danjou wrote:
> On Tue, May 29 2012, Julien Danjou wrote:
> 
>> We noticed that currently, on Stackforge, the ceilometer project has
>> only one Jenkins job, checking that the merge occurs correctly.
>>
>> We'd like to add jobs to run unit tests, pep8, etc… I took a look at the
>> openstack-ci-puppet repo, but I'm not sure of the changes, so I prefer
>> to ask here for someone that understand this to help.
>> From what I see, it's likely we would like to use the default
>> "python_jobs" template.
> 
> Is there a way to add more checks without touching the
> openstack-ci-puppet repository?

Nope. BUT - you can certainly add more checks to the ceilometer.yaml
file and submit them for merge. (feel free to follow up with us if that
doesn't make sense the first time) We actually _just_ finished doing a
ton of work to make it possible to control all of this via git so that
we could let anyone help hack the jenkins jobs that they need.

> I'd like to run tests with tox on the same py26/27 envs but using a
> different set of pip-requires. I found out how to write the tox.ini
> part, but I am not sure how the envlist is controlled on the Jenkins
> side.

It should be easy enough to add a few jobs to your file that will run
the run_tox.sh script with the names of your new environments. Find us
in #openstack-infra and we'll talk you through it.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] quantumclient=2012.1

[Monty Taylor]

On 06/05/2012 07:54 AM, Gary Kotton wrote:
> Hi Monty and Dan,
> Background: A short while ago I started to port bug fixes for Quantum
> from Folsom-1 to Stable Essex. Jenkins did not accept the patches due to
> the fact that the automatic tests did not pass. The failures are due to
> 2 reasons:
> 1. pep8 checks (addressed in the tox.ini)
> 2. missing files for automatic tests
> A fix for this issue was proposed -
> https://review.openstack.org/#/c/8023. Mark McLoughlin rightly pointed
> out that the fix was risky by adding a considerable amount of code and
> suggested adding quantumclient==2012.1 to the pip-requires. The problem
> with this is that the quantumclient does not exist in pypi
> (http://pypi.python.org/pypi/python-quantumclient).
> How do you suggest that we proceed?

For now, add:

https://github.com/openstack/python-quantumclient/zipball/2012.1#egg=python-quantumclient

to the quantum stable/essex pip-requires

We're really close to figuring out the client lib upload issue overall
(there are about 100 requirement from about 100 different people - but I
think we've almost got it sorted) but until we can get it done properly,
just do the github zipball bit.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] git-based jenkins jobs and pre-approval check jobs

[Monty Taylor]

On 05/28/2012 04:32 PM, Paul Belanger wrote:
> On 12-05-25 03:58 PM, Monty Taylor wrote:
>> Hey guys!
>>
>> We just finished rolling out the first in a sequence of upcoming changes
>> to gerrit and jenkins based on needs described at the design summit.
>>
>> We now have the basic jobs for all of the projects (except for horizon,
>> because it's slightly different and I want to spend a little more time
>> with it) applied to jenkins based on some scripts and some yaml files
>> that are in a git repo. This means that anybody could conceivably do
>> some hacking and submit a change to gerrit, instead of the current
>> status quo which is that you have to be a jenkins admin to touch
>> anything.
>>
>> If you want to look at it, it's all in this dir:
>>
>> https://github.com/openstack/openstack-ci-puppet/tree/master/modules/jenkins_jobs/files
>>
>>
> What was the reason for moving these files directly into puppet and not
> in a separate git repo? I'm surprise they are not actually packaged to
> be honest.

WELL... a couple of things. First of all, this is stab one, and as we
add more openstack projects and jobs we keep finding more features, etc.
that we need. It seemed like just doing it directly in one place at the
start while we're rapidly iterating on it was less painful. However, as
it solidifies, I'm hoping we can have a thing that is the jenkins job
filler that can be installed, and then the _config_ of that that's
specific to a project is something that puppet can manage. So let's call
it coming-soon. :)

We're also still a little specifically openstack oriented at the moment
... but we do want to go back through and re-generalize some things as
we get a handle on where we need config bits and whatnot.


>> Additionally, it's grouped/templated, so we've actually got identical
>> jenkins jobs running for all of the projects... which means we can
>> hopefully stop running in to the whole "oops, I forgot to update the
>> glance jobs" situation.
>>
>> But wait, that's not all ...
>>
>> By popular demand, as part of rolling out those jobs, we have added
>> pre-approval check jobs. We are now running merge checks, pep8 checks
>> and unittest jobs on the patch-uploaded event and reporting the results
>> into the code review so that people can skip code reviewing patches that
>> don't work yet. We're also still running tests post-approval so that we
>> ensure the patch as it is to be merged is still good.
>>
>> We have several more great bits that will be rolling out in the next
>> week or two, in the attempt to streamline the set of things that need
>> review, and also speed up the testing of reviewed things.
>>
>> Happy hacking
>>
> Nice work! I always enjoy looking to your work and see what you guys are
> doing in the world of automation.

Thanks!

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] git-based jenkins jobs and pre-approval check jobs

[Monty Taylor]

Hey guys!

We just finished rolling out the first in a sequence of upcoming changes
to gerrit and jenkins based on needs described at the design summit.

We now have the basic jobs for all of the projects (except for horizon,
because it's slightly different and I want to spend a little more time
with it) applied to jenkins based on some scripts and some yaml files
that are in a git repo. This means that anybody could conceivably do
some hacking and submit a change to gerrit, instead of the current
status quo which is that you have to be a jenkins admin to touch anything.

If you want to look at it, it's all in this dir:

https://github.com/openstack/openstack-ci-puppet/tree/master/modules/jenkins_jobs/files

Additionally, it's grouped/templated, so we've actually got identical
jenkins jobs running for all of the projects... which means we can
hopefully stop running in to the whole "oops, I forgot to update the
glance jobs" situation.

But wait, that's not all ...

By popular demand, as part of rolling out those jobs, we have added
pre-approval check jobs. We are now running merge checks, pep8 checks
and unittest jobs on the patch-uploaded event and reporting the results
into the code review so that people can skip code reviewing patches that
don't work yet. We're also still running tests post-approval so that we
ensure the patch as it is to be merged is still good.

We have several more great bits that will be rolling out in the next
week or two, in the attempt to streamline the set of things that need
review, and also speed up the testing of reviewed things.

Happy hacking

Monty

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] How to add quantum-client coverage report on jenkins site

[Monty Taylor]

Gimme a day or two - we're about to roll out a thing which adds all of
the standard jobs for everything driven from files in git ... so this
all gets much easier real soon.

On 05/16/2012 07:21 PM, Yong Sheng Gong wrote:
> 
> Hi,
> I cannot find the quantum-client coverage report on jenkins sie:
> https://jenkins.openstack.org/
> 
> 
> How to add it?
> Thanks
> Yong Sheng Gong
> 
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Translation and Internationalization in OpenStack

[Monty Taylor]

On 05/08/2012 09:56 PM, Thierry Carrez wrote:
> Gabriel Hurley wrote:
>> Having worked with all three tools, I would strongly suggest Transifex, 
>> particularly given that we as a community have to do almost no work to 
>> maintain it, it's the only tool that supports OpenStack as a "project hub" 
>> with shared teams and management, and it offers us a strong crowdsourced 
>> translation community.
> 
> Getting a strong crowdsourced translation community is the most
> important aspect to me. We can relatively easily bridge the
> tooling/integration gap, but we can't invent a translation community :)
> 
> I know for a fact that Launchpad has a magic translation community (just
> pushing stuff there makes it translated). If Transifex's community is as
> efficient and gives us better integration, then we should go for it.
> 
>> As per Thierry's call for volunteers, I will throw my hat in the ring to 
>> spearhead translation efforts in OpenStack for the time being.
> 
> Great! I'm happy to defer the tool decision to the people that will own
> and push that work forward ;)

Same here. Transifex seems like it meets our needs - and if it's got a
champion who wants to do the work, then stellar!

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Nova subsystem branches and feature branches

[Monty Taylor]

On 05/03/2012 08:50 AM, Mark McLoughlin wrote:
> On Thu, 2012-05-03 at 16:46 +0100, Mark McLoughlin wrote:
>> Hey,
>>
>> On Thu, 2012-05-03 at 14:24 +0200, Thierry Carrez wrote:
>>> Mark McLoughlin wrote:
> 
 And how about feature branches?

   - Feature branches are relatively short-lived (i.e. weeks or months
 rather than years) branches for a specific feature. They are a
 mechanism for developers to work on a patch series in the open until
 the feature is complete enough to be merged into a subsystem branch
 or master.

 (I'm not sure gerrit is right for this. Why not just do it in 
 folk's github forks? I think all people are looking for is for 
 people to be more aware of feature branches. How about if you put 
 details of your feature branch in the blueprint for the feature?)

 (If not using gerrit, can developers configure Jenkins to CI their 
 branch? Or is Smokestack the right tool?)
>>>
>>> I think preserving the ability to run your branch through integration
>>> testing is a necessary prerequisite of the new model.
>>
>> Yes, it's only really needed at the point where the feature branch is
>> merged into the subsystem tree. The developer should be able to break
>> stuff on their WIP feature branch if they wish, since they'll be expect
>> to rebase/fix before proposing it to be merged into the subsystem tree.
>>
>> So Jenkins/Smokestack would be nice tools to provide to folks working on
>> feature branches, but they don't need to be gating.
> 
> Put another way - the gating tests should be run on every single commit
> that is ultimately merged into master.
> 
> However, this should not mean that we apply gating to every commit on
> feature branches, since feature branches are allowed to rebase until
> they are merged into a subsystem tree or directly into master.

Yes. This all makes sense to me and I can be in support of this plan
(and continue work on making that process of moving from feature branch
to subsystem branch is not painful)

Monty

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Nova subsystem branches and feature branches

[Monty Taylor]

On 05/03/2012 05:24 AM, Thierry Carrez wrote:

(snip things I pretty much agree with)

>> (I'm not sure gerrit is right for this. Why not just do it in 
>> folk's github forks? I think all people are looking for is for 
>> people to be more aware of feature branches. How about if you put 
>> details of your feature branch in the blueprint for the feature?)
>>
>> (If not using gerrit, can developers configure Jenkins to CI their 
>> branch? Or is Smokestack the right tool?)
> 
> I think preserving the ability to run your branch through integration
> testing is a necessary prerequisite of the new model.

Agree, otherwise it gets weird.

HOWEVER - I also agree that the current UI as we're presenting it might
not be optimal. Let me follow up with some sketched out thoughts on how
we can address both concerns.

> Thanks again for starting the discussion on this. I'll meet with Jim
> Blair (and hopefully Monty) next week to brainstorm solutions, so
> discussing the needed properties of the system is a very nice step forward.
> 
> Regards,
> 

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Client branches?

[Monty Taylor]

On 04/30/2012 05:06 PM, Joshua Harlow wrote:
> Hi all,
> 
> I was wondering if the clients (ie novaclient, keystoneclient) are
> supposed to have branches for “stable/essex” or not, they currently just
> have master and milestone proposed?

Hey!

They are as of right now NOT supposed to have stable/* branches - the
idea being that trunk *client should always work with trunk openstack as
well as previous stables. The key thing here being that an end user is
likely to want to use a client library to talk to an openstack cloud,
and they should not need to know which code release their cloud ran on.

We have a todo list item to expand gating for the client libs to run
against both trunk and stable server stuff ... but it's been a little
busy since the ODS and we haven't gotten to it yet.

Monty

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Metering] Metering repository in stackforge

[Monty Taylor]

Hey!

On 04/27/2012 06:20 PM, Loic Dachary wrote:
> Hi,
> 
> I would like to create a repository ceilometer in 
> https://github.com/stackforge to host the code for the newborn Metering 
> project ( https://launchpad.net/ceilometer , first meeting held this thursday 
> http://wiki.openstack.org/Meetings/MeteringAgenda ).

We would be more than thrilled to get you set up. I'm including Andrew
here, as he's been doing most of the StackForge work. I'll also file a
bug on openstack-ci to make sure we don't lose this.

> I'm not sure how to proceed, could you please guide me ? My github account 
> name is dachary
> 
> Thanks in advance
> 

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Mailing-list split

[Monty Taylor]

On 04/27/2012 09:44 AM, Duncan McGreggor wrote:
> On Fri, Apr 27, 2012 at 8:46 AM, Monty Taylor  wrote:
>> Hey everyone!
>>
>> On 04/27/2012 05:04 AM, Thierry Carrez wrote:
>>
>>> To avoid Launchpad list slowness, we would run the new openstack-dev
>>> list off lists.openstack.org. Given the potential hassle of dealing with
>>> spam and delivery issues on mission-critical MLs, we are looking into
>>> the possibility of outsourcing the maintenance of lists.openstack.org to
>>> a group with established expertise running mailman instances. Please let
>>> us know ASAP if you could offer such services. We are not married to
>>> mailman either -- if an alternative service offers good performance and
>>> better integration (like OpenID-based subscription to integrate with our
>>> SSO), we would definitely consider it.
>>
>> Just to be clear - I definitely think that mailing lists are an
>> important part of dev infrastructure and would love for this to be a
>> fully integrated part of all of the rest of our tools. However, the
>> current set of active infrastructure team members have huge todo lists
>> at the moment. So the biggest home run from my perspective would be if
>> someone out there had time or resources and wanted to join us on the
>> infra team to manage this on our existing resources (turns out we have
>> plenty of servers for running this, and even a decent amount of
>> expertise, just missing manpower). The existing team would be more than
>> happy to be involved, and it would help avoid get-hit-by-a-truck issues.
>> We're a pretty friendly bunch, I promise.
>>
>> Any takers? Anybody want to pony up somebody with some bandwidth to
>> admin a mailman? Respond back here or just find us in #openstack-infra
>> and we'll get you plugged in and stuff.
>>
>> Thanks!
>> Monty
> 
> Count me in, Monty.
> 
> I've been managing mailman lists for about 12 years now (and,
> incidentally, Barry and I are bruthas from anutha mutha), so I'd be
> quite comfortable handling those responsibilities. I can couple it
> with the python.org SIG mail list that I manage, so there'd be zero
> context switching.

You make me very happy! Let's work out the details and stuff...

Check it out - it's like we're, you know, a collaborative community or
something!

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Mailing-list split

[Monty Taylor]

Hey everyone!

On 04/27/2012 05:04 AM, Thierry Carrez wrote:

> To avoid Launchpad list slowness, we would run the new openstack-dev
> list off lists.openstack.org. Given the potential hassle of dealing with
> spam and delivery issues on mission-critical MLs, we are looking into
> the possibility of outsourcing the maintenance of lists.openstack.org to
> a group with established expertise running mailman instances. Please let
> us know ASAP if you could offer such services. We are not married to
> mailman either -- if an alternative service offers good performance and
> better integration (like OpenID-based subscription to integrate with our
> SSO), we would definitely consider it.

Just to be clear - I definitely think that mailing lists are an
important part of dev infrastructure and would love for this to be a
fully integrated part of all of the rest of our tools. However, the
current set of active infrastructure team members have huge todo lists
at the moment. So the biggest home run from my perspective would be if
someone out there had time or resources and wanted to join us on the
infra team to manage this on our existing resources (turns out we have
plenty of servers for running this, and even a decent amount of
expertise, just missing manpower). The existing team would be more than
happy to be involved, and it would help avoid get-hit-by-a-truck issues.
We're a pretty friendly bunch, I promise.

Any takers? Anybody want to pony up somebody with some bandwidth to
admin a mailman? Respond back here or just find us in #openstack-infra
and we'll get you plugged in and stuff.

Thanks!
Monty

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Mailing-list split

[Monty Taylor]

On 04/27/2012 06:11 AM, Jim Meyering wrote:
> Daniel P. Berrange wrote:
>> On Fri, Apr 27, 2012 at 12:04:34PM +0200, Thierry Carrez wrote:
>>> To avoid Launchpad list slowness, we would run the new openstack-dev
>>> list off lists.openstack.org. Given the potential hassle of dealing with
>>> spam and delivery issues on mission-critical MLs, we are looking into
>>> the possibility of outsourcing the maintenance of lists.openstack.org to
>>> a group with established expertise running mailman instances. Please let
>>> us know ASAP if you could offer such services. We are not married to
>>> mailman either -- if an alternative service offers good performance and
>>> better integration (like OpenID-based subscription to integrate with our
>>> SSO), we would definitely consider it.
>>
>> FYI for libvirt mailing lists we are using the GNU project's spam
>> filter:
>>
>>   https://savannah.gnu.org/maintenance/ListHelperAntiSpam
> 
> I think the procedure documented there works fine for gnu projects,
> but that something special has to be done for projects hosted elsewhere.
> I'll check and get back to you.

Awesome - and thanks. It's entirely possible that our Jim Blair already
knows a decent amount of that, as he set up most of the FSF's mail and
mailing list infrastructure, iirc. BUT - that was several years ago, so
it's also possible something has changed. :)

>> Thanks to this we get essentially zero spam on our mailing lists,
>> with practically no burden on / work required by our list admins.
>> Jim Meyering (CC'd) set it up for us originally & may have some
>> recommendations or tips if OpenStack want to make use of it too.
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
> 

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Using Foreign Keys

[Monty Taylor]

On 04/26/2012 10:14 AM, Sean Dague wrote:
> On 04/25/2012 05:17 PM, Vishvananda Ishaya wrote:
>> The main issue is when the relevant tables are moved into a separate
>> service a la quantum or cinder. We can't keep referential integrity
>> across multiple databases, so the foreign keys in this case need to be
>> removed. It leads to an odd situation when there is still an internal
>> implementation in addition to the external implementation because the
>> internal implementation no longer has foreign keys.
>>
>> As an example, we used to have foreign key relationships between
>> instances and networks. We can no longer have these because we support
>> networks declared externally. The internal network management now has no
>> referential integrity, but this is the price we pay for separation of
>> concerns. We are going through a similar set of relationship-breaking
>> with the volume code.
> 
> There are definitely the practical aspects of where this "can't" be done
> because the services have split out, and I think that's fine.
> 
> But enforcing the ref constraints where possible just provides another
> level of safety in the data. A policy where we break FK relationships if
> the preferred core model is 2 services (i.e. Nova / Quantum), but we add
> FK constraints within a service might be a good idea.

SO ... in a production MySQL service in this situation, under no
circumstances should foreign keys actually be applied to the database.
Specifying them as part of the SqlAlchemy model is fine, and I believe
conveys the informational relationships that are important. But it turns
out that in practice, especially with an ORM running things, the
performance hit of adding them is pretty bad (generates tons of unneeded
index scans, for one thing) If all of your db access is via the ORM
layer, there is absolutely zero actual benefit.

I think the real key is to have a config option to tell sqlalchemy to
not, even if we're running innodb, add the foreign keys to the DDL sent
to the database. If sqlalchemy doesn't have that ability, we should
write it and contribute it, because anyone using MySQL at scale via
sqlalchemy actually wants the feature, whether they recognize it yet or not.

Monty

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] openstack.common setup code

[Monty Taylor]

Hey guys,

Quick follow up from the summit on things that should happen in projects
from the setup module of openstack common as I understand it. (to make
sure we're all on the same page)

There are currently 5 essential things in openstack.common.setup:

parse_requirements
parse_dependency_links
write_requirements
write_git_changelog
generate_authors

that are being used to varying levels in the various projects. What
should happen at this point is this:

parse_requirements
parse_dependency_links

Should be in all of the client libraries and should be removed from all
the not-client libraries. These are essential for pip installation of
client libs (which is important) as they allow pip to follow the
depends. The make things hard for non-client libs, as setuptools doesn't
understand git urls, which we use in non-client lib pip-requires files.

write_requirements

Should die everywhere. It was an attempt to record in our tarballs the
state of what was actually tested ... but did not actually provide
benefit to anyone - and the distros hate it.

write_git_changelog
generate_authors

Should be added/used everywhere. When generate_authors is added, unit
tests testing authors content should be removed.

Is this how everyone else understood the outcome of conversations at the
summit too?

Thanks!
Monty

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [OpenStack][Nova] Minimum required code coverage per file

[Monty Taylor]

Hey - funny story - in responding to Justin I re-read the original email
and realized it was asking for a static low number, which we _can_ do -
at least project-wide. We can't do per-file yet, nor can we fail on a
downward inflection... and I've emailed Justin about that.

If we have consensus on gating on project-wide threshold, I can
certainly add adding that to the gate to the todo list. (If we decide to
do that, I'd really like to make that be openstack-wide rather than just
nova... although I imagine it might take a few weeks to come to
consensus on what the project-wide low number should be.

Current numbers on project-wide lines numbers:

nova: 79%
glance: 75%
keystone: 81%
swift: 80%
horizon: 91%

Perhaps we get nova and glance up to 80 and then set the threshold for 80?

Also, turns out we're not running this on the client libs...

Monty

On 04/25/2012 03:53 PM, Justin Santa Barbara wrote:
> If you let me know in a bit more detail what you're looking for, I can
> probably whip something up.  Email me direct?
> 
> Justin
> 
> 
> On Wed, Apr 25, 2012 at 6:59 AM, Monty Taylor  <mailto:mord...@inaugust.com>> wrote:
> 
> 
> 
> On 04/24/2012 10:08 PM, Lorin Hochstein wrote:
> >
> > On Apr 24, 2012, at 4:11 PM, Joe Gordon wrote:
> >
> >> Hi All,
> >>
> >> I would like to propose a minimum required code coverage level per
> >> file in Nova.  Say 80%.  This would mean that any new feature/file
> >> should only be accepted if it has over 80% code coverage.  Exceptions
> >> to this rule would be allowed for code that is covered by skipped
> >> tests (as long as 80% is reached when the tests are not skipped).
> >>
> >
> > I like the idea of looking at code coverage numbers. For any
> particular
> > merge proposal, I'd also like to know whether it increases or
> decreases
> > the overall code coverage of the project. I don't think we should gate
> > on this, but it would be helpful for a reviewer to see that,
> especially
> > for larger proposals.
> 
> Yup... Nati requested this a couple of summits ago - main issue is that
> while we run code coverage and use the jenkins code coverage plugin to
> track the coverage numbers, the plugin doesn't fully support this
> particular kind of report.
> 
> HOWEVER - if any of our fine java friends out there want to chat with me
> about adding support to the jenkins code coverage plugin to track and
> report this, I will be thrilled to put it in as a piece of reported
> information.
> 
> >> With 193 python files in nova/tests, Nova unit tests produce 85%
> >> overall code coverage (calculated with ./run_test.sh -c [1]).
>  But 23%
> >> of files (125 files) have lower then 80% code coverage (30 tests
> >> skipped on my machine).  Getting all files to hit the 80% code
> >> coverage mark should be one of the goals for Folsom.
> >>
> >
> > I would really like to see a visualization of the code coverage
> > distribution, in order to help spot the outliers.
> >
> >
> > Along these lines, there's been a lot of work in the software
> > engineering research community about predicting which parts of the
> code
> > are most likely to contain bugs ("fault prone" is a good keyword
> to find
> > this stuff, e.g.: http://scholar.google.com/scholar?q=fault+prone, big
> > names include Nachi Nagappan at MS Research and Elaine Weyuker,
> formerly
> > of AT&T Research). I would *love* to see some academic researchers try
> > to apply those techniques to OpenStack to help guide QA activities by
> > identifying which parts of the code should get more rigorous  testing
> > and review.
> 
> ++
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> <mailto:openstack@lists.launchpad.net>
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
> 
> 

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [OpenStack][Nova] Minimum required code coverage per file

[Monty Taylor]

On 04/24/2012 10:08 PM, Lorin Hochstein wrote:
> 
> On Apr 24, 2012, at 4:11 PM, Joe Gordon wrote:
> 
>> Hi All,
>>
>> I would like to propose a minimum required code coverage level per
>> file in Nova.  Say 80%.  This would mean that any new feature/file
>> should only be accepted if it has over 80% code coverage.  Exceptions
>> to this rule would be allowed for code that is covered by skipped
>> tests (as long as 80% is reached when the tests are not skipped).
>>
> 
> I like the idea of looking at code coverage numbers. For any particular
> merge proposal, I'd also like to know whether it increases or decreases
> the overall code coverage of the project. I don't think we should gate
> on this, but it would be helpful for a reviewer to see that, especially
> for larger proposals.

Yup... Nati requested this a couple of summits ago - main issue is that
while we run code coverage and use the jenkins code coverage plugin to
track the coverage numbers, the plugin doesn't fully support this
particular kind of report.

HOWEVER - if any of our fine java friends out there want to chat with me
about adding support to the jenkins code coverage plugin to track and
report this, I will be thrilled to put it in as a piece of reported
information.

>> With 193 python files in nova/tests, Nova unit tests produce 85%
>> overall code coverage (calculated with ./run_test.sh -c [1]).  But 23%
>> of files (125 files) have lower then 80% code coverage (30 tests
>> skipped on my machine).  Getting all files to hit the 80% code
>> coverage mark should be one of the goals for Folsom.
>>
> 
> I would really like to see a visualization of the code coverage
> distribution, in order to help spot the outliers. 
> 
> 
> Along these lines, there's been a lot of work in the software
> engineering research community about predicting which parts of the code
> are most likely to contain bugs ("fault prone" is a good keyword to find
> this stuff, e.g.: http://scholar.google.com/scholar?q=fault+prone, big
> names include Nachi Nagappan at MS Research and Elaine Weyuker, formerly
> of AT&T Research). I would *love* to see some academic researchers try
> to apply those techniques to OpenStack to help guide QA activities by
> identifying which parts of the code should get more rigorous  testing
> and review. 

++

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] New Gerrit version (and server)

[Monty Taylor]

On 04/13/2012 04:36 AM, Kiall Mac Innes wrote:
> The new gerrit version also supports per user namespaces, if enabled.
> 
> These allow users to create private branches with full push privileges etc..
> 
> Have these been enabled?

They have not - we need to verify what the behavior looks like over the
event interface so that we're not all of a sudden attempting to devstack
a bunch of in-progress stuff without meaning to.

We'll put it on the list of new things to investigate our possible use of!

Monty

> On Apr 13, 2012 12:33 p.m., "Thierry Carrez"  > wrote:
> 
> Mark McLoughlin wrote:
> >> One new addition in 2.3 is "draft changes".  The idea behind a draft
> >> change in Gerrit is that it is a change that is not ready for
> merging,
> >> or even general code review, but you would like to share it with some
> >> people to get early comments.  If you upload a change as a draft, by
> >> default, no one else can see it.  You must explicitly add each person
> >> you would like to share it with as a reviewer.  Reviewers you add can
> >> leave comments, but can not vote at this stage.  You can continue to
> >> upload new patchsets to the change as it evolves, and once it is
> ready
> >> for general review, you can click the "Publish" button.  It will then
> >> become a normal change in Gerrit that everyone can see, including the
> >> earlier reviews from the draft stage.  This is a one way transition;
> >> once a draft is published, it can't be made a draft again.
> >
> > This sounds cool. Will the vulnerability management team use this for
> > embargoed security fixes?
> 
> I'll definitely look into that possibility... Depends a bit on the
> security model around drafts (not being listed is not the same as being
> private).
> 
> --
> Thierry Carrez (ttx)
> Release Manager, OpenStack
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> 
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
> 
> 
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp