date:20150224

On 24 February 2015 at 01:07, Salvatore Orlando  wrote:
> Lazy-Stacker summary:
...
> In the medium term, there are a few things we might consider for Neutron's
> "built-in IPAM".
> 1) Move the allocation logic out of the driver, thus making IPAM an
> independent service. The API workers will then communicate with the IPAM
> service through a message bus, where IP allocation requests will be
> "naturally serialized"
> 2) Use 3-party software as dogpile, zookeeper but even memcached to
> implement distributed coordination. I have nothing against it, and I reckon
> Neutron can only benefit for it (in case you're considering of arguing that
> "it does not scale", please also provide solid arguments to support your
> claim!). Nevertheless, I do believe API request processing should proceed
> undisturbed as much as possible. If processing an API requests requires
> distributed coordination among several components then it probably means
> that an asynchronous paradigm is more suitable for that API request.

So data is great. It sounds like as long as we have an appropriate
retry decorator in place, that write locks are better here, at least
for up to 30 threads. But can we trust the data?

One thing I'm not clear on is the SQL statement count.  You say 100
queries for A-1 with a time on Galera of 0.06*1.2=0.072 seconds per
allocation ? So is that 2 queries over 50 allocations over 20 threads?

I'm not clear on what the request parameter in the test json files
does, and AFAICT your threads each do one request each. As such I
suspect that you may be seeing less concurrency - and thus contention
- than real-world setups where APIs are deployed to run worker
processes in separate processes and requests are coming in
willy-nilly. The size of each algorithms workload is so small that its
feasible to imagine the thread completing before the GIL bytecount
code trigger (see
https://docs.python.org/2/library/sys.html#sys.setcheckinterval) and
the GIL's lack of fairness would exacerbate that.

If I may suggest:
 - use multiprocessing or some other worker-pool approach rather than threads
 - or set setcheckinterval down low (e.g. to 20 or something)
 - do multiple units of work (in separate transactions) within each
worker, aim for e.g. 10 seconds or work or some such.
 - log with enough detail that we can report on the actual concurrency
achieved. E.g. log the time in us when each transaction starts and
finishes, then we can assess how many concurrent requests were
actually running.

If the results are still the same - great, full steam ahead. If not,
well lets revisit :)

-Rob


-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] (RE: Change in openstack/neutron-specs[master]: Introducing Tap-as-a-Service)

2015-02-24 Thread Kyle Mestery

There is a -2 (from me). And this was done from the auto-abandon script
which I try to run once a month.

As Kevin said, the suggestion multiple times was to do a StackForge project
for this work, that's the best way forward here.

On Tue, Feb 24, 2015 at 5:01 PM, CARVER, PAUL  wrote:

> Maybe I'm misreading review.o.o, but I don't see the -2. There was a -2
> from Salvatore Orlando with the comment "The -2 on this patch is only to
> deter further comments" and a link to 140292, but 140292 has a comment from
> Kyle saying it's been abandoned in favor of going back to 96149. Are we in
> a loop here?
>
> We're moving forward internally with proprietary mechanisms for attaching
> analyzers but it sure would be nice if there were a standard API. Anybody
> who thinks switches don't need SPAN/mirror ports has probably never working
> in Operations on a real production network where SLAs were taken seriously
> and enforced.
>
> I know there's been a lot of heated discussion around this spec for a
> variety of reasons, but there isn't an enterprise class hardware switch on
> the market that doesn't support SPAN/mirror. Lack of this capability is a
> glaring omission in Neutron that keeps Operations type folks opposed to
> using it because it causes them to lose visibility that they've had for
> ages. We're getting a lot of pressure to continue deploying hardware
> analyzers and/or deploy non-OpenStack mechanisms for implementing
> tap/SPAN/mirror capability when I'd much rather integrate the analyzers
> into OpenStack.
>
>
> -Original Message-
> From: Kyle Mestery (Code Review) [mailto:rev...@openstack.org]
> Sent: Tuesday, February 24, 2015 17:37
> To: vinay yadhav
> Cc: CARVER, PAUL; Marios Andreou; Sumit Naiksatam; Anil Rao; Carlos
> Gonçalves; YAMAMOTO Takashi; Ryan Moats; Pino de Candia; Isaku Yamahata;
> Tomoe Sugihara; Stephen Wong; Kanzhe Jiang; Bao Wang; Bob Melander;
> Salvatore Orlando; Armando Migliaccio; Mohammad Banikazemi; mark mcclain;
> Henry Gessau; Adrian Hoban; Hareesh Puthalath; Subrahmanyam Ongole; Fawad
> Khaliq; Baohua Yang; Maruti Kamat; Stefano Maffulli 'reed'; Akihiro Motoki;
> ijw-ubuntu; Stephen Gordon; Rudrajit Tapadar; Alan Kavanagh; Zoltán Lajos
> Kis
> Subject: Change in openstack/neutron-specs[master]: Introducing
> Tap-as-a-Service
>
> Kyle Mestery has abandoned this change.
>
> Change subject: Introducing Tap-as-a-Service
> ..
>
>
> Abandoned
>
> This review is > 4 weeks without comment and currently blocked by a core
> reviewer with a -2. We are abandoning this for now. Feel free to reactivate
> the review by pressing the restore button and contacting the reviewer with
> the -2 on this review to ensure you address their concerns.
>
> --
> To view, visit https://review.openstack.org/96149
> To unsubscribe, visit https://review.openstack.org/settings
>
> Gerrit-MessageType: abandon
> Gerrit-Change-Id: I087d9d2a802ea39c02259f17d2b8c4e2f6d8d714
> Gerrit-PatchSet: 8
> Gerrit-Project: openstack/neutron-specs
> Gerrit-Branch: master
> Gerrit-Owner: vinay yadhav 
> Gerrit-Reviewer: Adrian Hoban 
> Gerrit-Reviewer: Akihiro Motoki 
> Gerrit-Reviewer: Alan Kavanagh 
> Gerrit-Reviewer: Anil Rao 
> Gerrit-Reviewer: Armando Migliaccio 
> Gerrit-Reviewer: Bao Wang 
> Gerrit-Reviewer: Baohua Yang 
> Gerrit-Reviewer: Bob Melander 
> Gerrit-Reviewer: Carlos Gonçalves 
> Gerrit-Reviewer: Fawad Khaliq 
> Gerrit-Reviewer: Hareesh Puthalath 
> Gerrit-Reviewer: Henry Gessau 
> Gerrit-Reviewer: Isaku Yamahata 
> Gerrit-Reviewer: Jenkins
> Gerrit-Reviewer: Kanzhe Jiang 
> Gerrit-Reviewer: Kyle Mestery 
> Gerrit-Reviewer: Marios Andreou 
> Gerrit-Reviewer: Maruti Kamat 
> Gerrit-Reviewer: Mohammad Banikazemi 
> Gerrit-Reviewer: Paul Carver 
> Gerrit-Reviewer: Pino de Candia 
> Gerrit-Reviewer: Rudrajit Tapadar 
> Gerrit-Reviewer: Ryan Moats 
> Gerrit-Reviewer: Salvatore Orlando 
> Gerrit-Reviewer: Stefano Maffulli 'reed' 
> Gerrit-Reviewer: Stephen Gordon 
> Gerrit-Reviewer: Stephen Wong 
> Gerrit-Reviewer: Subrahmanyam Ongole 
> Gerrit-Reviewer: Sumit Naiksatam 
> Gerrit-Reviewer: Tomoe Sugihara 
> Gerrit-Reviewer: Welcome, new contributor!
> Gerrit-Reviewer: YAMAMOTO Takashi 
> Gerrit-Reviewer: Zoltán Lajos Kis 
> Gerrit-Reviewer: ijw-ubuntu 
> Gerrit-Reviewer: mark mcclain 
> Gerrit-Reviewer: vinay yadhav 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] ML2 versus core plugin for OVN

2015-02-24 Thread Amit Kumar Saha (amisaha)

Hi,

I am new to OpenStack (and am particularly interested in networking). I am 
getting a bit confused by this discussion. Aren’t there already a few 
monolithic plugins (that is what I could understand from reading the Networking 
chapter of the OpenStack Cloud Administrator Guide. Table 7.3 Available 
networking plugi-ins)? So how do we have interoperability between those (or do 
we not intend to)?

BTW, it is funny that the acronym ML can also be used for “monolithic” ☺

Regards,
Amit Saha
Cisco, Bangalore



From: Sukhdev Kapur [mailto:sukhdevka...@gmail.com]
Sent: Wednesday, February 25, 2015 6:04 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] ML2 versus core plugin for OVN

Folks,

A great discussion. I am not expert at OVN, hence, want to ask a question. The 
answer may make a  case that it should probably be a ML2 driver as oppose to 
monolithic plugin.

Say a customer want to deploy an OVN based solution and use HW devices from one 
vendor for L2 and L3 (e.g. Arista or Cisco), and want to use another vendor for 
services (e.g. F5 or A10) - how can that be supported?

If OVN goes in as ML2 driver, I can then run ML2 and Service plugin to achieve 
above solution. For a monolithic plugin, don't I have an issue?

regards..
-Sukhdev


On Tue, Feb 24, 2015 at 8:58 AM, Salvatore Orlando 
mailto:sorla...@nicira.com>> wrote:
I think we're speculating a lot about what would be best for OVN whereas we 
should probably just expose pro and cons of ML2 drivers vs standalone plugin 
(as I said earlier on indeed it does not necessarily imply monolithic *)

I reckon the job of the Neutron community is to provide a full picture to OVN 
developers - so that they could make a call on the integration strategy that 
best suits them.
On the other hand, if we're planning to commit to a model where ML2 is not 
anymore a plugin but the interface with the API layer, then any choice which is 
not a ML2 driver does not make any sense. Personally I'm not sure we ever want 
to do that, at least not in the near/medium term, but I'm one and hardly 
representative of the developer/operator communities.

Salvatore


* In particular with the advanced service split out the term monolithic simply 
does not mean anything anymore.

On 24 February 2015 at 17:48, Robert Kukura 
mailto:kuk...@noironetworks.com>> wrote:
Kyle, What happened to the long-term potential goal of ML2 driver APIs becoming 
neutron's core APIs? Do we really want to encourage new monolithic plugins?

ML2 is not a control plane - its really just an integration point for control 
planes. Although co-existence of multiple mechanism drivers is possible, and 
sometimes very useful, the single-driver case is fully supported. Even with 
hierarchical bindings, its not really ML2 that controls what happens - its the 
drivers within the framework. I don't think ML2 really limits what drivers can 
do, as long as a virtual network can be described as a set of static and 
possibly dynamic network segments. ML2 is intended to impose as few constraints 
on drivers as possible.

My recommendation would be to implement an ML2 mechanism driver for OVN, along 
with any needed new type drivers or extension drivers. I believe this will 
result in a lot less new code to write and maintain.

Also, keep in mind that even if multiple driver co-existence doesn't sound 
immediately useful, there are several potential use cases to consider. One is 
that it allows new technology to be introduced into an existing cloud alongside 
what previously existed. Migration from one ML2 driver to another may be a lot 
simpler (and/or flexible) than migration from one plugin to another. Another is 
that additional drivers can support special cases, such as bare metal, 
appliances, etc..

-Bob

On 2/24/15 11:11 AM, Kyle Mestery wrote:
On Tue, Feb 24, 2015 at 3:19 AM, Salvatore Orlando 
mailto:sorla...@nicira.com>> wrote:
On 24 February 2015 at 01:34, Kyle Mestery 
mailto:mest...@mestery.com>> wrote:
Russel and I have already merged the initial ML2 skeleton driver [1].
The thinking is that we can always revert to a non-ML2 driver if needed.

If nothing else an authoritative decision on a design direction saves us the 
hassle of going through iterations and discussions.
The integration through ML2 is definitely viable. My opinion however is that 
since OVN implements a full control plane, the control plane bits provided by 
ML2 are not necessary, and a plugin which provides only management layer 
capabilities might be the best solution. Note: this does not mean it has to be 
monolithic. We can still do L3 with a service plugin.
However, since the same kind of approach has been adopted for ODL I guess this 
provides some sort of validation.

To be honest, after thinking about this last night, I'm now leaning towards 
doing this as a full plugin. I don't really envision OVN running with other 
plugins, as OVN is implementing it's own control plane, as

Re: [openstack-dev] ECMP on Neutron virtual router

2015-02-24 Thread henry hly

On Wed, Feb 25, 2015 at 3:11 AM, Kevin Benton  wrote:
> I wonder if there is a way we can easily abuse the extra routes extension to
> do this? Maybe two routes to the same network would imply ECMP.
>

It's a good idea, and we deploy a system with similar concept(by extra
routes) by a tiny patch on existing neutron L3 plugin and agent code.

> If not, maybe this can fit into a larger refactoring for route management
> (dynamic routing, etc).
>
> On Feb 24, 2015 11:02 AM, "Carl Baldwin"  wrote:
>>
>> It doesn't support this at this time.  There are no current plans to
>> make it work.  I'm curious to know how you would like for this to work
>> in your deployment.
>>
>> Carl
>>
>> On Tue, Feb 24, 2015 at 11:32 AM, NAPIERALA, MARIA H 
>> wrote:
>> > Does Neutron router support ECMP across multiple static routes to the
>> > same
>> > destination network but with different next-hops?
>> >
>> > Maria
>> >
>> >
>> >
>> > __
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe:
>> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo] Graduating oslo.reports: Request to review "clean copy"

2015-02-24 Thread Solly Ross

 
> Those scripts have mostly moved into oslotest, so they don't need to be
> synced any more to be used. If you have all of your code, and it follows
> the cookiecutter template, we can look at it and propose post-import
> tweaks. What's the URL for the repository?

Heh, whoops.  I should probably have included that.  It's at
https://github.com/directxman12/oslo.reports

Thanks!

Best Regards,
Solly

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Kerberos in OpenStack

2015-02-24 Thread Sanket Lawangare

Thanks a lot for taking out time and replying back Tim. Will let you know
if i have any further questions.

On Tue, Feb 24, 2015 at 1:22 PM, Tim Bell  wrote:

>  You may also get some information from how we set up Kerberos at CERN at
> http://openstack-in-production.blogspot.fr/2014/10/kerberos-and-single-sign-on-with.html
>
>
>
> From my understanding, the only connection is between Keystone and KDC.
> There is a standard Keystone token issues based off the Kerberos ticket and
> the rest is the same as if a password had been supplied.
>
>
>
> Tim
>
>
>
> *From:* Sanket Lawangare [mailto:sanket.lawang...@gmail.com]
> *Sent:* 24 February 2015 19:53
> *To:* openstack-dev@lists.openstack.org
> *Subject:* [openstack-dev] Kerberos in OpenStack
>
>
>
> Hello  Everyone,
>
>
>
> My name is Sanket Lawangare. I am a graduate Student studying at The
> University of Texas, at San Antonio.* For my Master’s Thesis I am working
> on the Identity component of OpenStack. My research is to investigate
> external authentication with Identity(keystone) using Kerberos.*
>
>
>
> Based on reading Jammie lennox's Blogs on Kerberos implementation in
> OpenStack and my understanding of Kerberos I have come up with a figure
> explaining possible interaction of KDC with the OpenStack client, keystone
> and the OpenStack services(Nova, Cinder, Swift...).
>
> These are the Blogs -
>
>
> http://www.jamielennox.net/blog/2015/02/12/step-by-step-kerberized-keystone/
>
> http://www.jamielennox.net/blog/2013/10/22/keystone-token-binding/
>
> I am trying to understand the working of Kerberos in OpenStack.
>
>
>
> Please click this link to view the figure:
> https://docs.google.com/drawings/d/1re0lNbiMDTbnkrqGMjLq6oNoBtR_GA0x7NWacf0Ulbs/edit?usp=sharing
>
>
>
> P.S. - [The steps in this figure are self explanatory the basic
> understanding of Kerberos is expected]
>
>
>
> Based on the figure i had couple of questions:
>
>
>
> 1. Is Nova or other services registered with the KDC?
>
>
>
> 2. What does keystone do with Kerberos ticket/credentials? Does
> Keystone authenticates the users and gives them direct access to other
> services such as Nova, Swift etc..
>
>
>
> 3. After receiving the Ticket from the KDC does keystone embed some
> kerberos credential information in the token?
>
>
>
> 4. What information does the service (e.g.Nova) see in the Ticket and
> the token (Does the token have some kerberos info or some customized info
> inside it?).
>
>
>
> If you could share your insights and guide me on this. I would be really
> appreciate it. Thank you all for your time.
>
>
>
> Regards,
>
> Sanket Lawangare
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [devstack] [Cinder-GlusterFS CI] centos7 gate job abrupt failures

2015-02-24 Thread Jeremy Stanley

On 2015-02-25 01:02:07 +0530 (+0530), Bharat Kumar wrote:
[...]
> After running 971 test cases VM inaccessible for 569 ticks
[...]

Glad you're able to reproduce it. For the record that is running
their 8GB performance flavor with a CentOS 7 PVHVM base image. The
steps to recreate are http://paste.openstack.org/show/181303/ as
discussed in IRC (for the sake of others following along). I've held
a similar worker in HPCloud (15.126.235.20) which is a 30GB flavor
artifically limited to 8GB through a kernel boot parameter.
Hopefully following the same steps there will help either confirm
the issue isn't specific to running in one particular service
provider, or will yield some useful difference which could help
highlight the cause.

Either way, once 104.239.136.99 and 15.126.235.20 are no longer
needed, please let one of the infrastructure root admins know to
delete them.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] ML2 versus core plugin for OVN

2015-02-24 Thread Sukhdev Kapur

Folks,

A great discussion. I am not expert at OVN, hence, want to ask a question.
The answer may make a  case that it should probably be a ML2 driver as
oppose to monolithic plugin.

Say a customer want to deploy an OVN based solution and use HW devices from
one vendor for L2 and L3 (e.g. Arista or Cisco), and want to use another
vendor for services (e.g. F5 or A10) - how can that be supported?

If OVN goes in as ML2 driver, I can then run ML2 and Service plugin to
achieve above solution. For a monolithic plugin, don't I have an issue?

regards..
-Sukhdev


On Tue, Feb 24, 2015 at 8:58 AM, Salvatore Orlando 
wrote:

> I think we're speculating a lot about what would be best for OVN whereas
> we should probably just expose pro and cons of ML2 drivers vs standalone
> plugin (as I said earlier on indeed it does not necessarily imply
> monolithic *)
>
> I reckon the job of the Neutron community is to provide a full picture to
> OVN developers - so that they could make a call on the integration strategy
> that best suits them.
> On the other hand, if we're planning to commit to a model where ML2 is not
> anymore a plugin but the interface with the API layer, then any choice
> which is not a ML2 driver does not make any sense. Personally I'm not sure
> we ever want to do that, at least not in the near/medium term, but I'm one
> and hardly representative of the developer/operator communities.
>
> Salvatore
>
>
> * In particular with the advanced service split out the term monolithic
> simply does not mean anything anymore.
>
> On 24 February 2015 at 17:48, Robert Kukura 
> wrote:
>
>>  Kyle, What happened to the long-term potential goal of ML2 driver APIs
>> becoming neutron's core APIs? Do we really want to encourage new monolithic
>> plugins?
>>
>> ML2 is not a control plane - its really just an integration point for
>> control planes. Although co-existence of multiple mechanism drivers is
>> possible, and sometimes very useful, the single-driver case is fully
>> supported. Even with hierarchical bindings, its not really ML2 that
>> controls what happens - its the drivers within the framework. I don't think
>> ML2 really limits what drivers can do, as long as a virtual network can be
>> described as a set of static and possibly dynamic network segments. ML2 is
>> intended to impose as few constraints on drivers as possible.
>>
>> My recommendation would be to implement an ML2 mechanism driver for OVN,
>> along with any needed new type drivers or extension drivers. I believe this
>> will result in a lot less new code to write and maintain.
>>
>> Also, keep in mind that even if multiple driver co-existence doesn't
>> sound immediately useful, there are several potential use cases to
>> consider. One is that it allows new technology to be introduced into an
>> existing cloud alongside what previously existed. Migration from one ML2
>> driver to another may be a lot simpler (and/or flexible) than migration
>> from one plugin to another. Another is that additional drivers can support
>> special cases, such as bare metal, appliances, etc..
>>
>> -Bob
>>
>>
>> On 2/24/15 11:11 AM, Kyle Mestery wrote:
>>
>>  On Tue, Feb 24, 2015 at 3:19 AM, Salvatore Orlando 
>> wrote:
>>
>>>  On 24 February 2015 at 01:34, Kyle Mestery  wrote:
>>>
  Russel and I have already merged the initial ML2 skeleton driver [1].

>>>   The thinking is that we can always revert to a non-ML2 driver if
 needed.

>>>
>>>  If nothing else an authoritative decision on a design direction saves
>>> us the hassle of going through iterations and discussions.
>>> The integration through ML2 is definitely viable. My opinion however is
>>> that since OVN implements a full control plane, the control plane bits
>>> provided by ML2 are not necessary, and a plugin which provides only
>>> management layer capabilities might be the best solution. Note: this does
>>> not mean it has to be monolithic. We can still do L3 with a service plugin.
>>>  However, since the same kind of approach has been adopted for ODL I
>>> guess this provides some sort of validation.
>>>
>>>
>> To be honest, after thinking about this last night, I'm now leaning
>> towards doing this as a full plugin. I don't really envision OVN running
>> with other plugins, as OVN is implementing it's own control plane, as you
>> say. So the value of using ML2 is quesitonable.
>>
>>
>>>I'm not sure how useful having using OVN with other drivers will be,
 and that was my initial concern with doing ML2 vs. full plugin. With the HW
 VTEP support in OVN+OVS, you can tie in physical devices this way. Anyways,
 this is where we're at for now. Comments welcome, of course.

>>>
>>>  That was also kind of my point regarding the control plane bits
>>> provided by ML2 which OVN does not need. Still, the fact that we do not use
>>> a function does not make any harm.
>>> Also i'm not sure if OVN needs at all a type manager. If not, we can
>>> always implement a no-op type manager,

Re: [openstack-dev] [all] Re-evaluating the suitability of the 6 month release cycle

On 25 February 2015 at 13:13, Jeremy Stanley  wrote:
> On 2015-02-24 11:27:05 + (+), Daniel P. Berrange wrote:
> [...]
>> It would be reasonable for the vulnerability team to take the decision
>> that they'll support fixes for master, and any branches that the stable
>> team decide to support.
> [...]
>
> Well, it's worth noting that the VMT doesn't even "support" (i.e.
> issue advisories for bugs in) master branches now, the exception
> being branchless projects where the bug appears in master prior to
> an existing release tag.
>
> But I think Thierry's earlier point is that as soon as you start
> marking _some_ releases as special (supported by VMT, stable maint,
> docs, translators...) then those become your new actual releases and
> the other interim releases become your milestones, and we're back to
> the current model again.

I don't think thats true actually. We'd still have a major smoothing
effect on work, which means less high peaks at release time and less
troughs at 'review direction' time and so on.

-Rob


-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Ironic] Stepping down from Ironic Core

Like with TripleO, I've not been pulling my weight as a core reviewer
for a bit; I'd be very hesitant to +A in the project as a result.

I'm still very interested in Ironic, but its not where my immediate focus is.

Cheers,
Rob

-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Re-evaluating the suitability of the 6 month release cycle

2015-02-24 Thread Jeremy Stanley

On 2015-02-24 11:27:05 + (+), Daniel P. Berrange wrote:
[...]
> It would be reasonable for the vulnerability team to take the decision
> that they'll support fixes for master, and any branches that the stable
> team decide to support.
[...]

Well, it's worth noting that the VMT doesn't even "support" (i.e.
issue advisories for bugs in) master branches now, the exception
being branchless projects where the bug appears in master prior to
an existing release tag.

But I think Thierry's earlier point is that as soon as you start
marking _some_ releases as special (supported by VMT, stable maint,
docs, translators...) then those become your new actual releases and
the other interim releases become your milestones, and we're back to
the current model again.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Re-evaluating the suitability of the 6 month release cycle

On 25 February 2015 at 00:05, Thierry Carrez  wrote:
> Daniel P. Berrange wrote:
>> [...]

>
> I think you're judging the cycle from the perspective of developers
> only.

You said that in the other thread, and I think its false. There is a
very good case to be made that the release cycle is directly
responsible for:
 - increased feature latency [when users get new shiny] - already
explained here.
 - decreased stability due to feature crush
 - decreased quality for the same reason - we don't put things in and
bake them before exposing, we get them in and since there is a long
gap before they can be consumed they're generally just enabled
immediately. Keystone's steps to address that are excellent, but made
harder by the gap between releases: there is social pressure to make
things graduate to support within each release.

>  6 months was not an arbitrary decision. Translations and
> documentation teams basically need a month of feature/string freeze in
> order to complete their work. Since we can't reasonably freeze one month
> every 2 months, we picked 6 months.

Do they need a month because 6 months worth of changes have been
aggregated? E.g. if they have 2 months of changes to deal with, do
they need 1.5 weeks?

Perhaps we could branch the code release in advance of docs and
translations, and announce the release when those artifacts are ready?

> It's also worth noting that we were on a 3-month cycle at the start of
> OpenStack. That was dropped after a cataclysmic release that managed the
> feat of (a) not having anything significant done, and (b) have out of
> date documentation and translations.

Oh! 
https://wiki.openstack.org/w/index.php?title=Obsolete:3MonthReleaseCycle&direction=prev&oldid=14016
appears to be the page about that, but my google-fu isn't turning up a
post-mortem of the C release which prompted the change. Perhaps some
old-hand like you can fill us in on the details :).

> Stable branches may have the luxury of skipping releases and designate a
> "stable" one from time to time (I reject the Linux comparison because
> the kernel is at a very different moment in software lifecycle). The
> trick being, making one release "special" is sure to recreate the peak
> issues you're trying to solve.
>
> I'm actually more concerned about the teams that don't have the luxury
> of skipping a release (like the vulnerability management team). Docs and
> translations, as mentioned above, will also have a hard time producing
> quality docs and translations every 2 months with a very short freeze
> period.

I agree that the vulnerability team will be rather more impacted by
this than regular project teams - I made a nod to that in the other
thread. More thought needed still :).

> I may appear defensive on my answers, but it's not my goal to defend the
> current system: it's just that most of those proposals generally ignore
> the diversity of the needs of the teams that make OpenStack possible, to
> focus on a particular set of contributors' woes. I'm trying to bring
> that wider perspective in -- the current system is a trade-off and the
> result of years of evolution, not an arbitrary historic choice that we
> can just change at will.

I agree that its not arbitrary and that changing it requires some
appropriate wide-spread consultation; OTOH the benefits of rolling and
higher frequency releases are really substantial: but we have to
backup the change with the appropriate engineering (such as reducing
our frictions that cause teams practicing CD to be weeks behind tip)
to make it feasible. My greatest concern about the proposals happening
now is that we may bite off more than we can chew. OTGH the reality is
that all the negative things multiply out, so we probably need to just
start somewhere and /do/ to give us the space to fix other things.

-Rob

-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] (RE: Change in openstack/neutron-specs[master]: Introducing Tap-as-a-Service)

2015-02-24 Thread Kevin Benton

I think Kyle's auto-abandon script made a mistake in this case, unless it
was seeing into the future and saw it's own -2... :)

More seriously, have you considered starting a tap-as-a-service project on
stackforge now that the services split has established a framework for
advanced services? Uploading the code you are using to do it is a great way
to get people motivated to try it, propose new features, critique it, etc.
If you can't upload it because your approach would be proprietary, then
would upstream support even be relevant?

On Tue, Feb 24, 2015 at 3:01 PM, CARVER, PAUL  wrote:

> Maybe I'm misreading review.o.o, but I don't see the -2. There was a -2
> from Salvatore Orlando with the comment "The -2 on this patch is only to
> deter further comments" and a link to 140292, but 140292 has a comment from
> Kyle saying it's been abandoned in favor of going back to 96149. Are we in
> a loop here?
>
> We're moving forward internally with proprietary mechanisms for attaching
> analyzers but it sure would be nice if there were a standard API. Anybody
> who thinks switches don't need SPAN/mirror ports has probably never working
> in Operations on a real production network where SLAs were taken seriously
> and enforced.
>
> I know there's been a lot of heated discussion around this spec for a
> variety of reasons, but there isn't an enterprise class hardware switch on
> the market that doesn't support SPAN/mirror. Lack of this capability is a
> glaring omission in Neutron that keeps Operations type folks opposed to
> using it because it causes them to lose visibility that they've had for
> ages. We're getting a lot of pressure to continue deploying hardware
> analyzers and/or deploy non-OpenStack mechanisms for implementing
> tap/SPAN/mirror capability when I'd much rather integrate the analyzers
> into OpenStack.
>
>
> -Original Message-
> From: Kyle Mestery (Code Review) [mailto:rev...@openstack.org]
> Sent: Tuesday, February 24, 2015 17:37
> To: vinay yadhav
> Cc: CARVER, PAUL; Marios Andreou; Sumit Naiksatam; Anil Rao; Carlos
> Gonçalves; YAMAMOTO Takashi; Ryan Moats; Pino de Candia; Isaku Yamahata;
> Tomoe Sugihara; Stephen Wong; Kanzhe Jiang; Bao Wang; Bob Melander;
> Salvatore Orlando; Armando Migliaccio; Mohammad Banikazemi; mark mcclain;
> Henry Gessau; Adrian Hoban; Hareesh Puthalath; Subrahmanyam Ongole; Fawad
> Khaliq; Baohua Yang; Maruti Kamat; Stefano Maffulli 'reed'; Akihiro Motoki;
> ijw-ubuntu; Stephen Gordon; Rudrajit Tapadar; Alan Kavanagh; Zoltán Lajos
> Kis
> Subject: Change in openstack/neutron-specs[master]: Introducing
> Tap-as-a-Service
>
> Kyle Mestery has abandoned this change.
>
> Change subject: Introducing Tap-as-a-Service
> ..
>
>
> Abandoned
>
> This review is > 4 weeks without comment and currently blocked by a core
> reviewer with a -2. We are abandoning this for now. Feel free to reactivate
> the review by pressing the restore button and contacting the reviewer with
> the -2 on this review to ensure you address their concerns.
>
> --
> To view, visit https://review.openstack.org/96149
> To unsubscribe, visit https://review.openstack.org/settings
>
> Gerrit-MessageType: abandon
> Gerrit-Change-Id: I087d9d2a802ea39c02259f17d2b8c4e2f6d8d714
> Gerrit-PatchSet: 8
> Gerrit-Project: openstack/neutron-specs
> Gerrit-Branch: master
> Gerrit-Owner: vinay yadhav 
> Gerrit-Reviewer: Adrian Hoban 
> Gerrit-Reviewer: Akihiro Motoki 
> Gerrit-Reviewer: Alan Kavanagh 
> Gerrit-Reviewer: Anil Rao 
> Gerrit-Reviewer: Armando Migliaccio 
> Gerrit-Reviewer: Bao Wang 
> Gerrit-Reviewer: Baohua Yang 
> Gerrit-Reviewer: Bob Melander 
> Gerrit-Reviewer: Carlos Gonçalves 
> Gerrit-Reviewer: Fawad Khaliq 
> Gerrit-Reviewer: Hareesh Puthalath 
> Gerrit-Reviewer: Henry Gessau 
> Gerrit-Reviewer: Isaku Yamahata 
> Gerrit-Reviewer: Jenkins
> Gerrit-Reviewer: Kanzhe Jiang 
> Gerrit-Reviewer: Kyle Mestery 
> Gerrit-Reviewer: Marios Andreou 
> Gerrit-Reviewer: Maruti Kamat 
> Gerrit-Reviewer: Mohammad Banikazemi 
> Gerrit-Reviewer: Paul Carver 
> Gerrit-Reviewer: Pino de Candia 
> Gerrit-Reviewer: Rudrajit Tapadar 
> Gerrit-Reviewer: Ryan Moats 
> Gerrit-Reviewer: Salvatore Orlando 
> Gerrit-Reviewer: Stefano Maffulli 'reed' 
> Gerrit-Reviewer: Stephen Gordon 
> Gerrit-Reviewer: Stephen Wong 
> Gerrit-Reviewer: Subrahmanyam Ongole 
> Gerrit-Reviewer: Sumit Naiksatam 
> Gerrit-Reviewer: Tomoe Sugihara 
> Gerrit-Reviewer: Welcome, new contributor!
> Gerrit-Reviewer: YAMAMOTO Takashi 
> Gerrit-Reviewer: Zoltán Lajos Kis 
> Gerrit-Reviewer: ijw-ubuntu 
> Gerrit-Reviewer: mark mcclain 
> Gerrit-Reviewer: vinay yadhav 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Kevin Benton
_

Re: [openstack-dev] [stable][all] Revisiting the 6 month release cycle

2015-02-24 Thread Jeremy Stanley

On 2015-02-24 10:00:51 -0800 (-0800), Johannes Erdfelt wrote:
[...]
> Recently, I have spent a lot more time waiting on reviews than I
> have spent writing the actual code.

That's awesome, assuming what you mean here is that you've spent
more time reviewing submitted code than writing more. That's where
we're all falling down as a project and should be doing better, so I
applaud your efforts in this area.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Re-evaluating the suitability of the 6 month release cycle

On 24 February 2015 at 22:53, Daniel P. Berrange  wrote:
> I was writing this mail for the past few days, but the nova thread
> today prompted me to finish it off & send it :-)

++

> The first two observations strongly suggest that the choice of 6
> months as a cycle length is a fairly arbitrary decision that can be
> changed without unreasonable pain. The third observation suggests a
> much shorter cycle length would smooth out the bumps and lead to a
> more efficient & satisfying development process for all involved.

I'm very glad to see this being discussed (again :)). Any proposal
that reduces our cycle time is going to get my enthusiastic support.

...
> Upgrades & deprecation
> --
>
> It is common right now for projects to say upgrades are only
> supported between releases N-1 and N. ie to go from Icehouse
> to Kilo, you need to first deploy Juno. This is passable when
> you're talking 6 month gaps between cycles, but when there are
> 2 month gaps it is not reasonable to expect everyone to be
> moving fast enough to keep up with every release. If an
> organization's beurocracy means they can't deploy more often
> than every 12 months, forcing them to deploy the 5 intermediate
> releases to run upgrade scripts is quite unpleasant. We would
> likely have to look at officially allowing upgrades between
> any (N-3, N-2, N-1) to N. From a database POV this should not
> be hard, since the DB migration scripts don't have any built
> in assumptions about this. Similarly the versioned objects used
> by Nova are quite flexible in this regard too, as long as the
> compat code isn't deleted too soon.
>
> Deprecation warnings would need similar consideration. It would
> not be sufficient to deprecate in one release and delete in the
> next. We'd likely want to say that depecations last for a given
> time period rather than number of releases, eg 6 months. This
> could be easily handled by simply including the date of initial
> deprecation in the deprecation message. It would thus be clear
> when the feature will be removed to all involved.

I think a useful thing here is to consider online upgrades vs offline
upgrades. If we care to we could say that online upgrades are only
supported release to release, with offline upgrades being supported by
releases up to 8 months apart. The benefit of this would be to reduce
our test matrix: online upgrades are subject to much greater
interactions between concurrent processes, and its hard enough to
validate that N -> N+1 works with any deep confidence vs also checking
that N->N+2, N->N+3 also work: for a 6 month sliding window to match
the current thing, we need to allow upgrades from Feb through August:
a Feb->Apr
b Feb->Jun
c Feb->Aug
d Apr-> Jun
e Apr->Aug
f Jun->Aug

We'd need to be testing all the combinations leading to the branch a
patch is for, so changes to Aug would need c, e and f all tested.

Thats 3 times the overhead of supporting:
Feb->Apr and then
Apr->Jun and then
Jun->Aug
serially where we'd only be testing one combination at a time.

-Rob

-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] client library release versions

On 25 February 2015 at 11:18, Matt Riedemann  wrote:
>
>

> I was hoping to do a sqlalchemy-migrate release this week so I'm interested
> in not screwing that up. :)
>
> The current release is 0.9.4 and there was one change to requirements.txt,
> cee9136, since then, so if I'm reading this correctly the next version for
> sqlalchemy-migrate should really be 0.10.0.

So as its a 0.* project today, the convention we usually follow is to
right-shift the versions: X.Y.Z -> 0.X.Y, because its not stable.

So, a Y change in such a project would give you 0.9.5. And we say this
is ok because only also no-public-API projects should be depending on
it and we're expecting to break the API lots, and need a way to signal
that (an X equivalent) without first committing to a public API
We're perhaps not following that rule all the well (in
requirements.txt constraints...)

> Regarding public API and 1.x.y, I don't think there is really anything
> holding sqlalchemy-migrate back from that, it's hella old so we should
> probably be 1.0.0 by now.

+1.

-Rob

-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Fuel] Bug statuses definition in Fuel

2015-02-24 Thread Mike Scherbakov

Hi Dmitry,
thanks for extending info on what is different in Fuel for Confirmed & Fix
Released statuses [1]

It is pretty hard to go in history of the page now, but I think I like
original OpenStack Imporance description in [2], than Fuel-specific [3]:

> - Critical = can't deploy anything and there's no trivial workaround; data
> loss; or security vulnerability

- High = specific hardware, configurations, or components are unusable and
> there's no workaround; or everything is broken but there's a workaround
>
So, "can't deploy anything" is Critical. If you can deploy cloud, but it
doesn't work afterwards - is not Critical anymore with current description.
I do not think that it is just High that you can't open Horizon page after
deployment.

Why don't we stick to the original OpenStack criteria?

> - Critical if the bug prevents a key feature from working properly
> (regression) for all users (or without a simple workaround) or result in
> data loss
> - High if the bug prevents a key feature from working properly for some
> users (or with a workaround)



[1]
https://wiki.openstack.org/w/index.php?title=Fuel%2FHow_to_contribute&diff=73079&oldid=72329
[2]
https://wiki.openstack.org/wiki/BugTriage#Task_2:_Prioritize_confirmed_bugs_.28bug_supervisors.29

[3]
https://wiki.openstack.org/wiki/Fuel/How_to_contribute#Confirm_and_triage_bugs

On Thu, Feb 5, 2015 at 8:22 PM, Dmitry Mescheryakov <
dmescherya...@mirantis.com> wrote:

> Guys,
>
> I was looking for a page where bug statuses in out LP projects are
> described and found none. Mike suggested to add this to How To Contribute
> page and so I did. Please take a look at the section [1], just to make sure
> that we are on the same page. The status descriptions are located in the
> second from the top list.
>
> Thanks,
>
> Dmitry
>
> [1]
> https://wiki.openstack.org/wiki/Fuel/How_to_contribute#Confirm_and_triage_bugs
>



-- 
Mike Scherbakov
#mihgen
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [stable][all] Revisiting the 6 month release cycle

2015-02-24 Thread Zane Bitter


On 23/02/15 19:14, Joe Gordon wrote:

Was:
http://lists.openstack.org/pipermail/openstack-dev/2015-February/057578.html

There has been frustration with our current 6 month development cadence.
This is an attempt to explain those frustrations and propose a very
rough outline of a possible alternative.


Currently we follow a 6 month release cadence, with 3 intermediate
milestones (6 weeks apart), as explained here:
https://wiki.openstack.org/wiki/Kilo_Release_Schedule


Current issues

  * 3 weeks of feature freeze for all projects at the end of each cycle
(3 out of the 26 feature blocked)
  * 3 weeks of release candidates. Once a candidate is cut development
is open for next release. While this is good in theory, not much
work actually starts on next release.
  * some projects have non priority feature freezes and at Milestone 2
(so 9 out of 26 weeks restricted in those projects)
  * vicious development circle
  o vicious circle:
  + big push right to land lots of features right before the release
  + a lot of effort is spent getting the release ready
  + after the release people are a little burnt out and take it
easy until the next summit
  + Blueprints have to be re-discussed and re-approved for the
next cycle


I'm sure there's a good reason for this one that I'm not aware of, but 
it certainly sounds a lot like declaring a war of attrition against a 
project's own contributors.


If core teams think that a design that they previously approved needs to 
change because of other changes _they_ approved in the previous release 
cycle or a decision _they_ made during the design summit, the onus 
should be on _them_ to actively make the change (even if that's as 
simple as proposing a patch deleting *a* previously-approved spec - not 
the whole lot en masse). To delete everything by default and force the 
contributor to get it reapproved is tantamount to handing them a shovel 
and forcing them to shift the goalposts themselves. It's flirting with 
the line between poor policy and outright evil.



  + makes it hard to land blueprints early in the cycle casing
the bug rush at the end of the cycle, see step 1
  o Makes it hard to plan things across a release
  o This actually destabilizes things right as we go into the
stabilization period (We actually have great data on this too)
  o Means postponing blueprints that miss a deadline several months


I'm yet to be convinced by the solution offered here, but I do think 
this is a fairly accurate description of the problem. I always tell 
folks that if you want big features to land in the next release, you 
have to start working on them as soon as you can finish up on any 
release blockers and well before Summit. Mostly that month feels like 
dead time, though.


Sometimes I daydream about what it would be like if we had the design 
summit a few weeks _before_ the release instead of after. Usually I snap 
out of it when I remember the downsides, like having developers under 
pressure to fix bugs at the same time as the design summit, or not 
having a shiny new release to trumpet at the conference, or everyone 
getting excited about new feature discussions and not working on urgent 
bugs when they get back. Still, I keep thinking about it...


cheers,
Zane.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][oslo] Dealing with database connection sharing issues

On 23 February 2015 at 13:54, Michael Bayer  wrote:

> Correct me if I'm wrong but the register_after_fork seems to apply only to
> the higher level Process abstraction.   If someone calls os.fork(), as is
> the case now, there's no hook to use.
>
> Hence the solution I have in place right now, which is that Oslo.db *can*
> detect a fork and adapt at the most basic level by checking for os.getpid()
> and recreating the connection, no need for anyone to call engine.dispose()
> anywhere. But that approach has been rejected.  Because the caller of the
> library should be aware they're doing this.
>
> If we can all read the whole thread here each time and be on the same page
> about what is acceptable and what's not, that would help.

I've read the whole thread :).

I don't agree with the rejection you received :(.

Here are my principles in the design:
 - oslo.db is meant to be a good [but opinionated] general purpose db
library: it is by and for OpenStack, but it can only assume as givens
those things which are guaranteed the same for all OpenStack projects,
and which we can guarantee we don't want to change in future.
Everything else it needs to do the usual thing of offering interfaces
and extension points where its behaviour can be modified.
 - failing closed is usually much much better than failing open. Other
libraries and app code may do things oslo.db doesn't expect, and
oslo.db failing in a hard to debug fashion is a huge timewaste for
everyone involved.
 - faults should be trapped as close to the moment that it happened as
possible. That is, at the first sign.
 - correctness is more important than aesthetics : ugly but doing the
right thing is better than looking nice but breaking.
 - where we want to improve things in a program in a way thats
incompatible, we should consider a deprecation period.


Concretely, I think we should do the following:
 - in olso.db today, detect the fork and reopen the connection (so the
users code works); and log a DEBUG/TRACE level message that this is a
deprecated pattern and will be removed.
 - follow that up with patches to all the projects to prevent this
happening at all
 - wait until we're no longer doing security fixes to any branch with
the pre-fixed code
 - at the next major release of oslo.db, change it from deprecated to
hard failure

That gives a graceful migration path and ensures safety.

As to the potential for someone to deliberately:
 - open an oslo.db connection
 - fork
 - expect it to work

I say phoooey. Pre-forking patterns don't need this (it won't use the
connect before work is handed off to the child). Privilege dropping
patterns could potentially use this, but they are rare enough that
they can explicitly close the connection and make a new one after the
fork. In general anything related to fork is going to break and one
should re-establish things after forking. The exceptions are
sufficiently rare that I think we can defer adding apis to support
them (e.g. a way to say 'ok, refresh your cache of the pid now') until
someone actually wants that.

-Rob


-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Neutron] (RE: Change in openstack/neutron-specs[master]: Introducing Tap-as-a-Service)

2015-02-24 Thread CARVER, PAUL

Maybe I'm misreading review.o.o, but I don't see the -2. There was a -2 from 
Salvatore Orlando with the comment "The -2 on this patch is only to deter 
further comments" and a link to 140292, but 140292 has a comment from Kyle 
saying it's been abandoned in favor of going back to 96149. Are we in a loop 
here?

We're moving forward internally with proprietary mechanisms for attaching 
analyzers but it sure would be nice if there were a standard API. Anybody who 
thinks switches don't need SPAN/mirror ports has probably never working in 
Operations on a real production network where SLAs were taken seriously and 
enforced.

I know there's been a lot of heated discussion around this spec for a variety 
of reasons, but there isn't an enterprise class hardware switch on the market 
that doesn't support SPAN/mirror. Lack of this capability is a glaring omission 
in Neutron that keeps Operations type folks opposed to using it because it 
causes them to lose visibility that they've had for ages. We're getting a lot 
of pressure to continue deploying hardware analyzers and/or deploy 
non-OpenStack mechanisms for implementing tap/SPAN/mirror capability when I'd 
much rather integrate the analyzers into OpenStack.


-Original Message-
From: Kyle Mestery (Code Review) [mailto:rev...@openstack.org] 
Sent: Tuesday, February 24, 2015 17:37
To: vinay yadhav
Cc: CARVER, PAUL; Marios Andreou; Sumit Naiksatam; Anil Rao; Carlos Gonçalves; 
YAMAMOTO Takashi; Ryan Moats; Pino de Candia; Isaku Yamahata; Tomoe Sugihara; 
Stephen Wong; Kanzhe Jiang; Bao Wang; Bob Melander; Salvatore Orlando; Armando 
Migliaccio; Mohammad Banikazemi; mark mcclain; Henry Gessau; Adrian Hoban; 
Hareesh Puthalath; Subrahmanyam Ongole; Fawad Khaliq; Baohua Yang; Maruti 
Kamat; Stefano Maffulli 'reed'; Akihiro Motoki; ijw-ubuntu; Stephen Gordon; 
Rudrajit Tapadar; Alan Kavanagh; Zoltán Lajos Kis
Subject: Change in openstack/neutron-specs[master]: Introducing Tap-as-a-Service

Kyle Mestery has abandoned this change.

Change subject: Introducing Tap-as-a-Service
..


Abandoned

This review is > 4 weeks without comment and currently blocked by a core 
reviewer with a -2. We are abandoning this for now. Feel free to reactivate the 
review by pressing the restore button and contacting the reviewer with the -2 
on this review to ensure you address their concerns.

-- 
To view, visit https://review.openstack.org/96149
To unsubscribe, visit https://review.openstack.org/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: I087d9d2a802ea39c02259f17d2b8c4e2f6d8d714
Gerrit-PatchSet: 8
Gerrit-Project: openstack/neutron-specs
Gerrit-Branch: master
Gerrit-Owner: vinay yadhav 
Gerrit-Reviewer: Adrian Hoban 
Gerrit-Reviewer: Akihiro Motoki 
Gerrit-Reviewer: Alan Kavanagh 
Gerrit-Reviewer: Anil Rao 
Gerrit-Reviewer: Armando Migliaccio 
Gerrit-Reviewer: Bao Wang 
Gerrit-Reviewer: Baohua Yang 
Gerrit-Reviewer: Bob Melander 
Gerrit-Reviewer: Carlos Gonçalves 
Gerrit-Reviewer: Fawad Khaliq 
Gerrit-Reviewer: Hareesh Puthalath 
Gerrit-Reviewer: Henry Gessau 
Gerrit-Reviewer: Isaku Yamahata 
Gerrit-Reviewer: Jenkins
Gerrit-Reviewer: Kanzhe Jiang 
Gerrit-Reviewer: Kyle Mestery 
Gerrit-Reviewer: Marios Andreou 
Gerrit-Reviewer: Maruti Kamat 
Gerrit-Reviewer: Mohammad Banikazemi 
Gerrit-Reviewer: Paul Carver 
Gerrit-Reviewer: Pino de Candia 
Gerrit-Reviewer: Rudrajit Tapadar 
Gerrit-Reviewer: Ryan Moats 
Gerrit-Reviewer: Salvatore Orlando 
Gerrit-Reviewer: Stefano Maffulli 'reed' 
Gerrit-Reviewer: Stephen Gordon 
Gerrit-Reviewer: Stephen Wong 
Gerrit-Reviewer: Subrahmanyam Ongole 
Gerrit-Reviewer: Sumit Naiksatam 
Gerrit-Reviewer: Tomoe Sugihara 
Gerrit-Reviewer: Welcome, new contributor!
Gerrit-Reviewer: YAMAMOTO Takashi 
Gerrit-Reviewer: Zoltán Lajos Kis 
Gerrit-Reviewer: ijw-ubuntu 
Gerrit-Reviewer: mark mcclain 
Gerrit-Reviewer: vinay yadhav 
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo] Graduating oslo.reports: Request to review "clean copy"

2015-02-24 Thread Doug Hellmann



On Tue, Feb 24, 2015, at 04:42 PM, Solly Ross wrote:
> Hello All,
> 
> I've finally had some time to finish up the graduation work for
> oslo.reports (previously
> openstack.common.report), and it should be ready for review by the Oslo
> team.  The only thing
> that I was unclear about was the "sync required tools from
> oslo-incubator" part.
> oslo.reports does not use any modules from oslo-incubator, and it is
> unclear what
> constitutes an "appropriate script".

Those scripts have mostly moved into oslotest, so they don't need to be
synced any more to be used. If you have all of your code, and it follows
the cookiecutter template, we can look at it and propose post-import
tweaks. What's the URL for the repository?

Doug


> 
> Best Regards,
> Solly Ross
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [OpenStack-Infra] [infra] Infra cloud: infra running a cloud for nodepool

2015-02-24 Thread Chmouel Boudjnah

cor...@inaugust.com (James E. Blair) writes:

> A group of folks from HP is interested in starting an effort to run a
> cloud as part of the Infrastructure program with the purpose of
> providing resources to nodepool for OpenStack testing.  HP is supplying
> two racks of machines, and we will operate each as an independent cloud.
> I think this is a really good idea, and will do a lot for OpenStack.
>
> Here's what we would get out of it:

Pretty cool! thanks to HP for providing this. If that's possible (with
HP and if the infra wants to allow that) it would be nice to allow a dev
to login into the failing vm for investigation.

Cheers,
Chmouel

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] client library release versions

2015-02-24 Thread Matt Riedemann




On 2/24/2015 3:34 PM, Robert Collins wrote:

Hi, in the cross project meeting a small but important thing came up.

Most (all?) of our client libraries run with semver: x.y.z version
numbers. http://semver.org/ and
http://docs.openstack.org/developer/pbr/semver.html

However we're seeing recent releases that are bumping .z inappropriately.

This makes the job of folk writing version constraints harder :(.

*most* of our releases should be an increment of .y - so 1.2.0, 1.3.0
etc. The only time a .z increase is expected is for
"backwards-compatible bug fixes". [1]

In particular, changing a dependency version is probably never a .z
increase, except - perhaps - when the dependency itself only changed
.z, and so on transitively.

Adding or removing a dependency really can't ever be a .z increase.

We're nearly finished on the pbr support to help automate the decision
making process, but the rule of thumb - expect to do .y increases - is
probably good enough for a while yet.

-Rob

[1]: The special case is for projects that have not yet committed to a
public API - 0.x.y versions. Don't do that. Commit to a public API :)



I was hoping to do a sqlalchemy-migrate release this week so I'm 
interested in not screwing that up. :)


The current release is 0.9.4 and there was one change to 
requirements.txt, cee9136, since then, so if I'm reading this correctly 
the next version for sqlalchemy-migrate should really be 0.10.0.


Regarding public API and 1.x.y, I don't think there is really anything 
holding sqlalchemy-migrate back from that, it's hella old so we should 
probably be 1.0.0 by now.


--

Thanks,

Matt Riedemann


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] novaclient functional test guidelines

On Tue, Feb 24, 2015 at 1:18 PM, melanie witt  wrote:

> On Feb 24, 2015, at 9:47, Sean Dague  wrote:
>
> > I'm happy if there are other theories about how we do these things,
> > being the first functional test in the python-novaclient tree that
> > creates and destroys real resources, there isn't an established pattern
> > yet. But I think doing all CLI calls in CLI tests is actually really
> > cumbersome, especially in the amount of output parsing code needed if
> > you are going to setup any complicated resource structure.
>
> I think I'm in agreement with the pattern you describe.
>
> I imagine having a set of functional tests for the API, that don't do any
> CLI calls at all. With that we test that the API works properly. Then have
> a separate set for the CLI, which only calls CLI for the command being
> tested, everything else to set up and tear down the test done by API calls.
> This would be done with the rationale that because the entire API
> functionality is tested separately, we can safely use it for setup/teardown
> with the intent to isolate the CLI test to the command being tested and
> avoid introducing side effects from the CLI commands.
>
> But I suppose one could make the same argument for using CLI everywhere
> (if they are all tested, they can also be trusted not to introduce side
> effects). I tend to favor using the API because it's the most bare bones
> setup/teardown we could use. At the same time I understand the idea of
> performing an entire test using the CLI, as a way of replicating the
> experience a real user might have using the CLI, from start to end. I don't
> think I feel strongly either way.


 I guess its time to revisit the actual status of novaclient and if we want
to actively move away from it to openstacksdk/OSC as well. If we are
actively trying to move away from novaclient, using the python API as much
as possible makes a lot of sense.



>
> For the --poll stuff, I agree the API should have it and the CLI uses it.
> And with and without "poll" functionality should be tested separately, API
> and CLI.
>
> melanie (melwitt)
>
>
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] novaclient functional test guidelines

On Tue, Feb 24, 2015 at 12:30 PM, Sean Dague  wrote:

> On 02/24/2015 03:28 PM, Ed Leafe wrote:
> > On Feb 24, 2015, at 1:49 PM, Sean Dague  wrote:
> >
> >>> IMHO the CLI should have an option to returned raw JSON back instead of
> >>> pretty tabled results as well.
> >>
> >> Um... isn't that just the API calls?
> >>
> >> I'm not sure creating a 3rd functional surface is really the answer
> >> here, because we still need to actually test the CLI / pretty table
> output.
> >
> > The python-openstacksdk project was originally envisioned to wrap the
> API calls and return usable Python objects. The nova client CLI (or any
> other CLI, for that matter) would then just provide the command line input
> parsing and output presentation. It's been a while since I was involved
> with that project, but it seems that decoupling the command line interface
> from the Python API wrapper would make testing much, much easier.
>
> Right, I think to some degree novaclient is legacy code, and we should
> focus on specific regressions and bugs without doing to much code change.
>
> The future should be more focussed on openstacksdk and openstackclient.
>

While I don't disagree with this, it seems like we have been waiting for
openstackclient and openstacksdk for a while now, do we even have a
timeline for moving off of novaclient?


>
> -Sean
>
> --
> Sean Dague
> http://dague.net
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [oslo] Graduating oslo.reports: Request to review "clean copy"

2015-02-24 Thread Solly Ross

Hello All,

I've finally had some time to finish up the graduation work for oslo.reports 
(previously
openstack.common.report), and it should be ready for review by the Oslo team.  
The only thing
that I was unclear about was the "sync required tools from oslo-incubator" part.
oslo.reports does not use any modules from oslo-incubator, and it is unclear 
what
constitutes an "appropriate script".

Best Regards,
Solly Ross

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] client library release versions

Hi, in the cross project meeting a small but important thing came up.

Most (all?) of our client libraries run with semver: x.y.z version
numbers. http://semver.org/ and
http://docs.openstack.org/developer/pbr/semver.html

However we're seeing recent releases that are bumping .z inappropriately.

This makes the job of folk writing version constraints harder :(.

*most* of our releases should be an increment of .y - so 1.2.0, 1.3.0
etc. The only time a .z increase is expected is for
"backwards-compatible bug fixes". [1]

In particular, changing a dependency version is probably never a .z
increase, except - perhaps - when the dependency itself only changed
.z, and so on transitively.

Adding or removing a dependency really can't ever be a .z increase.

We're nearly finished on the pbr support to help automate the decision
making process, but the rule of thumb - expect to do .y increases - is
probably good enough for a while yet.

-Rob

[1]: The special case is for projects that have not yet committed to a
public API - 0.x.y versions. Don't do that. Commit to a public API :)

-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] novaclient functional test guidelines

2015-02-24 Thread melanie witt

On Feb 24, 2015, at 9:47, Sean Dague  wrote:

> I'm happy if there are other theories about how we do these things,
> being the first functional test in the python-novaclient tree that
> creates and destroys real resources, there isn't an established pattern
> yet. But I think doing all CLI calls in CLI tests is actually really
> cumbersome, especially in the amount of output parsing code needed if
> you are going to setup any complicated resource structure.

I think I'm in agreement with the pattern you describe.

I imagine having a set of functional tests for the API, that don't do any CLI 
calls at all. With that we test that the API works properly. Then have a 
separate set for the CLI, which only calls CLI for the command being tested, 
everything else to set up and tear down the test done by API calls. This would 
be done with the rationale that because the entire API functionality is tested 
separately, we can safely use it for setup/teardown with the intent to isolate 
the CLI test to the command being tested and avoid introducing side effects 
from the CLI commands.

But I suppose one could make the same argument for using CLI everywhere (if 
they are all tested, they can also be trusted not to introduce side effects). I 
tend to favor using the API because it's the most bare bones setup/teardown we 
could use. At the same time I understand the idea of performing an entire test 
using the CLI, as a way of replicating the experience a real user might have 
using the CLI, from start to end. I don't think I feel strongly either way.

For the --poll stuff, I agree the API should have it and the CLI uses it. And 
with and without "poll" functionality should be tested separately, API and CLI.

melanie (melwitt)

signature.asc
Description: Message signed with OpenPGP using GPGMail
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [infra] Infra cloud: infra running a cloud for nodepool

2015-02-24 Thread James E. Blair

A group of folks from HP is interested in starting an effort to run a
cloud as part of the Infrastructure program with the purpose of
providing resources to nodepool for OpenStack testing.  HP is supplying
two racks of machines, and we will operate each as an independent cloud.
I think this is a really good idea, and will do a lot for OpenStack.

Here's what we would get out of it:

1) More test resources.  The primary goal of this cloud will be to
provide more instances to nodepool.  This would extend our pool to
include a third provider meaning that we are more resilient to service
disruptions, and increase our aggregate capacity meaning we can perform
more testing more quickly.  It's hard to say for certain until we have
something spun up that we can benchmark, but we are hoping for somewhere
between an additional 50% to 100% of our current capacity.

2) Closing the loop between OpenStack developers and ops.  This cloud
will be deployed as often as we are able (perhaps daily, perhaps less
often, depending on technology) meaning that if it is not behaving in a
way developers like, they can fix it fairly quickly.

3) A fully open deployment.  The infra team already runs a large
logstash and elasticsearch system for finding issues in devstack runs.
We will deploy the same technology (and perhaps more) to make sure that
anyone who wants to inspect the operational logs from the running
production cloud is able to do so.  We can even run the same
elastic-recheck queries to see if known bugs are visible in production.
The cloud will be deployed using the same tools and processes as the
rest of the project infrastructure, meaning anyone can edit the modules
that deploy the cloud to make changes.

How is this different from the TripleO cloud?

The primary goal of the TripleO cloud is to provide test infrastructure
so that the TripleO project can run tests that require real hardware and
complex environments.  The primary purpose of the infra cloud will be to
run a production service that will stand alongside other cloud providers
to supply virtual machines to nodepool.

What about the infra team's aversion to real hardware?

It's true that all of our current resources are virtual, and this would
be adding the first real, bare-metal, machines to the infra project.
However, there are a number of reasons I feel we're ready to take that
step now:

* This cloud will stand alongside two others to provide resources to
  nodepool.  If it completely fails, infra will continue to operate; so
  we don't need to be overly concerned with uptime and being on-call,
  etc.

* The deployment and operation of the cloud will use the same technology
  and processes as the infra project currently uses, so there should be
  minimal barriers for existing team members.

* A bunch of new people will be joining the team to help with this.  We
  expect them to become fully integrated with the rest of infra, so that
  they are able to help out in other areas and the whole team expands
  its collective capacity and expertise.

If this works well, it may become a template for incorporating other
hardware contributions into the system.

Next steps:

We've started the process of identifying the steps to make this happen,
as well as answering some deployment questions (specifics about
technology, topology, etc).  There is a StoryBoard story for the effort:

  https://storyboard.openstack.org/#!/story/2000175

And some notes that we took at a recent meeting to bootstrap the effort:

  https://etherpad.openstack.org/p/InfraCloudBootcamp

I think one of the next steps is to actually write all that up and push
it up as a change to the system-config documentation.  Once we're
certain we agree on all of that, it should be safe to divide up many of
the remaining tasks.

-Jim

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Fuel] Tag bugs with features from current release

2015-02-24 Thread Dmitry Borodaenko

During bugs triage and release notes preparation we often need to find
out whether a bug is a regression introduced by new code in the
current release, or may have been present in previous releases. In the
latter case, depending on its severity, it may need to be reflected in
the release notes, and its fix backported to previous release series.

I propose to create official tags out of names of all blueprints
targeted to current release, use these tags to label all related
regression bugs.

Thoughts, objections?

-- 
Dmitry Borodaenko

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] novaclient functional test guidelines

2015-02-24 Thread Ed Leafe

On Feb 24, 2015, at 2:30 PM, Sean Dague  wrote:

> Right, I think to some degree novaclient is legacy code, and we should
> focus on specific regressions and bugs without doing to much code change.
> 
> The future should be more focussed on openstacksdk and openstackclient.

IMO, openstackclient has an impossible task: taking the varied (and flawed) CLI 
clients and unite them under a single CLI interface. It is better to create the 
clean separation of the API wrapping into a Python library, and keep the CLI 
completely separate.

-- Ed Leafe

signature.asc
Description: Message signed with OpenPGP using GPGMail
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] novaclient functional test guidelines

On 02/24/2015 03:28 PM, Ed Leafe wrote:
> On Feb 24, 2015, at 1:49 PM, Sean Dague  wrote:
> 
>>> IMHO the CLI should have an option to returned raw JSON back instead of
>>> pretty tabled results as well.
>>
>> Um... isn't that just the API calls?
>>
>> I'm not sure creating a 3rd functional surface is really the answer
>> here, because we still need to actually test the CLI / pretty table output.
> 
> The python-openstacksdk project was originally envisioned to wrap the API 
> calls and return usable Python objects. The nova client CLI (or any other 
> CLI, for that matter) would then just provide the command line input parsing 
> and output presentation. It's been a while since I was involved with that 
> project, but it seems that decoupling the command line interface from the 
> Python API wrapper would make testing much, much easier.

Right, I think to some degree novaclient is legacy code, and we should
focus on specific regressions and bugs without doing to much code change.

The future should be more focussed on openstacksdk and openstackclient.

-Sean

-- 
Sean Dague
http://dague.net



signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [stable][all] Revisiting the 6 month release cycle

2015-02-24 Thread Ed Leafe

On Feb 24, 2015, at 1:50 PM, Joe Gordon  wrote:

>> I think the release candidate
>> period is the only thing that makes your code drops actually usable.
>> It's the only moment in the cycle where integrators test. It's the only
>> moment in the cycle where developers work on bugs they did not file
>> themselves, but focus on a project-wide priority list of release
>> blockers. If you remove that period, then nobody will ever work on
>> release blockers that do not directly affect them. Shorten that period
>> to one week, and no integrator will have the time to run a proper QA
>> program to detect those release blockers.
> 
> I still think the 3 week RC candidate cycle needs to happen, the difference 
> is it would be done by stable maintenance. And I agree, the RC candidate 
> period is one of the few moments where developers work on bugs they did not 
> file themselves. So I am not sure how this would actually work.  Perhaps the 
> answer is we have deeper issues if we don't want to fix bugs until the last 
> minute.

I like the notion that there isn't an overall release that all development is 
tied to, but that at some more or less regular interval, a new stable "release" 
is cut, and intense integration testing is done on that, with bug fixes as 
needed. But how to get developers who are intent on coding their new features 
to hold off and work on fixing bugs identified by the stable testing is a big 
question mark to me.


-- Ed Leafe







signature.asc
Description: Message signed with OpenPGP using GPGMail
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] novaclient functional test guidelines

2015-02-24 Thread Ed Leafe

On Feb 24, 2015, at 1:49 PM, Sean Dague  wrote:

>> IMHO the CLI should have an option to returned raw JSON back instead of
>> pretty tabled results as well.
> 
> Um... isn't that just the API calls?
> 
> I'm not sure creating a 3rd functional surface is really the answer
> here, because we still need to actually test the CLI / pretty table output.

The python-openstacksdk project was originally envisioned to wrap the API calls 
and return usable Python objects. The nova client CLI (or any other CLI, for 
that matter) would then just provide the command line input parsing and output 
presentation. It's been a while since I was involved with that project, but it 
seems that decoupling the command line interface from the Python API wrapper 
would make testing much, much easier.

-- Ed Leafe

signature.asc
Description: Message signed with OpenPGP using GPGMail
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Re-evaluating the suitability of the 6 month release cycle

On 02/24/2015 03:21 PM, Joe Gordon wrote:
> 
> 
> On Tue, Feb 24, 2015 at 6:57 AM, Daniel P. Berrange  > wrote:
> 
> On Tue, Feb 24, 2015 at 08:50:45AM -0500, Sean Dague wrote:
> > On 02/24/2015 07:48 AM, Russell Bryant wrote:
> > > On 02/24/2015 12:54 PM, Daniel P. Berrange wrote:
> > >> On Tue, Feb 24, 2015 at 11:48:29AM +, Chris Dent wrote:
> > >>> On Tue, 24 Feb 2015, Daniel P. Berrange wrote:
> > >>>
> >  need to do more work. If this is so, then I don't think this
> is a blocker,
> >  it is just a sign that the project needs to focus on
> providing more resources
> >  to the teams impacted in that way.
> > >>>
> > >>> What are the mechanisms whereby the project provides more
> resources
> > >>> to teams?
> > >>
> > >> The technical committee and / or foundation board can highlight
> the need
> > >> for investment of resources in critical areas of the project,
> to either
> > >> the community members or vendors involved. As an example, this
> was done
> > >> successfully recently to increase involvement in maintaining
> the EC2
> > >> API support.  There are plenty of vendors involved in OpenStack
> which
> > >> have the ability to target resources, if they can learn where those
> > >> resources are best spent.
> > >
> > > Indeed ... and if horizontal teams are the ones hit the most by the
> > > extra work, each project should help with that burden.  For example,
> > > projects may need to take their responsibility for documentation
> more
> > > seriously and require documentation with features (content at
> least, not
> > > necessarily integration into the proper documentation deliverables)
> > > instead of assuming it magically gets written later.
> >
> > Right, and I think this actually hits at the most important part
> of the
> > discussion. The question of:
> >
> > 1) what would we need to do to make different release cadences viable?
> > 2) are those good things to do regardless of release cadence?
> >
> > The horizontal teams really can't function at different cadences. It
> > completely breaks any flow and planning at turns them even further
> into
> > firefighting because now everyone has crunch time at different times,
> > and the horizontal efforts are required to just play catch up. I know
> > what that future looks like, the horizontal teams dry up because
> no one
> > wants that job.
> >
> > Ok, so that being said, what we'd need to do is have horizontal teams
> > move to more of a self supporting model. So that the relevant content
> > for a project (docs, full stack tests, requirements, etc) all live
> > within that project itself, and aren't centrally synchronized.
> > Installation of projects needs to be fully isolated from each other so
> > that upgrading project A can be done independent of project B, as
> their
> > release cadences might all bit disparate. Basically, ever OpenStack
> > project needs to reabsorb the cross project efforts they've
> externalized.
> >
> > Then if project A decided to move off the coupled release, it's impact
> > to the rest would be minimal. These are robust components that
> stand on
> > their own, and work well with robust other components.
> >
> > Which... is basically the point of the big tent / new project
> governance
> > model. Decompose OpenStack from a giant blob of goo into Robust
> elements
> > that are more loosely coupled (so independently robust, and robust in
> > their interaction with others). Move the horizontal teams into
> > infrastructure vs. content roles, have projects own more of this
> content
> > themselves.
> >
> > But it is a long hard process. Devstack external plugins was implement
> > to support this kind of model, but having walked a bunch of different
> > teams through this (at all skill levels) there ends up being a lot of
> > work to get this right, and a lot of rethinking by teams that assumed
> > their interaction with full stack testing is something they'd get to
> > contribute once and have someone else maintain (instead of something
> > they now need dedicated watchful eye on).
> >
> > The amount of full stack configurations immediately goes beyond
> anywhere
> > near testable, so it requires more robust project testing to ensure
> > every exposed interface is more robust (i.e. the testing in pyramids
> > from https://review.openstack.org/#/c/150653/).
> >
> > And, I think the answer to #2 is: yes, this just makes it all better.
> >
> > So, honestly, I'm massively supportive of the end game. I've been
> > carving out the bits of this I can for the last six months. But I
> think
>

Re: [openstack-dev] python-ceilometerclient 1.0.13 broke the gate

We saw the same issue elsewhere, and Doug had great explanation on how it
broke semver

https://www.mail-archive.com/openstack-dev@lists.openstack.org/msg46533.html

On Tue, Feb 24, 2015 at 12:11 PM, Matt Riedemann  wrote:

> https://bugs.launchpad.net/python-ceilometerclient/+bug/1425262
>
> mtreinish adjusted the cap on stable/icehouse here:
>
> https://review.openstack.org/#/c/158842/
>
> jogo now has a change to explicitly pin all clients in stable/icehouse to
> the version currently gated on:
>
> https://review.openstack.org/#/c/158846/
>
> --
>
> Thanks,
>
> Matt Riedemann
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Re-evaluating the suitability of the 6 month release cycle

On Tue, Feb 24, 2015 at 6:57 AM, Daniel P. Berrange 
wrote:

> On Tue, Feb 24, 2015 at 08:50:45AM -0500, Sean Dague wrote:
> > On 02/24/2015 07:48 AM, Russell Bryant wrote:
> > > On 02/24/2015 12:54 PM, Daniel P. Berrange wrote:
> > >> On Tue, Feb 24, 2015 at 11:48:29AM +, Chris Dent wrote:
> > >>> On Tue, 24 Feb 2015, Daniel P. Berrange wrote:
> > >>>
> >  need to do more work. If this is so, then I don't think this is a
> blocker,
> >  it is just a sign that the project needs to focus on providing more
> resources
> >  to the teams impacted in that way.
> > >>>
> > >>> What are the mechanisms whereby the project provides more resources
> > >>> to teams?
> > >>
> > >> The technical committee and / or foundation board can highlight the
> need
> > >> for investment of resources in critical areas of the project, to
> either
> > >> the community members or vendors involved. As an example, this was
> done
> > >> successfully recently to increase involvement in maintaining the EC2
> > >> API support.  There are plenty of vendors involved in OpenStack which
> > >> have the ability to target resources, if they can learn where those
> > >> resources are best spent.
> > >
> > > Indeed ... and if horizontal teams are the ones hit the most by the
> > > extra work, each project should help with that burden.  For example,
> > > projects may need to take their responsibility for documentation more
> > > seriously and require documentation with features (content at least,
> not
> > > necessarily integration into the proper documentation deliverables)
> > > instead of assuming it magically gets written later.
> >
> > Right, and I think this actually hits at the most important part of the
> > discussion. The question of:
> >
> > 1) what would we need to do to make different release cadences viable?
> > 2) are those good things to do regardless of release cadence?
> >
> > The horizontal teams really can't function at different cadences. It
> > completely breaks any flow and planning at turns them even further into
> > firefighting because now everyone has crunch time at different times,
> > and the horizontal efforts are required to just play catch up. I know
> > what that future looks like, the horizontal teams dry up because no one
> > wants that job.
> >
> > Ok, so that being said, what we'd need to do is have horizontal teams
> > move to more of a self supporting model. So that the relevant content
> > for a project (docs, full stack tests, requirements, etc) all live
> > within that project itself, and aren't centrally synchronized.
> > Installation of projects needs to be fully isolated from each other so
> > that upgrading project A can be done independent of project B, as their
> > release cadences might all bit disparate. Basically, ever OpenStack
> > project needs to reabsorb the cross project efforts they've externalized.
> >
> > Then if project A decided to move off the coupled release, it's impact
> > to the rest would be minimal. These are robust components that stand on
> > their own, and work well with robust other components.
> >
> > Which... is basically the point of the big tent / new project governance
> > model. Decompose OpenStack from a giant blob of goo into Robust elements
> > that are more loosely coupled (so independently robust, and robust in
> > their interaction with others). Move the horizontal teams into
> > infrastructure vs. content roles, have projects own more of this content
> > themselves.
> >
> > But it is a long hard process. Devstack external plugins was implement
> > to support this kind of model, but having walked a bunch of different
> > teams through this (at all skill levels) there ends up being a lot of
> > work to get this right, and a lot of rethinking by teams that assumed
> > their interaction with full stack testing is something they'd get to
> > contribute once and have someone else maintain (instead of something
> > they now need dedicated watchful eye on).
> >
> > The amount of full stack configurations immediately goes beyond anywhere
> > near testable, so it requires more robust project testing to ensure
> > every exposed interface is more robust (i.e. the testing in pyramids
> > from https://review.openstack.org/#/c/150653/).
> >
> > And, I think the answer to #2 is: yes, this just makes it all better.
> >
> > So, honestly, I'm massively supportive of the end game. I've been
> > carving out the bits of this I can for the last six months. But I think
> > the way we get there is to actually get the refactoring of the
> > horizontal efforts first.
>
> I pretty much fully agree that refactoring the horizontal efforts to
> distribute responsbility across the individual projects is the way
> forward. I don't think it needs to be a pre-requisite step for changing
> the release cycle. We can do both in parallel if we put our minds to
> it.
>
> My biggest fear is that we just keeping debating problems and alternatives,
> but continue to be too afraid of theore

[openstack-dev] python-ceilometerclient 1.0.13 broke the gate

2015-02-24 Thread Matt Riedemann


https://bugs.launchpad.net/python-ceilometerclient/+bug/1425262

mtreinish adjusted the cap on stable/icehouse here:

https://review.openstack.org/#/c/158842/

jogo now has a change to explicitly pin all clients in stable/icehouse 
to the version currently gated on:


https://review.openstack.org/#/c/158846/

--

Thanks,

Matt Riedemann


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [stable][all] Revisiting the 6 month release cycle

On Tue, Feb 24, 2015 at 2:59 AM, Daniel P. Berrange 
wrote:

> On Mon, Feb 23, 2015 at 04:14:36PM -0800, Joe Gordon wrote:
> > Was:
> >
> http://lists.openstack.org/pipermail/openstack-dev/2015-February/057578.html
> >
> > There has been frustration with our current 6 month development cadence.
> > This is an attempt to explain those frustrations and propose a very rough
> > outline of a possible alternative.
> >
> >
> > Currently we follow a 6 month release cadence, with 3 intermediate
> > milestones (6 weeks apart), as explained here:
> > https://wiki.openstack.org/wiki/Kilo_Release_Schedule
> >
> >
> > Current issues
> >
> >- 3 weeks of feature freeze for all projects at the end of each cycle
> (3
> >out of the 26 feature blocked)
> >- 3 weeks of release candidates. Once a candidate is cut development
> is
> >open for next release. While this is good in theory, not much work
> actually
> >starts on next release.
> >- some projects have non priority feature freezes and at Milestone 2
> (so
> >9 out of 26 weeks restricted in those projects)
> >- vicious development circle
> >   - vicious circle:
> >  - big push right to land lots of features right before the
> release
> >  - a lot of effort is spent getting the release ready
> >  - after the release people are a little burnt out and take it
> easy
> >  until the next summit
> >  - Blueprints have to be re-discussed and re-approved for the
> next
> >  cycle
> >  - makes it hard to land blueprints early in the cycle casing the
> >  bug rush at the end of the cycle, see step 1
> >   - Makes it hard to plan things across a release
> >   - This actually destabilizes things right as we go into the
> >   stabilization period (We actually have great data on this too)
> >   - Means postponing blueprints that miss a deadline several months
> >
> >
> > Requirements for a new model
> >
> >- Keep 6 month release cadence. Not everyone is willing to deploy from
> >trunk
> >- Keep stable branches for at least 6 months. In order to test
> upgrades
> >from the last stable branch, we need a stable branch to test against
> >- Keep supporting continuous deployment. Some people really want to
> >deploy from trunk
> >
> >
> > What We can drop
> >
> >- While we need to keep releasing a stable branch every 6 months, we
> >don't have to do all of our development planning around it. We can
> treat it
> >as just another milestone
> >
> >
> > I think a lot of the frustration with our current cadence comes out of
> the
> > big stop everything (development, planning etc.), and stabilize the
> release
> > process. Which in turn isn't really making things more stable. So I
> propose
> > we keep the 6 month release cycle, but change the development cycle from
> a
> > 6 month one with 3 intermediate milestones to a 6 week one with a
> milestone
> > at the end of it.
>
> You're solving some issues around developer experiance by letting
> developers
> iterate on a faster cycle which is something I agree with, but by keeping
> the 6 month release cycle I think you're missing the bigger opportunity
> here.
> Namely, the chance to get the features to the users faster, which is
> ultimtely
> the reason why contributors currently push us so hard towards the end of
> the
> release. I think we have to be more ambitious here and actually make the
> release
> cycle itself faster, putting it on a 2 month cycle. More detail about why
> I think
> this is needed is here:
>
>
> http://lists.openstack.org/pipermail/openstack-dev/2015-February/057614.html
>
>
Nothing like having to concurrent threads on the same thing with very
similar subjects.

[openstack-dev] [all] Re-evaluating the suitability of the 6 month release
cycle

vs

[openstack-dev][stable][all] Revisiting the 6 month release cycle

I'll respond to your proposal, on the other thread.



> > What this actually means:
> >
> >- Stop approving blueprints for specific stable releases, instead just
> >approve them and target them to milestones.
> >   - Milestones stop becoming Kilo-1, Kilo-2, Kilo-3 etc. and just
> >   become 1,2,3,4,5,6,7,8,9 etc.
> >   - If something misses what was previously known as Kilo-3 it has to
> >   wait a week for what for milestone 4.
> >- Development focuses on milestones only. So 6 week cycle with say 1
> >week of stabilization, finish things up before each milestone
> >- Process of cutting a stable branch (planning, making the branch,
> doing
> >release candidates, testing etc.) is done by a dedicated stable branch
> >team. And it should be done based on a specific milestone
> >- Goal: Change the default development planning mode to ignore stable
> >branches, and allow for us to think of things in terms of the number
> of
> >milestones needed, not will it make the stable branch or not
> >
> >
> > Gotchas, questio

Re: [openstack-dev] [stable][all] Revisiting the 6 month release cycle

On Tue, Feb 24, 2015 at 10:00 AM, Johannes Erdfelt 
wrote:

> On Tue, Feb 24, 2015, Thierry Carrez  wrote:
> > Agree on the pain of maintaining milestone plans though, which is why I
> > propose we get rid of most of it in Liberty. That will actually be
> > discussed at the cross-project meeting today:
> >
> >
> https://wiki.openstack.org/wiki/Release_Cycle_Management/Liberty_Tracking
>
> I'm happy to see this.
>

++


>
> "Assignees may target their blueprint to a future milestone, as an
> indication of when they intend to land it (not mandatory)"
>
> That seems useless to me. I have no control over when things land. I can
> only control when my code is put up for review.
>
> Recently, I have spent a lot more time waiting on reviews than I have
> spent writing the actual code.
>

I think this is a symptom of a much deeper problem. And adjusting the
release cadence won't make a big impact on this.


>
> JE
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [stable][all] Revisiting the 6 month release cycle

On Tue, Feb 24, 2015 at 2:38 AM, Thierry Carrez 
wrote:

> Joe Gordon wrote:
> > [...]
> > I think a lot of the frustration with our current cadence comes out of
> > the big stop everything (development, planning etc.), and stabilize the
> > release process. Which in turn isn't really making things more stable.
>
> I guess I have a different position. I think the release candidate
> period is the only thing that makes your code drops actually usable.
> It's the only moment in the cycle where integrators test. It's the only
> moment in the cycle where developers work on bugs they did not file
> themselves, but focus on a project-wide priority list of release
> blockers. If you remove that period, then nobody will ever work on
> release blockers that do not directly affect them. Shorten that period
> to one week, and no integrator will have the time to run a proper QA
> program to detect those release blockers.
>

I still think the 3 week RC candidate cycle needs to happen, the difference
is it would be done by stable maintenance. And I agree, the RC candidate
period is one of the few moments where developers work on bugs they did not
file themselves. So I am not sure how this would actually work.  Perhaps
the answer is we have deeper issues if we don't want to fix bugs until the
last minute.



>
> I understand that from your developer perspective it's annoying to have
> to work on project-wide priorities rather than your own, and therefore
> you'd like to get rid of those -- but the resulting drop in quality is
> just something we can't afford.
>
> > So I propose we keep the 6 month release cycle, but change the
> > development cycle from a 6 month one with 3 intermediate milestones to a
> > 6 week one with a milestone at the end of it.
> >
> > What this actually means:
> >
> >   * Stop approving blueprints for specific stable releases, instead just
> > approve them and target them to milestones.
> >   o Milestones stop becoming Kilo-1, Kilo-2, Kilo-3 etc. and just
> > become 1,2,3,4,5,6,7,8,9 etc.
> >   o If something misses what was previously known as Kilo-3 it has
> > to wait a week for what for milestone 4.
> >   * Development focuses on milestones only. So 6 week cycle with say 1
> > week of stabilization, finish things up before each milestone
> >   * Process of cutting a stable branch (planning, making the branch,
> > doing release candidates, testing etc.) is done by a dedicated
> > stable branch team. And it should be done based on a specific
> milestone
> >   * Goal: Change the default development planning mode to ignore stable
> > branches, and allow for us to think of things in terms of the number
> > of milestones needed, not will it make the stable branch or not
>
> I don't think that would solve any of the issues you mentioned:
> > Current issues
> >   * 3 weeks of feature freeze for all projects at the end of each cycle
> > (3 out of the 26 feature blocked)
>
> So you'll have 3 x 1 week of feature freeze for all projects, instead of
> 1 x 3 weeks. That will be less efficient (integrators need a >1week
> feature freeze period to actually start testing a non-moving target),
> more painful (have to organize it 3 times instead of 1), and likely
> inefficient (takes generally more than one week to find critical bugs,
> develop the fix, and get it reviewed). And in the end, it's still 3 out
> of the 26 feature blocked.
>

As said before, I don't envision integrator consuming every milestone. Just
the standard 3 week RC cycle for stable branch candidates.


>
> >   * 3 weeks of release candidates. Once a candidate is cut development
> > is open for next release. While this is good in theory, not much
> > work actually starts on next release.
>
> That is not really what I observe. People start landing their feature in
> the master branch starting the day after RC1. I actually observe the
> opposite: too many people switching to master development, and not
> enough people working on RC2+ bugs.
>

Unfortunately I think we are both right. To many people move on and don't
work on RC2 bugs, but development still slows down.


>
> >   * some projects have non priority feature freezes and at Milestone 2
> > (so 9 out of 26 weeks restricted in those projects)
>
> That was their own choice. I for one was really surprised that they
> would freeze earlier -- I think 3 weeks is the right balance.
>
> >   * vicious development circle
> >   o vicious circle:
> >   + big push right to land lots of features right before the
> release
>
> I think you'll have exactly the same push before the "stable" milestone
> (or the one that will be adopted by $DISTRO).
>

I am hoping the push would be smaller, but I don't think we can remove it
completely.


>
> >>   + a lot of effort is spent getting the release ready
> >>   + after the release people are a little burnt out and take it
> >> easy until the next summit
>
> No

Re: [openstack-dev] [stable] New to project stable maintenance, question on requirements changes

2015-02-24 Thread Trevor McKay

Sean, 

 thanks!  I feel better already. I'll check out the review.

Trevor

On Tue, 2015-02-24 at 14:39 -0500, Sean Dague wrote:
> On 02/24/2015 02:34 PM, Trevor McKay wrote:
> > Hi folks,
> > 
> > I've just joined the stable maintenance team for Sahara.
> > 
> > We have this review here, from OpenStack proposal bot:
> > 
> > https://review.openstack.org/158775/
> > 
> > Since it came from the proposal bot, there's no justification in the
> > commit message and no cherry pick.
> > 
> > I didn't see this case covered as one of the strict set in
> > 
> > https://wiki.openstack.org/wiki/StableBranch
> > 
> > Do we trust the proposal bot? How do I know I should trust it? On
> > master, I assume if there
> > is a mistake it will soon be rectified, but stable ...  Do we have a doc
> > that talks about stable maintenance
> > and requirements changes?  Should we?
> > 
> > Am I being paranoid? :)
> 
> Slightly, but that's probably good.
> 
> Requirements proposal bot changes had to first be Approved on the
> corresponding requirements stable branch, so that should be both safe,
> and mandatory to go in.
> 
> I agree that it would be nicer to have more justification in there.
> There is the beginning of the patch up to do something a bit better here
> - https://review.openstack.org/#/c/145932/ - though it could use to be
> improved.
> 
>   -Sean
> 



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] novaclient functional test guidelines

On 02/24/2015 01:47 PM, Joe Gordon wrote:
> 
> 
> On Tue, Feb 24, 2015 at 9:47 AM, Sean Dague  > wrote:
> 
> Towards the end of merging the regression test for the nova
> volume-attach bug - https://review.openstack.org/#/c/157959/ there was a
> discussion around what style the functional tests should take.
> Especially as that had a mix of CLI and API calls in it.
> 
> 
> 
> Thanks for starting this thread.  Once we reach general agreement lets
> put this in a in tree README for record keeping.

Absolutely.

> Here are my thoughts for why that test ended up that way:
> 
> 1) All resource setup that is table stakes for the test should be done
> via the API, regardless if it's a CLI or API test.
> 
> The reason for this is that structured data is returned, which removes
> one possible error in the tests by parsing incorrectly. The API objects
> returned also include things like .delete methods in most cases, which
> means that addCleanup is a little more clean.
> 
> 
> IMHO the CLI should have an option to returned raw JSON back instead of
> pretty tabled results as well.

Um... isn't that just the API calls?

I'm not sure creating a 3rd functional surface is really the answer
here, because we still need to actually test the CLI / pretty table output.

> 2) Main logic should touch which ever interface you are trying to test.
> This was demonstrating a CLI regression, so the volume-attach call was
> important to be done over the CLI.
> 
> 
> Now... here's where theory runs into issues.
> 
> #1 - nova boot is table stakes. Under the above guidelines it should be
> called via API. However --poll is a CLI construct and so saved a custom
> wait loop here. We should implement that custom wait loop down the road
> and make that an API call
> 
> 
> https://github.com/openstack/python-novaclient/blob/master/novaclient/tests/functional/test_instances.py#L116
> 
> 
> This issue is from a real short coming in the python client. So this
> isn't really an issue with your theory, just an issue with novaclient.

Sure, though, if we support the async form in the API we need to test
it. So while adding poll support is a good UX add, it doesn't actually
mean we get to not test the async versions. It just adds another feature
set we need to test.

> #2 - the volume create command is table stakes. It should be an API
> call. However, it can't be because the service catalog redirection only
> works at the CLI layer. This is actually also the crux of bug #1423695.
> The same reason the completion cache code failed is the reason we can't
> use the API for that.
> 
> 
> https://github.com/openstack/python-novaclient/blob/master/novaclient/tests/functional/test_instances.py#L129
> 
> 
> Issues like this are why I wrote the read only nova CLI tests in the
> first place. Unit testing the python API is doable, but unit testing the
> CLI is a little bit more tricky. So I expect issues like this to come up
> over and over again.

So this is actually an issue with the API code. We have a chunk of code
that's directly callable by consumers, but if they do, it will error.

> #3 - the cleanup of the volume should have been API call. See reason
> for #2.
> 
> 
> https://github.com/openstack/python-novaclient/blob/master/novaclient/tests/functional/test_instances.py#L131
> 
> #4 - the cleanup of the attachment should be an addCleanup via the API.
> See reason for #2 why it's not.
> 
> 
> https://github.com/openstack/python-novaclient/blob/master/novaclient/tests/functional/test_instances.py#L155
> 
> 
> I'm happy if there are other theories about how we do these things,
> being the first functional test in the python-novaclient tree that
> creates and destroys real resources, there isn't an established pattern
> yet. But I think doing all CLI calls in CLI tests is actually really
> cumbersome, especially in the amount of output parsing code needed if
> you are going to setup any complicated resource structure.
> 
> 
> Here is an alternate theory:
> 
> We should have both python API and CLI functional tests. But they should
> be kept separate.
> 
> This is to help us make sure both the CLI and python API are actually
> usable interfaces. As the exercise above has shown, they both have
> really major short comings.  I think having in tree functional testing
> covering both the CLI and python API will make it easier for us to
> review new client features in terms of usability.
> 
> Here is a very rough proof of concept patch showing the same
> tests: 
> https://review.openstack.org/#/c/157974/2/novaclient/tests/functional/test_volumes.py,cm
> 
> No matter how we define this functional testing model, I think its clear
> novaclient needs a decent amount of work before it can really be usable.

Agreed, but I think we should focus on testing the code we actually have
to prevent

Re: [openstack-dev] [stable] New to project stable maintenance, question on requirements changes

On 02/24/2015 02:34 PM, Trevor McKay wrote:
> Hi folks,
> 
> I've just joined the stable maintenance team for Sahara.
> 
> We have this review here, from OpenStack proposal bot:
> 
> https://review.openstack.org/158775/
> 
> Since it came from the proposal bot, there's no justification in the
> commit message and no cherry pick.
> 
> I didn't see this case covered as one of the strict set in
> 
> https://wiki.openstack.org/wiki/StableBranch
> 
> Do we trust the proposal bot? How do I know I should trust it? On
> master, I assume if there
> is a mistake it will soon be rectified, but stable ...  Do we have a doc
> that talks about stable maintenance
> and requirements changes?  Should we?
> 
> Am I being paranoid? :)

Slightly, but that's probably good.

Requirements proposal bot changes had to first be Approved on the
corresponding requirements stable branch, so that should be both safe,
and mandatory to go in.

I agree that it would be nicer to have more justification in there.
There is the beginning of the patch up to do something a bit better here
- https://review.openstack.org/#/c/145932/ - though it could use to be
improved.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [stable] New to project stable maintenance, question on requirements changes

2015-02-24 Thread Trevor McKay

Hi folks,

I've just joined the stable maintenance team for Sahara.

We have this review here, from OpenStack proposal bot:

https://review.openstack.org/158775/

Since it came from the proposal bot, there's no justification in the
commit message and no cherry pick.

I didn't see this case covered as one of the strict set in

https://wiki.openstack.org/wiki/StableBranch

Do we trust the proposal bot? How do I know I should trust it? On
master, I assume if there
is a mistake it will soon be rectified, but stable ...  Do we have a doc
that talks about stable maintenance
and requirements changes?  Should we?

Am I being paranoid? :)

Thanks,

Trevor 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [devstack] [Cinder-GlusterFS CI] centos7 gate job abrupt failures

2015-02-24 Thread Bharat Kumar


Ran the job manually on rax VM, provided by Jeremy. (Thank you Jeremy).

After running 971 test cases VM inaccessible for 569 ticks, then 
continues... (Look at the console.log [1])

And also have a look at dstat log. [2]

The summary is:
==
Totals
==
Ran: 1125 tests in 5835. sec.
 - Passed: 960
 - Skipped: 88
 - Expected Fail: 0
 - Unexpected Success: 0
 - Failed: 77
Sum of execute time for each test: 13603.6755 sec.


[1] https://etherpad.openstack.org/p/rax_console.txt
[2] https://etherpad.openstack.org/p/rax_dstat.log

On 02/24/2015 07:03 PM, Deepak Shetty wrote:
FWIW, we tried to run our job in a rax provider VM (provided by ianw 
from his personal account)
and we ran the tempest tests twice, but the OOM did not re-create. Of 
the 2 runs, one of the run
used the same PYTHONHASHSEED as we had in one of the failed runs, 
still no oom.


Jeremy graciously agreed to provide us 2 VMs , one each from rax and 
hpcloud provider

to see if provider platform has anything to do with it.

So we plan to run again wtih the VMs given from Jeremy , post which i 
will send

next update here.

thanx,
deepak


On Tue, Feb 24, 2015 at 4:50 AM, Jeremy Stanley > wrote:


Due to an image setup bug (I have a fix proposed currently), I was
able to rerun this on a VM in HPCloud with 30GB memory and it
completed in about an hour with a couple of tempest tests failing.
Logs at: http://fungi.yuggoth.org/tmp/logs3.tar

Rerunning again on another 8GB Rackspace VM with the job timeout
increased to 5 hours, I was able to recreate the network
connectivity issues exhibited previously. The job itself seems to
have run for roughly 3 hours while failing 15 tests, and the worker
was mostly unreachable for a while at the end (I don't know exactly
how long) until around the time it completed. The OOM condition is
present this time too according to the logs, occurring right near
the end of the job. Collected logs are available at:
http://fungi.yuggoth.org/tmp/logs4.tar

Given the comparison between these two runs, I suspect this is
either caused by memory constraints or block device I/O performance
differences (or perhaps an unhappy combination of the two).
Hopefully a close review of the logs will indicate which.
--
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


--
Warm Regards,
Bharat Kumar Kobagana
Software Engineer
OpenStack Storage – RedHat India
Mobile - +91 9949278005

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Kerberos in OpenStack

2015-02-24 Thread Adam Young

On 02/24/2015 01:53 PM, Sanket Lawangare wrote:

Hello Everyone,

My name is Sanket Lawangare. I am a graduate Student studying at The
University of Texas, at San Antonio.For my Master’s Thesis I am
working on the Identity component of OpenStack. My research is to
investigate external authentication with Identity(keystone) using
Kerberos.

Based on reading Jammie lennox's Blogs on Kerberos implementation in
OpenStack and my understanding of Kerberos I have come up with a
figure explaining possible interaction of KDC with the OpenStack
client, keystone and the OpenStack services(Nova, Cinder, Swift...).

These are the Blogs -

http://www.jamielennox.net/blog/2015/02/12/step-by-step-kerberized-keystone/

http://www.jamielennox.net/blog/2013/10/22/keystone-token-binding/

I am trying to understand the working of Kerberos in OpenStack.

Please click this link to view the figure:
https://docs.google.com/drawings/d/1re0lNbiMDTbnkrqGMjLq6oNoBtR_GA0x7NWacf0Ulbs/edit?usp=sharing

P.S. - [The steps in this figure are self explanatory the basic
understanding of Kerberos is expected]

Based on the figure i had couple of questions:

Is Nova or other services registered with the KDC?

Not yet. Kerberos is only used for Keystone at the moment, with work
underway to make Horizon work with Keystone. Since many of the services
only run in Eventlet, not in HTTPD, Kerberos support is hard to
support. Ideally, yes, we would do Kerberos direct to Nova, and weither
use the token binding mechanism, or better yet, not even provide a
token...but that is more work.

What does keystone do with Kerberos ticket/credentials? Does
Keystone authenticates the users and gives them direct access to
other services such as Nova, Swift etc..

THey are used for authentication, and then the Keystone server uses the
principal to resolve the username and user id. The rest of the data
comes out of LDAP.

After receiving the Ticket from the KDC does keystone embed some
kerberos credential information in the token?

No, it is mapped to the Openstack userid and username

What information does the service (e.g.Nova) see in the Ticket and
the token (Does the token have some kerberos info or some
customized info inside it?).

No kerberos ticket goes to Nova.

If you could share your insights and guide me on this. I would be
really appreciate it. Thank you all for your time.

Let me know if you have more questions. Really let me know if you want
to help coding.

Regards,

Sanket Lawangare

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Kerberos in OpenStack

2015-02-24 Thread Tim Bell

You may also get some information from how we set up Kerberos at CERN at 
http://openstack-in-production.blogspot.fr/2014/10/kerberos-and-single-sign-on-with.html

From my understanding, the only connection is between Keystone and KDC. There 
is a standard Keystone token issues based off the Kerberos ticket and the rest 
is the same as if a password had been supplied.

Tim

From: Sanket Lawangare [mailto:sanket.lawang...@gmail.com]
Sent: 24 February 2015 19:53
To: openstack-dev@lists.openstack.org
Subject: [openstack-dev] Kerberos in OpenStack

Hello  Everyone,


My name is Sanket Lawangare. I am a graduate Student studying at The University 
of Texas, at San Antonio. For my Master’s Thesis I am working on the Identity 
component of OpenStack. My research is to investigate external authentication 
with Identity(keystone) using Kerberos.


Based on reading Jammie lennox's Blogs on Kerberos implementation in OpenStack 
and my understanding of Kerberos I have come up with a figure explaining 
possible interaction of KDC with the OpenStack client, keystone and the 
OpenStack services(Nova, Cinder, Swift...).

These are the Blogs -

http://www.jamielennox.net/blog/2015/02/12/step-by-step-kerberized-keystone/

http://www.jamielennox.net/blog/2013/10/22/keystone-token-binding/

I am trying to understand the working of Kerberos in OpenStack.


Please click this link to view the figure: 
https://docs.google.com/drawings/d/1re0lNbiMDTbnkrqGMjLq6oNoBtR_GA0x7NWacf0Ulbs/edit?usp=sharing


P.S. - [The steps in this figure are self explanatory the basic understanding 
of Kerberos is expected]


Based on the figure i had couple of questions:


1. Is Nova or other services registered with the KDC?


2. What does keystone do with Kerberos ticket/credentials? Does Keystone 
authenticates the users and gives them direct access to other services such as 
Nova, Swift etc..


3. After receiving the Ticket from the KDC does keystone embed some 
kerberos credential information in the token?


4. What information does the service (e.g.Nova) see in the Ticket and the 
token (Does the token have some kerberos info or some customized info inside 
it?).


If you could share your insights and guide me on this. I would be really 
appreciate it. Thank you all for your time.


Regards,

Sanket Lawangare
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] tracking bugs superseded by blueprints

2015-02-24 Thread Aleksey Kasatkin

I think it is better to keep such bugs open. Please see
https://blueprints.launchpad.net/fuel/+spec/granular-network-functions .
There are some related bugs here. One is fixed, another one is in progress,
two are closed. If bug is strictly coherent with blueprint (like
https://bugs.launchpad.net/fuel/+bug/1355764 for this BP) is can be closed
almost without doubt. But some of them can be solved separately somehow or
have workarounds. Sometimes scope of BP can be changed (e.g. split to
several BPs) or its timeline is changed so bugs should not be lost without
care.



Aleksey Kasatkin


On Tue, Feb 24, 2015 at 12:01 AM, Mike Scherbakov 
wrote:

> Bogdan,
> I think we should keep bugs open and not "supersed" them by blueprint. I
> see following reasons for it.
>
> Often, we can find workaround in order to fix the bug. Even if bug
> naturally seems to be falling into some blueprint's scope. Then problem is
> that when you close the bug, you don't even try to think about workaround -
> and project gets shipped with some serious gaps from release to release.
>
> Another issue is that you lose real technical requirements for blueprints.
> If you keep bugs open and associated with blueprint, you will pass by bugs
> a few times before you deliver blueprint's functionality, and finally close
> bugs if code covers bug's case. At least, I'd like it to be so.
>
> Finally, QA and users will keep opening duplicates, as no one will be
> happy with won't fix. You can vote for bug (by "affecting" it), and you
> can't for blueprint in LP, unfortunately. This just keeps door open for
> getting feedback.
>
> I don't really see any cons of moving bugs into Won't fix state instead.
>
> Examples of bugs which I would certainly avoid putting into Won't fix:
> https://bugs.launchpad.net/bugs/1398817 - disable computes by default
> during scale up
> https://bugs.launchpad.net/fuel/+bug/1422856 - separate /var & /var/log
> on master node
>
> Thanks,
>
> On Wed, Feb 18, 2015 at 8:46 PM, Andrew Woodward  wrote:
>
>> Bogdan,
>>
>> Yes I think tracking the bugs like this would be beneficial. We should
>> also link them from the BP so that the imperilmenter can track them. It
>> adds "related blueprints" in the bottom of the right column under the
>> subscribers so we probably should also edit the description so that the
>> data is easy to see
>>
>> On Wed, Feb 18, 2015 at 8:12 AM, Bogdan Dobrelya 
>> wrote:
>>
>>> Hello.
>>> There is inconsistency in the triage process for Fuel bugs superseded by
>>> blueprints.
>>> The current approach is to set "won't fix" status for such bugs.
>>> But there are some cases we should clarify [0], [1].
>>>
>>> I vote to not track superseded bugs separately and keep them as "won't
>>> fix" but update the status back to "confirmed" in case of regression
>>> discovered. And if we want to backport an improvement tracked by a
>>> blueprint (just for an exceptional case) let's assign milestones for
>>> related bugs.
>>>
>>> If we want to change the triage rules, let's announce that so the people
>>> won't get confused.
>>>
>>> [0] https://bugs.launchpad.net/fuel/+bug/1383741
>>> [1] https://bugs.launchpad.net/fuel/+bug/1422856
>>>
>>> --
>>> Best regards,
>>> Bogdan Dobrelya,
>>> Skype #bogdando_at_yahoo.com 
>>> Irc #bogdando
>>>
>>>
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>>
>> --
>> Andrew
>> Mirantis
>> Fuel community ambassador
>> Ceph community
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> --
> Mike Scherbakov
> #mihgen
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Tempest] Tempest will deny extra properties on Nova v2/v2.1 API

2015-02-24 Thread David Kranz


On 02/24/2015 06:55 AM, Ken'ichi Ohmichi wrote:

Hi Ghanshyam,

2015-02-24 20:28 GMT+09:00 GHANSHYAM MANN :

On Tue, Feb 24, 2015 at 6:48 PM, Ken'ichi Ohmichi 
wrote:

Hi

Nova team is developing Nova v2.1 API + microversions in this cycle,
and the status of Nova v2.1 API has been changed to CURRENT from
EXPERIMENTAL.
That said new API properties should be added via microversions, and
v2/v2.1 API(*without* microversions) should return the same response
without any new properties.
Now Tempest allows extra properties of a Nova API response because we
thought Tempest should not block Nova API development.

However, I think Tempest needs to deny extra properties in
non-microversions test cases because we need to block accidental
changes of v2/v2.1 API and encourage to use microversions for API
changes.
https://review.openstack.org/#/c/156130/ is trying to do that, but I'd
like to get opinions before that.

If the above change is merged, we can not use Tempest on OpenStack
environments which provide the original properties.


I think that will be nice to block additional properties.

Do you mean OpenStack environment with micro-versions enabled?
In those cases too tempest should run successfully as it requests on V2 or
V2.1 endpoint not on microversion.

My previous words were unclear, sorry.
The above "OpenStack environment" means the environment which is
customized by a cloud service provider and it returns a response which
includes the provider original properties.

On microversions discussion, we considered the customized API by
a cloud service provider for the design. Then I guess there are some
environments return extra properties and Tempest will deny them if
the patch is merged. I'd like to know the situation is acceptable or not
as Tempest purpose.
Ken'ichi, can you please provide a pointer to the referenced 
microversions discussion and/or summarize the conclusion?


The commit message is saying that returning extra values without a new 
microversion is an incompatible (disallowed) change. This was already 
true, unless creating a new extension, according to 
https://wiki.openstack.org/wiki/APIChangeGuidelines.


Seems to me that extra properties (unless using a syntax marking them as 
such), are either allowed or not. If not, tempest should fail on them. 
If service providers are allowed to add returned properties, and not 
required to use some special syntax to distinguish them, that is a bad 
api. If tempest can't tell the difference between a legitimate added 
property and some one misspelling while returning an optional property, 
I'm not sure how we test for the unintentional change case.


 -David



Thanks
Ken Ohmichi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] ECMP on Neutron virtual router

2015-02-24 Thread Kevin Benton

I wonder if there is a way we can easily abuse the extra routes extension
to do this? Maybe two routes to the same network would imply ECMP.

If not, maybe this can fit into a larger refactoring for route management
(dynamic routing, etc).
On Feb 24, 2015 11:02 AM, "Carl Baldwin"  wrote:

> It doesn't support this at this time.  There are no current plans to
> make it work.  I'm curious to know how you would like for this to work
> in your deployment.
>
> Carl
>
> On Tue, Feb 24, 2015 at 11:32 AM, NAPIERALA, MARIA H 
> wrote:
> > Does Neutron router support ECMP across multiple static routes to the
> same
> > destination network but with different next-hops?
> >
> > Maria
> >
> >
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Congress][Delegation] Initial workflow design

2015-02-24 Thread Yathiraj Udupi (yudupi)

Hi Tim,

Thanks for your updated doc on Delegation from Congress to a domain-specific 
policy engine, in this case, you are planning to build a LP-based VM-Placement 
engine to be the domain specific policy engine.
I agree your main goal is to first get the delegation interface sorted out.  It 
will be good so that external services (like Solver-Scheduler) can also easily 
integrate to the delegation model.

>From the Solver-Scheduler point of view,  we would actually want to start 
>working on a PoC effort to start integrating Congress and the Solver-Scheduler.
We believe rather than pushing this effort to a long-term,  it would add value 
to both the Solver Scheduler effort, as well as the Congress effort to try some 
early integration now, as most of the LP solver work for VM placements is ready 
available now in Solver scheduler, and we need to spend some time thinking 
about translating your domain-agnostic policy to constraints that the Solver 
scheduler can use.

I would definitely need your help from the Congress interfaces and I hope you 
will share your early interfaces for the delegation, so I can start the effort 
from the Solver scheduler side for integration.
I will reach out to you to get some initial help for integration w.r.t. 
Congress, and also keep you posted about the progress from our side.

Thanks,
Yathi.

On 2/23/15, 11:28 AM, "Tim Hinrichs" 
mailto:thinri...@vmware.com>> wrote:

Hi all,

I made a heavy editing pass of the Delegation google doc, incorporating many of 
your comments and my latest investigations into VM-placement.  I left the old 
stuff in place at the end of the doc and put the new stuff at the top.

My goal was to propose an end-to-end workflow for a PoC that we could put 
together quickly to help us explore the delegation interface.  We should 
iterate on this design until we have something that we think is workable.   And 
by all means pipe up if you think we need a totally different starting point to 
begin the iteration.

(BTW I'm thinking of the integration with solver-scheduler as a long-term 
solution to VM-placement, once we get the delegation interface sorted out.)

https://docs.google.com/document/d/1ksDilJYXV-5AXWON8PLMedDKr9NpS8VbT0jIy_MIEtI/edit#

Tim
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] ECMP on Neutron virtual router

2015-02-24 Thread Carl Baldwin

It doesn't support this at this time.  There are no current plans to
make it work.  I'm curious to know how you would like for this to work
in your deployment.

Carl

On Tue, Feb 24, 2015 at 11:32 AM, NAPIERALA, MARIA H  wrote:
> Does Neutron router support ECMP across multiple static routes to the same
> destination network but with different next-hops?
>
> Maria
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Kerberos in OpenStack

2015-02-24 Thread Sanket Lawangare

Hello Everyone,

My name is Sanket Lawangare. I am a graduate Student studying at The
University of Texas, at San Antonio. For my Master’s Thesis I am working on
the Identity component of OpenStack. My research is to investigate external
authentication with Identity(keystone) using Kerberos.

Based on reading Jammie lennox's Blogs on Kerberos implementation in
OpenStack and my understanding of Kerberos I have come up with a figure
explaining possible interaction of KDC with the OpenStack client, keystone
and the OpenStack services(Nova, Cinder, Swift...).

These are the Blogs -

http://www.jamielennox.net/blog/2015/02/12/step-by-step-kerberized-keystone/

http://www.jamielennox.net/blog/2013/10/22/keystone-token-binding/

I am trying to understand the working of Kerberos in OpenStack.

Please click this link to view the figure:
https://docs.google.com/drawings/d/1re0lNbiMDTbnkrqGMjLq6oNoBtR_GA0x7NWacf0Ulbs/edit?usp=sharing

P.S. - [The steps in this figure are self explanatory the basic
understanding of Kerberos is expected]

Based on the figure i had couple of questions:

Is Nova or other services registered with the KDC?

What does keystone do with Kerberos ticket/credentials? Does Keystone
authenticates the users and gives them direct access to other services such
as Nova, Swift etc..

After receiving the Ticket from the KDC does keystone embed some
kerberos credential information in the token?

What information does the service (e.g.Nova) see in the Ticket and the
token (Does the token have some kerberos info or some customized info
inside it?).

If you could share your insights and guide me on this. I would be really
appreciate it. Thank you all for your time.

Regards,

Sanket Lawangare
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] novaclient functional test guidelines

On Tue, Feb 24, 2015 at 9:47 AM, Sean Dague  wrote:

> Towards the end of merging the regression test for the nova
> volume-attach bug - https://review.openstack.org/#/c/157959/ there was a
> discussion around what style the functional tests should take.
> Especially as that had a mix of CLI and API calls in it.
>


Thanks for starting this thread.  Once we reach general agreement lets put
this in a in tree README for record keeping.


>
> Here are my thoughts for why that test ended up that way:
>
> 1) All resource setup that is table stakes for the test should be done
> via the API, regardless if it's a CLI or API test.
>
> The reason for this is that structured data is returned, which removes
> one possible error in the tests by parsing incorrectly. The API objects
> returned also include things like .delete methods in most cases, which
> means that addCleanup is a little more clean.
>

IMHO the CLI should have an option to returned raw JSON back instead of
pretty tabled results as well.


>
> 2) Main logic should touch which ever interface you are trying to test.
> This was demonstrating a CLI regression, so the volume-attach call was
> important to be done over the CLI.
>
>
> Now... here's where theory runs into issues.
>
> #1 - nova boot is table stakes. Under the above guidelines it should be
> called via API. However --poll is a CLI construct and so saved a custom
> wait loop here. We should implement that custom wait loop down the road
> and make that an API call
>
>
> https://github.com/openstack/python-novaclient/blob/master/novaclient/tests/functional/test_instances.py#L116


This issue is from a real short coming in the python client. So this isn't
really an issue with your theory, just an issue with novaclient.


>
>
> #2 - the volume create command is table stakes. It should be an API
> call. However, it can't be because the service catalog redirection only
> works at the CLI layer. This is actually also the crux of bug #1423695.
> The same reason the completion cache code failed is the reason we can't
> use the API for that.
>
>
> https://github.com/openstack/python-novaclient/blob/master/novaclient/tests/functional/test_instances.py#L129


Issues like this are why I wrote the read only nova CLI tests in the first
place. Unit testing the python API is doable, but unit testing the CLI is a
little bit more tricky. So I expect issues like this to come up over and
over again.


>
>
> #3 - the cleanup of the volume should have been API call. See reason for
> #2.
>
>
> https://github.com/openstack/python-novaclient/blob/master/novaclient/tests/functional/test_instances.py#L131
>
> #4 - the cleanup of the attachment should be an addCleanup via the API.
> See reason for #2 why it's not.
>
>
> https://github.com/openstack/python-novaclient/blob/master/novaclient/tests/functional/test_instances.py#L155
>
>
> I'm happy if there are other theories about how we do these things,
> being the first functional test in the python-novaclient tree that
> creates and destroys real resources, there isn't an established pattern
> yet. But I think doing all CLI calls in CLI tests is actually really
> cumbersome, especially in the amount of output parsing code needed if
> you are going to setup any complicated resource structure.
>
>
Here is an alternate theory:

We should have both python API and CLI functional tests. But they should be
kept separate.

This is to help us make sure both the CLI and python API are actually
usable interfaces. As the exercise above has shown, they both have really
major short comings.  I think having in tree functional testing covering
both the CLI and python API will make it easier for us to review new client
features in terms of usability.

Here is a very rough proof of concept patch showing the same tests:
https://review.openstack.org/#/c/157974/2/novaclient/tests/functional/test_volumes.py,cm

No matter how we define this functional testing model, I think its clear
novaclient needs a decent amount of work before it can really be usable.



> -Sean
>
> --
> Sean Dague
> http://dague.net
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [thirdpartyCI][cinder] Question about certification

2015-02-24 Thread Mike Perez

On 12:47 Tue 24 Feb , Eduard Matei wrote:
> The question is: does the CI need voting rights (validated) , or just
> check/comment to be considered "working"?

See:

https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers#When_thirdparty_CI_voting_will_be_required.3F

-- 
Mike Perez

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] ECMP on Neutron virtual router

2015-02-24 Thread NAPIERALA, MARIA H

Does Neutron router support ECMP across multiple static routes to the same 
destination network but with different next-hops?

Maria

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [stable][all] Revisiting the 6 month release cycle

2015-02-24 Thread Johannes Erdfelt

On Tue, Feb 24, 2015, Thierry Carrez  wrote:
> Agree on the pain of maintaining milestone plans though, which is why I
> propose we get rid of most of it in Liberty. That will actually be
> discussed at the cross-project meeting today:
> 
> https://wiki.openstack.org/wiki/Release_Cycle_Management/Liberty_Tracking

I'm happy to see this.

"Assignees may target their blueprint to a future milestone, as an
indication of when they intend to land it (not mandatory)"

That seems useless to me. I have no control over when things land. I can
only control when my code is put up for review.

Recently, I have spent a lot more time waiting on reviews than I have
spent writing the actual code.

JE


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Re-evaluating the suitability of the 6 month release cycle

On 02/24/2015 12:33 PM, John Griffith wrote:
> 
> 
> On Tue, Feb 24, 2015 at 10:04 AM, David Kranz  > wrote:
> 
> On 02/24/2015 09:37 AM, Chris Dent wrote:
> 
> On Tue, 24 Feb 2015, Sean Dague wrote:
> 
> That also provides a very concrete answer to "will people
> show up".
> Because if they do, and we get this horizontal refactoring
> happening,
> then we get to the point of being able to change release
> cadences
> faster. If they don't, we remain with the existing system.
> Vs changing
> the system and hoping someone is going to run in and
> backfill the breaks.
> 
> 
> Isn't this the way of the world? People only put halon in the
> machine room after the fire.
> 
> I agree that "people showing up" is a real concern, but I also think
> that we shy away too much from the productive energy of stuff
> breaking. It's the breakage that shows where stuff isn't good
> enough.
> 
> [Flavio said]:
> 
> To this I'd also add that bug fixing is way easier when you have
> aligned releases for projects that are expected to be deployed
> together. It's easier to know what the impact of a change/bug is
> throughout the infrastructure.
> 
> 
> Can't this be interpreted as an excuse for making software which
> does not have a low surface area and a good API?
> 
> (Note I'm taking a relatively unrealistic position for sake of
> conversation.)
> 
> I'm not so sure about that. IMO, much of this goes back to the
> question of whether OpenStack services are APIs or implementations.
> This was debated with much heat at the Diablo summit (Hi Jay). I
> frequently have conversations where there is an issue about release
> X vs Y when it is really about api versions. Even if we say that we
> are about implementations as well as apis, we can start to organize
> our processes and code as if we were just apis. If each service had
> a well-defined, versioned, discoverable, well-tested api, then
> projects could follow their own release schedule, relying on distros
> or integrators to put the pieces together and verify the quality of
> the whole stack to the users. Such entities could still collaborate
> on that task, and still identify longer release cycles, using
> "stable branches". The upstream project could still test the latest
> released versions together. Some of these steps are now being taken
> to resolve gate issues and horizontal resource issues. Doing this
> would vastly increase agility but with some costs:
> 
> 1. The upstream project would likely have to give up on the worthy
> goal of providing an actual deployable stack that could be used as
> an alternative to AWS, etc. That saddens me, but for various
> reasons, including that we do no scale/performance testing on the
> upstream code, we are not achieving that goal anyway. The big tent
> proposals are also a move away from that goal.
> 
> 2. We would have to give up on incompatible api changes. But with
> the replacement of nova v3 with microversions we are already doing
> that. Massive adoption with release agility is simply incompatible
> with allowing incompatible api changes.
> 
> Most of this is just echoing what Jay said. I think this is the way
> any SOA would be designed. If we did this, and projects released
> frequently, would there be a reason for any one to be chasing master?
> 
>  -David
> 
> 
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> OpenStack-dev-request@lists.__openstack.org?subject:__unsubscribe
> 
> http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev 
> 
> 
> 
> 
> Seems like some of the proposals around frequency (increasing in
> particular) just sort of move the bottlenecks around.  Honestly, I
> thought we were already on a path with some of the ideas that Sean and
> David (and indirectly Jay) proposed.  Get rid of the whole "coordinated
> release" altogether.  I think there should still be some sort of tagging
> or something at some interval that just says "here's a point in time
> collection that we call X".
> 
> Another proposal I think I've talked to some folks about is a true
> CI/Train model.  Cut out some of the artificial milestone deadlines etc,
> just keep rolling and what's ready at the release point is what's ready;
> you make a cut of what's there at that time and roll on.  Basically
> eliminate the feature freeze and other com

[openstack-dev] [nova] novaclient functional test guidelines