Re: [openstack-dev] Event Service

[Qing He]

Thanks Michael!
I found it:
https://wiki.openstack.org/wiki/NotificationEventExamples

-Original Message-
From: Michael Still [mailto:mi...@stillhq.com] 
Sent: Friday, July 12, 2013 6:38 PM
To: OpenStack Development Mailing List
Subject: Re: [openstack-dev] Event Service

OpenStack has a system called notifications which does what you're looking for. 
I've never used it, but I am sure its documented.

Cheers,
Michael

On Sat, Jul 13, 2013 at 10:12 AM, Qing He  wrote:
> All,
>
> Does open stack have pub/sub event service? I would like to be 
> notified of the event of VM creation/deletion/Migration etc. What is 
> the best way to do this?
>
>
>
> Thanks,
>
>
> Qing
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



--
Rackspace Australia

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Not good idea to specify host in EVACUATE API

[Chaoyi Huang]

Hello,

 I found it's not good idea to specify host in calling EVACUATE API for HA
purpose.

In my case, one application includes many VMs, and each of them should run
on seperate host, eg. co-location of VMs is not allowed for application
robust purpose. The VM is created with scheduler hint to avoiding
co-location.

If a host is specified in calling evacuate API,  then the task to find a
proper host (avoiding co-location) has to be done before calling, and the
logic is similar with nova scheduler.

The HA function must reuse the NOVA scheduler logic, but not to duplicate
it outside nova.

Your comment is welcome .


Joehuang
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Event Service

[Michael Still]

OpenStack has a system called notifications which does what you're
looking for. I've never used it, but I am sure its documented.

Cheers,
Michael

On Sat, Jul 13, 2013 at 10:12 AM, Qing He  wrote:
> All,
>
> Does open stack have pub/sub event service? I would like to be notified of
> the event of VM creation/deletion/Migration etc. What is the best way to do
> this?
>
>
>
> Thanks,
>
>
> Qing
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Rackspace Australia

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Review] Use of exception for non-exceptional cases

[Mark Washenberger]

I'm curious if folks know about this?


import sys

def foo():
raise Exception('Foo Exception')


def bar():
try:
foo()
except Exception:
exc_info = sys.exc_info()
raise Exception('Bar Exception'), None, exc_info[2]


bar()

"""
Traceback (most recent call last):
  File "test.py", line 15, in 
bar()
  File "test.py", line 9, in bar
foo()
  File "test.py", line 4, in foo
raise Exception('Foo Exception')
Exception: Bar Exception

"""


On Fri, Jul 12, 2013 at 3:53 PM, Doug Hellmann
wrote:

>
>
>
> On Fri, Jul 12, 2013 at 2:40 PM, Brant Knudson  wrote:
>
>>
>> Somewhat offtopic, but others might find it interesting. On another
>> project I used a different technique for exceptions, where the original
>> exception is "enhanced" with new fields as it propagates up the stack to
>> where it's handled. In this way all the information needed to handle the
>> exception is available (even if it's just to log it).
>>
>> Often there's some information that would be useful for the exception
>> handler that isn't available at the place where the exception is raised.
>> The classic example is an error writing to a file where you'd like to know
>> the filename but all you've got is the file descriptor. An example from an
>> OpenStack REST API might be that you get an exception and you'd like to log
>> a message that includes the resource path, but the resource path isn't
>> known at this point.
>>
>> So rather than logging the exception when it happens, enhance the
>> exception, re-raise it, and only once it's got all the information where
>> you can print out a useful log message you log it.
>>
>> Here's a short example to illustrate. An exception is raised by f1(), but
>> at this point we don't know the status code that the server should respond
>> with or the request path which would be useful in a log message. These bits
>> of information are added as the exception propagates and then where it's
>> caught we've got all the information for a useful log message.
>>
>> def f1():
>>   raise Exception('something')
>>
>>
>> def f2():
>>   try:
>> f1()
>>   except Exception as e:
>> e.status_code = 403
>> raise
>>
>>
>> def do_tokens():
>>   try:
>> f2()
>>   except Exception as e:
>> e.req_url = '/v3/tokens'
>> raise
>>
>>
>> status_code = 200
>> try:
>>   do_tokens()
>> except Exception as e:
>>   status_code = getattr(e, 'status_code', 500)
>>   req_url = getattr(e, 'req_url', 'unknown')
>>
>> if status_code != 200:
>>   print 'Got %s from %s' % (status_code, req_url)
>>   # build an error document for the response.
>>
>
>
> One problem with this approach is it spreads knowledge of the error
> construction up and down the stack through different layers of the
> application, and that brings with it assumptions about the implementation
> at the different layers. For example, should the application code above
> know that do_tokens() is making web requests to URLs? Why does it need that
> information?
>
> SRP-ly,
> Doug
>
>
>
>>
>>
>>
>> On Fri, Jul 12, 2013 at 12:25 PM, Nachi Ueno  wrote:
>>
>>> Hi folks
>>>
>>> > Monty
>>> Thank you for your adding good topic for this.
>>>
>>> I agree, hiding true cause by exception get hard to debug
>>>
>>> For, example, Sqlalchemy gives me this error for general sql errors..
>>> "exceptions must be old-style classes or derived from BaseException, not
>>> str"
>>>
>>> (It should be hidden from REST API users though)
>>>
>>> Also, I agree with you about log policies.
>>> sometimes 404 get warn log..
>>>
>>> IMO, The log more than warn level should help Operators.
>>> also If the exception is truly, the exceptional case which needs help
>>> of operators.
>>> It should be logged as error level.
>>>
>>> Thanks
>>> Nachi
>>>
>>>
>>>
>>> 2013/7/12 Dolph Mathews :
>>> >
>>> > On Fri, Jul 12, 2013 at 10:09 AM, Monty Taylor 
>>> wrote:
>>> >>
>>> >>
>>> >>
>>> >> On 07/11/2013 05:20 AM, Mark McLoughlin wrote:
>>> >> > On Wed, 2013-07-10 at 19:49 -0400, Monty Taylor wrote:
>>> >> >> I'd like top-post and hijack this thread for another exception
>>> related
>>> >> >> thing:
>>> >> >>
>>> >> >> a) Anyone writing code such as:
>>> >> >>
>>> >> >> try:
>>> >> >>   blah()
>>> >> >> except SomeException:
>>> >> >>   raise SomeOtherExceptionLeavingOutStackContextFromSomeException
>>> >> >>
>>> >> >> should be mocked ruthlessly.
>>> >> >
>>> >> > Ok, mock me ruthlessly then.
>>> >>
>>> >> Mock. Mock. Mock. Mock.
>>> >>
>>> >> > Part of designing any API is specifying what predictable exceptions
>>> it
>>> >> > will raise. For any predictable error condition, you don't want
>>> callers
>>> >> > to have to catch random exceptions from the underlying libraries you
>>> >> > might be calling into.
>>> >>
>>> >> Absolutely. But you don't want to go so overboard that you remove the
>>> >> ability for a developer to debug it. More importantly, INSIDE of
>>> server
>>> >> code, we're not designing fun apis for the consumption of people we
>>> >> 

[openstack-dev] Event Service

[Qing He]

All,
Does open stack have pub/sub event service? I would like to be notified of the 
event of VM creation/deletion/Migration etc. What is the best way to do this?

Thanks,

Qing
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [OSLO] Comments/Questions on Messaging Wiki

[William Henry]

Sent from my iPhone

On Jul 12, 2013, at 5:27 PM, Doug Hellmann  wrote:

> 
> 
> 
> On Fri, Jul 12, 2013 at 5:40 PM, William Henry  wrote:
>> Hi all,
>> 
>> I've been reading through the Messaging Wiki and have some comments. Not 
>> criticisms, just comments and questions.
>>  I have found this to be a very useful document. Thanks.
>> 
>> 1. "There are multiple backend transport drivers which implement the API 
>> semantics using different messaging systems - e.g. RabbitMQ, Qpid, ZeroMQ. 
>> While both sides of a connection must use the same transport driver 
>> configured in the same way, the API avoids exposing details of transports so 
>> that code written using one transport should work with any other transport."
>> 
>> The good news for AMQP 1.0 users is that technically "boths sides of the 
>> connection" do not have to use same transport driver. In pre-AMQP 1.0 days 
>> this was the case. But today interoperability between AMQP 1.0 
>> implementations has been demonstrated.
> 
> In this case I think we mean the Transport driver from within Oslo. So you 
> could not connect a ZMQ Transport on one end to an AMQP Transport on the 
> other. It will be an implementation detail of the AMQP Transport class to 
> decide whether it supports more than one version of AMQP, or if the different 
> versions are implemented as different Transports.
>  
>> 
>> 2. I notice under the RPC concepts section that you mention Exchanges as a 
>> container in which topics are scoped. Is this exchange a pre AMQP 1.0 
>> artifact or just a general term for oslo.messaging that is loosely based on 
>> the pre-AMQP 1.0 artifact called an Exchange? i.e. are you assuming that 
>> messaging implementations have something called an exchange? Or do you mean 
>> that messaging implementations can scope a topic and in oslo we call that 
>> scoping an exchange?
> 
> The latter.

Ack. Good. Fits very well into AMQP 1.0 then ;-)

>  
>> 
>> 3. Some messaging nomenclature: The way the wiki describes RPC "Invoke 
>> Method on One of Multiple Servers" is more like a queue than a topic. In 
>> messaging a queue is something that multiple consumers can attach to and one 
>> of them gets and services a message/request.  A topic is where 1+ consumers 
>> are "connected" and each receives a the message and each can service it as 
>> it sees fit.  In pre-AMQP 1.0 terms what this seems to describe is a direct 
>> exchange. And a direct excahnge can have multiple consumers listening to a 
>> queue on that exchange.  (Remember that fanout is just a generalization of 
>> topic in that all consumers get all fanout messages - there are no 
>> sub-topics etc.)
>> 
>> In AMQP 1.0 the addressing doesn't care or know about exchanges but it can 
>> support this queue type behavior on an address or topic type behavior on an 
>> address. 
>> 
>> I know this isn't about AMQP specifically but therefore this is even more 
>> important. Topics are pub/sub with multiple consumer/services responding to 
>> a single message. Queues are next consumer up gets the next message. 
>> 
>> 
>> (BTW I've seen this kind of confusion also in early versions of MCollective 
>> in Puppet.)
>> 
>> It might be better to change some of the references to "topic" to "address". 
>> This would solve the problem. i.e. a use case where one of many servers 
>> listening on an address services a message/request. And later all of servers 
>> listening on an address service a message/request. Addressing also solves 
>> the one-to-one as the address is specific to the server (and the others 
>> don't have to receive and reject the message).
> 
> Too many of these terms are overloaded. :-)

Yep. But topic pup/sub is certainly different to a queue. ;-)

> 
> I'm not sure of the details of how "topic" and "address" are different in 
> AMQP 1.0. The word "address" implies to me that the message sender knows 
> where the message receiver is in some concrete sense. We don't want those 
> semantics in a lot of our use cases. If the "address" is abstract, then it 
> sounds like it works much as a topic does. Maybe you can expand on the 
> differences?


Nope the address is essentially a namespace. The send knows not where it ends 
up. Hence in some applications it doesn't even know of its a topic or a queue 
an it could go to one or many depending.

William

> 
> Thanks,
> Doug
> 
>  
>> 
>> Please feel free to respond and critique my comments/suggestions.
>> 
>> Best,
>> William
>> 
>> 
>> 
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin

Re: [openstack-dev] [OSLO] Comments/Questions on Messaging Wiki

[Doug Hellmann]

On Fri, Jul 12, 2013 at 5:40 PM, William Henry  wrote:

> Hi all,
>
> I've been reading through the Messaging Wiki and have some comments. Not
> criticisms, just comments and questions.
>  I have found this to be a very useful document. Thanks.
>
> 1. "There are multiple backend transport drivers which implement the API
> semantics using different messaging systems - e.g. RabbitMQ, Qpid, ZeroMQ.
> While both sides of a connection must use the same transport driver
> configured in the same way, the API avoids exposing details of transports
> so that code written using one transport should work with any other
> transport."
>
> The good news for AMQP 1.0 users is that technically "boths sides of the
> connection" do not have to use same transport driver. In pre-AMQP 1.0 days
> this was the case. But today interoperability between AMQP 1.0
> implementations has been demonstrated.
>

In this case I think we mean the Transport driver from within Oslo. So you
could not connect a ZMQ Transport on one end to an AMQP Transport on the
other. It will be an implementation detail of the AMQP Transport class to
decide whether it supports more than one version of AMQP, or if the
different versions are implemented as different Transports.


>
> 2. I notice under the RPC concepts section that you mention Exchanges as a
> container in which topics are scoped. Is this exchange a pre AMQP 1.0
> artifact or just a general term for oslo.messaging that is loosely based on
> the pre-AMQP 1.0 artifact called an Exchange? i.e. are you assuming that
> messaging implementations have something called an exchange? Or do you mean
> that messaging implementations can scope a topic and in oslo we call that
> scoping an exchange?
>

The latter.


>
> 3. Some messaging nomenclature: The way the wiki describes RPC "Invoke
> Method on One of Multiple Servers" is more like a queue than a topic. In
> messaging a queue is something that multiple consumers can attach to and
> one of them gets and services a message/request.  A topic is where 1+
> consumers are "connected" and each receives a the message and each can
> service it as it sees fit.  In pre-AMQP 1.0 terms what this seems to
> describe is a direct exchange. And a direct excahnge can have multiple
> consumers listening to a queue on that exchange.  (Remember that fanout is
> just a generalization of topic in that all consumers get all fanout
> messages - there are no sub-topics etc.)
>
> In AMQP 1.0 the addressing doesn't care or know about exchanges but it can
> support this queue type behavior on an address or topic type behavior on an
> address.
>
> I know this isn't about AMQP specifically but therefore this is even more
> important. Topics are pub/sub with multiple consumer/services responding to
> a single message. Queues are next consumer up gets the next message.
>

>
> (BTW I've seen this kind of confusion also in early versions of
> MCollective in Puppet.)
>
> It might be better to change some of the references to "topic" to
> "address". This would solve the problem. i.e. a use case where one of many
> servers listening on an address services a message/request. And later all
> of servers listening on an address service a message/request. Addressing
> also solves the one-to-one as the address is specific to the server (and
> the others don't have to receive and reject the message).
>

Too many of these terms are overloaded. :-)

I'm not sure of the details of how "topic" and "address" are different in
AMQP 1.0. The word "address" implies to me that the message sender knows
where the message receiver is in some concrete sense. We don't want those
semantics in a lot of our use cases. If the "address" is abstract, then it
sounds like it works much as a topic does. Maybe you can expand on the
differences?

Thanks,
Doug



>
> Please feel free to respond and critique my comments/suggestions.
>
> Best,
> William
>
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Review] Use of exception for non-exceptional cases

[Doug Hellmann]

On Fri, Jul 12, 2013 at 2:40 PM, Brant Knudson  wrote:

>
> Somewhat offtopic, but others might find it interesting. On another
> project I used a different technique for exceptions, where the original
> exception is "enhanced" with new fields as it propagates up the stack to
> where it's handled. In this way all the information needed to handle the
> exception is available (even if it's just to log it).
>
> Often there's some information that would be useful for the exception
> handler that isn't available at the place where the exception is raised.
> The classic example is an error writing to a file where you'd like to know
> the filename but all you've got is the file descriptor. An example from an
> OpenStack REST API might be that you get an exception and you'd like to log
> a message that includes the resource path, but the resource path isn't
> known at this point.
>
> So rather than logging the exception when it happens, enhance the
> exception, re-raise it, and only once it's got all the information where
> you can print out a useful log message you log it.
>
> Here's a short example to illustrate. An exception is raised by f1(), but
> at this point we don't know the status code that the server should respond
> with or the request path which would be useful in a log message. These bits
> of information are added as the exception propagates and then where it's
> caught we've got all the information for a useful log message.
>
> def f1():
>   raise Exception('something')
>
>
> def f2():
>   try:
> f1()
>   except Exception as e:
> e.status_code = 403
> raise
>
>
> def do_tokens():
>   try:
> f2()
>   except Exception as e:
> e.req_url = '/v3/tokens'
> raise
>
>
> status_code = 200
> try:
>   do_tokens()
> except Exception as e:
>   status_code = getattr(e, 'status_code', 500)
>   req_url = getattr(e, 'req_url', 'unknown')
>
> if status_code != 200:
>   print 'Got %s from %s' % (status_code, req_url)
>   # build an error document for the response.
>


One problem with this approach is it spreads knowledge of the error
construction up and down the stack through different layers of the
application, and that brings with it assumptions about the implementation
at the different layers. For example, should the application code above
know that do_tokens() is making web requests to URLs? Why does it need that
information?

SRP-ly,
Doug



>
>
>
> On Fri, Jul 12, 2013 at 12:25 PM, Nachi Ueno  wrote:
>
>> Hi folks
>>
>> > Monty
>> Thank you for your adding good topic for this.
>>
>> I agree, hiding true cause by exception get hard to debug
>>
>> For, example, Sqlalchemy gives me this error for general sql errors..
>> "exceptions must be old-style classes or derived from BaseException, not
>> str"
>>
>> (It should be hidden from REST API users though)
>>
>> Also, I agree with you about log policies.
>> sometimes 404 get warn log..
>>
>> IMO, The log more than warn level should help Operators.
>> also If the exception is truly, the exceptional case which needs help
>> of operators.
>> It should be logged as error level.
>>
>> Thanks
>> Nachi
>>
>>
>>
>> 2013/7/12 Dolph Mathews :
>> >
>> > On Fri, Jul 12, 2013 at 10:09 AM, Monty Taylor 
>> wrote:
>> >>
>> >>
>> >>
>> >> On 07/11/2013 05:20 AM, Mark McLoughlin wrote:
>> >> > On Wed, 2013-07-10 at 19:49 -0400, Monty Taylor wrote:
>> >> >> I'd like top-post and hijack this thread for another exception
>> related
>> >> >> thing:
>> >> >>
>> >> >> a) Anyone writing code such as:
>> >> >>
>> >> >> try:
>> >> >>   blah()
>> >> >> except SomeException:
>> >> >>   raise SomeOtherExceptionLeavingOutStackContextFromSomeException
>> >> >>
>> >> >> should be mocked ruthlessly.
>> >> >
>> >> > Ok, mock me ruthlessly then.
>> >>
>> >> Mock. Mock. Mock. Mock.
>> >>
>> >> > Part of designing any API is specifying what predictable exceptions
>> it
>> >> > will raise. For any predictable error condition, you don't want
>> callers
>> >> > to have to catch random exceptions from the underlying libraries you
>> >> > might be calling into.
>> >>
>> >> Absolutely. But you don't want to go so overboard that you remove the
>> >> ability for a developer to debug it. More importantly, INSIDE of server
>> >> code, we're not designing fun apis for the consumption of people we
>> >> don't know. There is clearly an API, but we are the consumers of our
>> own
>> >> API.
>> >
>> >
>> > Lies! Write everything to be intuitive for new contributors or we won't
>> have
>> > any :(
>> >
>> >>
>> >>
>> >> > Say if I was designing an image downloading API, I'd do something
>> like
>> >> > this:
>> >> >
>> >> >   https://gist.github.com/markmc/5973868
>> >> >
>> >> > Assume there's a tonne more stuff that the API would do. You don't
>> want
>> >> > callers to have to catch socket.error exceptions and whatever other
>> >> > exceptions might be thrown.
>> >> >
>> >> > That works out as:
>> >> >
>> >> >   Traceback (most recent call last):
>> >> > File "t.py", line 20, in 
>> >> >

Re: [openstack-dev] AMQP Version upgrade plans?

[William Henry]

- Original Message -
> On 07/11/2013 12:06 PM, William Henry wrote:
> > 
> > 
> > - Original Message -
> >> On 07/08/2013 10:51 AM, Ted Ross wrote:
> >>> If someone from the Qpid community were to work on integrating the new
> >>> AMQP 1.0 technology into OpenStack, where would be the right place to
> >>> start?  Would it be to add a new transport to oslo.messaging?
> >>
> >> I think so, yes.  oslo.messaging is new, but it will deprecate the
> >> existing 'rpc' library in oslo-incubator.  All projects will need to
> >> move to oslo.messaging, so for something new I would focus efforts there.
> > 
> > I think that one of the important points that Ted brought up is that AMQP
> > 1.0 doesn't have the concepts of broker artifacts like exchanges etc.
> > 
> > A recent change I proposed to the existing impl_qpid.py which focuses more
> > on addressing and not exchanges is a very important first step to solve
> > several issues: a recent qpidd leak issue, transitioning to AMQP 1.0
> > (addressing), and possible HA solutions.
> > 
> > This is an area I'd really like to continue to help out in. I'm back from
> > some vacation and would like to get stuck in soon.
> 
> Regarding the qpid exchange leak, that issue is mitigated largely by the
> fact that the only time we declare a direct exchange is for replies to
> an rpc.  In previous versions there was a new one of these for *every*
> method call, which made this problem really bad.  In the current code,
> we only create a single one.
> 
> However, we're still left with a leak.  The fact that RabbitMQ supports
> auto-delete on exchanges and Qpid doesn't is what got us into this spot,
> since this code works just like the kombu driver with respect to all of
> this.

Right you still have the leak. Right auto-delete on exchanges in Qpid would 
mitigate. 
 
Qpid never supported this because, to my knowledge, auto-delete on exchanges 
didn't mean anything or have any value in Qpid. I'm not sure it was ever 
mentioned in the old pre AMQP 1.0 spec. (I'd need to check).

And anyway exchanges don't mean anything in AMQP anymore. And don't mean 
anything in nonAMQP messaging - though AMQP implementations are not prohibited 
from implementing the queue/topic/fanout using exchange patterns.

> 
> As for migration to AMQP 1.0, how do these changes help?  Supporting
> AMQP 1.0 requires an entirely new driver that uses Qpid Proton, right?
> How does changing addressing in the current Qpid driver (that will never
> do 1.0) help?

Actually Qpid project saw the pending AMQP 1.0 ramifications early on and 
created an addressing mechanism that supported AMQP 1.0 addressing in Qpid long 
before Qpid/Proton.

So the changes I made recently would mean a simplified driver migration process.

But you are correct, it would still need that migration to a new driver.

It's my opinion that moving from a driver I recently submitted to the new 
driver would be very simple because of the way I've restructured the addressing.

All great points you raise Russell.

William

> 
> I'm curious what you mean by "possible HA solutions".  Can you elaborate?
> 
> --
> Russell Bryant
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [OSLO] Comments/Questions on Messaging Wiki

[William Henry]

Woops the wiki I am referring to is: 

https://wiki.openstack.org/wiki/Oslo/Messaging 

William 

- Original Message -

> Hi all,

> I've been reading through the Messaging Wiki and have some comments. Not
> criticisms, just comments and questions.
> I have found this to be a very useful document. Thanks.

> 1. "There are multiple backend transport drivers which implement the API
> semantics using different messaging systems - e.g. RabbitMQ, Qpid, ZeroMQ.
> While both sides of a connection must use the same transport driver
> configured in the same way, the API avoids exposing details of transports so
> that code written using one transport should work with any other transport."

> The good news for AMQP 1.0 users is that technically "boths sides of the
> connection" do not have to use same transport driver. In pre-AMQP 1.0 days
> this was the case. But today interoperability between AMQP 1.0
> implementations has been demonstrated.

> 2. I notice under the RPC concepts section that you mention Exchanges as a
> container in which topics are scoped. Is this exchange a pre AMQP 1.0
> artifact or just a general term for oslo.messaging that is loosely based on
> the pre-AMQP 1.0 artifact called an Exchange? i.e. are you assuming that
> messaging implementations have something called an exchange? Or do you mean
> that messaging implementations can scope a topic and in oslo we call that
> scoping an exchange?

> 3. Some messaging nomenclature: The way the wiki describes RPC " Invoke
> Method on One of Multiple Servers " is more like a queue than a topic. In
> messaging a queue is something that multiple consumers can attach to and one
> of them gets and services a message/request. A topic is where 1+ consumers
> are "connected" and each receives a the message and each can service it as
> it sees fit. In pre-AMQP 1.0 terms what this seems to describe is a direct
> exchange. And a direct excahnge can have multiple consumers listening to a
> queue on that exchange. (Remember that fanout is just a generalization of
> topic in that all consumers get all fanout messages - there are no
> sub-topics etc.)

> In AMQP 1.0 the addressing doesn't care or know about exchanges but it can
> support this queue type behavior on an address or topic type behavior on an
> address.

> I know this isn't about AMQP specifically but therefore this is even more
> important. Topics are pub/sub with multiple consumer/services responding to
> a single message. Queues are next consumer up gets the next message.

> (BTW I've seen this kind of confusion also in early versions of MCollective
> in Puppet.)

> It might be better to change some of the references to "topic" to "address".
> This would solve the problem. i.e. a use case where one of many servers
> listening on an address services a message/request. And later all of servers
> listening on an address service a message/request. Addressing also solves
> the one-to-one as the address is specific to the server (and the others
> don't have to receive and reject the message).

> Please feel free to respond and critique my comments/suggestions.

> Best,
> William

> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [OSLO] Comments/Questions on Messaging Wiki

[William Henry]

Hi all, 

I've been reading through the Messaging Wiki and have some comments. Not 
criticisms, just comments and questions. 
I have found this to be a very useful document. Thanks. 

1. "There are multiple backend transport drivers which implement the API 
semantics using different messaging systems - e.g. RabbitMQ, Qpid, ZeroMQ. 
While both sides of a connection must use the same transport driver configured 
in the same way, the API avoids exposing details of transports so that code 
written using one transport should work with any other transport." 

The good news for AMQP 1.0 users is that technically "boths sides of the 
connection" do not have to use same transport driver. In pre-AMQP 1.0 days this 
was the case. But today interoperability between AMQP 1.0 implementations has 
been demonstrated. 

2. I notice under the RPC concepts section that you mention Exchanges as a 
container in which topics are scoped. Is this exchange a pre AMQP 1.0 artifact 
or just a general term for oslo.messaging that is loosely based on the pre-AMQP 
1.0 artifact called an Exchange? i.e. are you assuming that messaging 
implementations have something called an exchange? Or do you mean that 
messaging implementations can scope a topic and in oslo we call that scoping an 
exchange? 

3. Some messaging nomenclature: The way the wiki describes RPC " Invoke Method 
on One of Multiple Servers " is more like a queue than a topic. In messaging a 
queue is something that multiple consumers can attach to and one of them gets 
and services a message/request. A topic is where 1+ consumers are "connected" 
and each receives a the message and each can service it as it sees fit. In 
pre-AMQP 1.0 terms what this seems to describe is a direct exchange. And a 
direct excahnge can have multiple consumers listening to a queue on that 
exchange. (Remember that fanout is just a generalization of topic in that all 
consumers get all fanout messages - there are no sub-topics etc.) 

In AMQP 1.0 the addressing doesn't care or know about exchanges but it can 
support this queue type behavior on an address or topic type behavior on an 
address. 

I know this isn't about AMQP specifically but therefore this is even more 
important. Topics are pub/sub with multiple consumer/services responding to a 
single message. Queues are next consumer up gets the next message. 

(BTW I've seen this kind of confusion also in early versions of MCollective in 
Puppet.) 

It might be better to change some of the references to "topic" to "address". 
This would solve the problem. i.e. a use case where one of many servers 
listening on an address services a message/request. And later all of servers 
listening on an address service a message/request. Addressing also solves the 
one-to-one as the address is specific to the server (and the others don't have 
to receive and reject the message). 

Please feel free to respond and critique my comments/suggestions. 

Best, 
William 


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone] sqlite doesn't support migrations

[Monty Taylor]

On 07/11/2013 01:12 PM, Dolph Mathews wrote:
> Just as a general statement, outside the scope of openstack, I don't
> think sqlite is intended to support schema evolution. From the sqlite
> docs [1]: "SQLite supports a limited subset of ALTER TABLE. [...] It is
> not possible to rename a column, remove a column, or add or remove
> constraints from a table."
> 
> We've been through hell trying to support migrations on sqlite, because
> we test against sqlite, and because we test our migrations... on sqlite.
> So, we've already shot ourselves in the foot. We're clearly moving
> towards gating against mysql + postgresql, so in the mean time, let's
> limit the amount of effort we put into further support sqlite migrations
> until we can safely rip it out altogether.
> 
> [1]: http://www.sqlite.org/lang_altertable.html

I agree. The reason to use sqlite in unitests and stuff is because it's
easy and doesn't require users and system things and everything. If
we're spending extra effort to maintain the simple thing, then it's
probably not a simple thing.

As an aside, (ignore the fact that I'm a former Drizzle core dev) it
might be worthwhile taking 30 minutes one day and exploring a drizzle
database test fixture. One of the things we did in drizzle was make it
not need any bootstrapping and to work sanely with no config files ...
so launching a drizzle on a spare port, running database tests against
it and then deleting it should actually be super simple - and at the
worst no harder than doing what glance does in their functional tests.


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Revert Pass instance host-id to Quantum using port bindings extension.

[Aaron Rosen]

Hi,


On Fri, Jul 12, 2013 at 6:47 AM, Robert Kukura  wrote:

> On 07/11/2013 04:30 PM, Aaron Rosen wrote:
> > Hi,
> >
> > I think we should revert this patch that was added here
> > (https://review.openstack.org/#/c/29767/). What this patch does is when
> > nova-compute calls into quantum to create the port it passes in the
> > hostname on which the instance was booted on. The idea of the patch was
> > that providing this information would "allow hardware device vendors
> > management stations to allow them to segment the network in a more
> > precise manager (for example automatically trunk the vlan on the
> > physical switch port connected to the compute node on which the vm
> > instance was started)."
> >
> > In my opinion I don't think this is the right approach. There are
> > several other ways to get this information of where a specific port
> > lives. For example, in the OVS plugin case the agent running on the
> > nova-compute node can update the port in quantum to provide this
> > information. Alternatively, quantum could query nova using the
> > port.device_id to determine which server the instance is on.
> >
> > My motivation for removing this code is I now have the free cycles to
> > work on
> > https://blueprints.launchpad.net/nova/+spec/nova-api-quantum-create-port
> >  discussed here
> > (http://lists.openstack.org/pipermail/openstack-dev/2013-May/009088.html)
>  .
> > This was about moving the quantum port creation from the nova-compute
> > host to nova-api if a network-uuid is passed in. This will allow us to
> > remove all the quantum logic from the nova-compute nodes and
> > simplify orchestration.
> >
> > Thoughts?
>
> Aaron,
>
> The ml2-portbinding BP I am currently working on depends on nova setting
> the binding:host_id attribute on a port before accessing
> binding:vif_type. The ml2 plugin's MechanismDrivers will use the
> binding:host_id with the agents_db info to see what (if any) L2 agent is
> running on that host, or what other networking mechanisms might provide
> connectivity for that host. Based on this, the port's binding:vif_type
> will be set to the appropriate type for that agent/mechanism.
>
> When an L2 agent is involved, the associated ml2 MechanismDriver will
> use the agent's interface or bridge mapping info to determine whether
> the agent on that host can connect to any of the port's network's
> segments, and select the specific segment (network_type,
> physical_network, segmentation_id) to be used. If there is no
> connectivity possible on the host (due to either no L2 agent or other
> applicable mechanism, or no mapping for any of the network's segment's
> physical_networks), the ml2 plugin will set the binding:vif_type
> attribute to BINDING_FAILED. Nova will then be able to gracefully put
> the instance into an error state rather than have the instance boot
> without the required connectivity.
>
> I don't see any problem with nova creating the port before scheduling it
> to a specific host, but the binding:host_id needs to be set before the
> binding:vif_type attribute is accessed. Note that the host needs to be
> determined before the vif_type can be determined, so it is not possible
> to rely on the agent discovering the VIF, which can't be created until
> the vif_type is determined.
>

So what your saying is the current workflow is this: nova-compute creates a
port in quantum passing in the host-id (which is the hostname of the
compute host). Now quantum looks in the agent table in it's database to
determine the VIF type that should be used based on the agent that is
running on the nova-compute node? My question would be why the nova-compute
node doesn't already know which VIF_TYPE it should be using?


> Back when the port binding extension was originally being hashed out, I
> had suggested using an explicit bind() operation on port that took the
> host_id as a parameter and returned the vif_type as a result. But the
> current attribute-based approach was chosen instead. We could consider
> adding a bind() operation for the next neutron API revision, but I don't
> see any reason the current attribute-based binding approach cannot work
> for now.
>
> -Bob
>
> >
> > Best,
> >
> > Aaron
> >
> >
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] savanna version 0.3 - added UI mockups for EDP workflow

[Alexander Kuznetsov]

On the tab with parameters, we see case for Hadoop streaming API. Could you
please add more examples for parameters tab including cases for Hadoop jar,
Pig and Hive scripts?

Thanks,
Alexander Kuznetsov.


On Fri, Jul 12, 2013 at 7:14 PM, Chad Roberts  wrote:

> I have added some initial UI mockups for version 0.3.
> Any comments are appreciated.
>
> https://wiki.openstack.org/wiki/Savanna/UIMockups/JobCreation
>
> Thanks,
> Chad
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Program Description for OpenStack QA

[Sean Dague]

For general review (took us a little longer, we needed to elected a PTL 
first).


Official Program Name: OpenStack Quality Assurance
PTL: Sean Dague
Mission Statement: Develop, maintain, and initiate tools and plans to 
ensure the upstream stability and quality of OpenStack, and its release 
readiness at any point during the release cycle.


The OpenStack QA program starts with 2 git trees
 * tempest - https://github.com/openstack/tempest
 * grenade - https://github.com/openstack-dev/grenade

We'll get more info up on the wiki over the coming weeks to fill out 
full description.


-Sean

--
Sean Dague
http://dague.net

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Review] Use of exception for non-exceptional cases

[Brant Knudson]

Somewhat offtopic, but others might find it interesting. On another project
I used a different technique for exceptions, where the original exception
is "enhanced" with new fields as it propagates up the stack to where it's
handled. In this way all the information needed to handle the exception is
available (even if it's just to log it).

Often there's some information that would be useful for the exception
handler that isn't available at the place where the exception is raised.
The classic example is an error writing to a file where you'd like to know
the filename but all you've got is the file descriptor. An example from an
OpenStack REST API might be that you get an exception and you'd like to log
a message that includes the resource path, but the resource path isn't
known at this point.

So rather than logging the exception when it happens, enhance the
exception, re-raise it, and only once it's got all the information where
you can print out a useful log message you log it.

Here's a short example to illustrate. An exception is raised by f1(), but
at this point we don't know the status code that the server should respond
with or the request path which would be useful in a log message. These bits
of information are added as the exception propagates and then where it's
caught we've got all the information for a useful log message.

def f1():
  raise Exception('something')


def f2():
  try:
f1()
  except Exception as e:
e.status_code = 403
raise


def do_tokens():
  try:
f2()
  except Exception as e:
e.req_url = '/v3/tokens'
raise


status_code = 200
try:
  do_tokens()
except Exception as e:
  status_code = getattr(e, 'status_code', 500)
  req_url = getattr(e, 'req_url', 'unknown')

if status_code != 200:
  print 'Got %s from %s' % (status_code, req_url)
  # build an error document for the response.



On Fri, Jul 12, 2013 at 12:25 PM, Nachi Ueno  wrote:

> Hi folks
>
> > Monty
> Thank you for your adding good topic for this.
>
> I agree, hiding true cause by exception get hard to debug
>
> For, example, Sqlalchemy gives me this error for general sql errors..
> "exceptions must be old-style classes or derived from BaseException, not
> str"
>
> (It should be hidden from REST API users though)
>
> Also, I agree with you about log policies.
> sometimes 404 get warn log..
>
> IMO, The log more than warn level should help Operators.
> also If the exception is truly, the exceptional case which needs help
> of operators.
> It should be logged as error level.
>
> Thanks
> Nachi
>
>
>
> 2013/7/12 Dolph Mathews :
> >
> > On Fri, Jul 12, 2013 at 10:09 AM, Monty Taylor 
> wrote:
> >>
> >>
> >>
> >> On 07/11/2013 05:20 AM, Mark McLoughlin wrote:
> >> > On Wed, 2013-07-10 at 19:49 -0400, Monty Taylor wrote:
> >> >> I'd like top-post and hijack this thread for another exception
> related
> >> >> thing:
> >> >>
> >> >> a) Anyone writing code such as:
> >> >>
> >> >> try:
> >> >>   blah()
> >> >> except SomeException:
> >> >>   raise SomeOtherExceptionLeavingOutStackContextFromSomeException
> >> >>
> >> >> should be mocked ruthlessly.
> >> >
> >> > Ok, mock me ruthlessly then.
> >>
> >> Mock. Mock. Mock. Mock.
> >>
> >> > Part of designing any API is specifying what predictable exceptions it
> >> > will raise. For any predictable error condition, you don't want
> callers
> >> > to have to catch random exceptions from the underlying libraries you
> >> > might be calling into.
> >>
> >> Absolutely. But you don't want to go so overboard that you remove the
> >> ability for a developer to debug it. More importantly, INSIDE of server
> >> code, we're not designing fun apis for the consumption of people we
> >> don't know. There is clearly an API, but we are the consumers of our own
> >> API.
> >
> >
> > Lies! Write everything to be intuitive for new contributors or we won't
> have
> > any :(
> >
> >>
> >>
> >> > Say if I was designing an image downloading API, I'd do something like
> >> > this:
> >> >
> >> >   https://gist.github.com/markmc/5973868
> >> >
> >> > Assume there's a tonne more stuff that the API would do. You don't
> want
> >> > callers to have to catch socket.error exceptions and whatever other
> >> > exceptions might be thrown.
> >> >
> >> > That works out as:
> >> >
> >> >   Traceback (most recent call last):
> >> > File "t.py", line 20, in 
> >> >   download_image('localhost', 3366, 'foobar')
> >> > File "t.py", line 18, in download_image
> >> >   raise ImageDownloadFailure(host, port, path, e.strerror)
> >> >   __main__.ImageDownloadFailure: Failed to download foobar from
> >> > localhost:3366: Connection refused
> >> >
> >> > Which is a pretty clear exception.
> >>
> >> Right, but what if the message from the exception does not give you
> >> enough information to walk down the stack and see it...
> >>
> >> Also, I have more than once seen:
> >>
> >> class MyIOError(Exception):
> >> pass
> >>
> >> try:
> >> s = socket.create_connection((host, port))
> >> except

[openstack-dev] [Savanna-all] Cluster scaling discussion

[Matthew Farrellee]

A comment on how you go about this. I believe you've already run into 
issues w/ using the start/stop-*.sh scripts as a foundation for this 
feature.


Long term I believe that an active cluster need not mean every instance 
is up and running. The core infrastructure must be (ambari + jt + nn), 
and some % of worker instances (jt + dn). For example, if I want to make 
a 500 instance cluster, I won't need to wait for all 500 instances 
before I can reasonably start using the cluster. In fact, I may never 
have 500 instances at any given time, 98% may be acceptable operating 
procedure. The start/stop-*.sh scripts are not good for that use case 
either.


However you go about this, keep the 98% cluster use case in mind.

Best,


matt



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Tempest testing for optional middlewares

[Sean Dague]

On 07/12/2013 02:15 PM, Joe Hakim Rahme wrote:

Hello everyone,

I'm addressing this email to the dev list because I couldn't find a way
to get in touch with the testing team. Hopefully someone here will have
the answer to my question or can point me to the correct people to ask.

I am writing Tempest tests that cover the behavior of some optional
Swift middlewares (precisely account_quota and container_quota).
It occurs to me that these middlewares are optional and may not be
  present in every Swift installation. In this case, I'd like Tempest to skip
this test rather than fail it.

What's the correct way of detecting the presence of the middleware
before launching the test?


In the tempest.conf you should create a variable for "foo_available", 
defaulting to false. Then if someone wants to test it we set it to true. 
Then you can decorate your tests (or class) to skip if that variable is 
false.


This is an example of it in the code - 
https://github.com/openstack/tempest/blob/master/tempest/api/compute/servers/test_server_actions.py#L150


-Sean

--
Sean Dague
http://dague.net

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Tempest testing for optional middlewares

[Joe Hakim Rahme]

Hello everyone,

I'm addressing this email to the dev list because I couldn't find a way
to get in touch with the testing team. Hopefully someone here will have
the answer to my question or can point me to the correct people to ask.

I am writing Tempest tests that cover the behavior of some optional
Swift middlewares (precisely account_quota and container_quota). 
It occurs to me that these middlewares are optional and may not be
 present in every Swift installation. In this case, I'd like Tempest to skip 
this test rather than fail it.

What's the correct way of detecting the presence of the middleware
before launching the test?

Thank you,
Joe H. Rahme

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] mid-cycle sprint?

[Clint Byrum]

Excerpts from Devananda van der Veen's message of 2013-07-12 09:02:49 -0700:
> Neither week is possible for me. There's that thing in the desert... So
> while I'd like to attend, I don't think my absence will affect the rest of
> you being productive :)
> 

I see no reason to write off the Mojave desert in August as a potential
sprint location. What could possibly go wrong?

I am unavailable 8/26 - 9/5.. so the week before that would be better
for me.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Review] Use of exception for non-exceptional cases

[Nachi Ueno]

Hi folks

> Monty
Thank you for your adding good topic for this.

I agree, hiding true cause by exception get hard to debug

For, example, Sqlalchemy gives me this error for general sql errors..
"exceptions must be old-style classes or derived from BaseException, not str"

(It should be hidden from REST API users though)

Also, I agree with you about log policies.
sometimes 404 get warn log..

IMO, The log more than warn level should help Operators.
also If the exception is truly, the exceptional case which needs help
of operators.
It should be logged as error level.

Thanks
Nachi



2013/7/12 Dolph Mathews :
>
> On Fri, Jul 12, 2013 at 10:09 AM, Monty Taylor  wrote:
>>
>>
>>
>> On 07/11/2013 05:20 AM, Mark McLoughlin wrote:
>> > On Wed, 2013-07-10 at 19:49 -0400, Monty Taylor wrote:
>> >> I'd like top-post and hijack this thread for another exception related
>> >> thing:
>> >>
>> >> a) Anyone writing code such as:
>> >>
>> >> try:
>> >>   blah()
>> >> except SomeException:
>> >>   raise SomeOtherExceptionLeavingOutStackContextFromSomeException
>> >>
>> >> should be mocked ruthlessly.
>> >
>> > Ok, mock me ruthlessly then.
>>
>> Mock. Mock. Mock. Mock.
>>
>> > Part of designing any API is specifying what predictable exceptions it
>> > will raise. For any predictable error condition, you don't want callers
>> > to have to catch random exceptions from the underlying libraries you
>> > might be calling into.
>>
>> Absolutely. But you don't want to go so overboard that you remove the
>> ability for a developer to debug it. More importantly, INSIDE of server
>> code, we're not designing fun apis for the consumption of people we
>> don't know. There is clearly an API, but we are the consumers of our own
>> API.
>
>
> Lies! Write everything to be intuitive for new contributors or we won't have
> any :(
>
>>
>>
>> > Say if I was designing an image downloading API, I'd do something like
>> > this:
>> >
>> >   https://gist.github.com/markmc/5973868
>> >
>> > Assume there's a tonne more stuff that the API would do. You don't want
>> > callers to have to catch socket.error exceptions and whatever other
>> > exceptions might be thrown.
>> >
>> > That works out as:
>> >
>> >   Traceback (most recent call last):
>> > File "t.py", line 20, in 
>> >   download_image('localhost', 3366, 'foobar')
>> > File "t.py", line 18, in download_image
>> >   raise ImageDownloadFailure(host, port, path, e.strerror)
>> >   __main__.ImageDownloadFailure: Failed to download foobar from
>> > localhost:3366: Connection refused
>> >
>> > Which is a pretty clear exception.
>>
>> Right, but what if the message from the exception does not give you
>> enough information to walk down the stack and see it...
>>
>> Also, I have more than once seen:
>>
>> class MyIOError(Exception):
>> pass
>>
>> try:
>> s = socket.create_connection((host, port))
>> except socket.error:
>> raise MyIOError("something went wrong!")
>>
>> Which is an extreme example of where my rage comes from, but not a made
>> up example. I have, actually, seen code exactly like that - in Nova.
>>
>> BTW - I'd have download_image return None if it can't download the
>> image, and I'd have it either deal with the socket.error or not locally
>> at the source. But now we're nitpicking.
>>
>> > But I think what you're saying is missing is the stack trace from the
>> > underlying exception.
>>
>> YES - and I'll let David's response respond to the details of the rest...
>>
>> But essentially, what I was really mocking earlier is throwing a new
>> exception in such a way that you lose the exception propagation path. So
>> if you throw an exception inside of an except block, you should be sure
>> that you're passing on all of the info, because if a developer gets an
>> exception, it's quite likely that they want to know how to fix it. :)
>>
>> > As I understood it, Python doesn't have a way of chaining exceptions
>> > like this but e.g. Java does. A little bit more poking right now shows
>> > up this:
>> >
>> >   http://www.python.org/dev/peps/pep-3134/
>> >
>> > i.e. we can't do the right thing until Python 3, where we'd do:
>> >
>> >  def download_image(host, port, path):
>> >  try:
>> >  s = socket.create_connection((host, port))
>> >  except socket.error as e:
>> >  raise ImageDownloadFailure(host, port, path, e.strerror) from e
>>
>> This will certainly be cleaner to write and read.
>>
>> > I haven't read the PEP in detail yet, though.
>> >
>> > Cheers,
>> > Mark.
>> >
>> >
>> > ___
>> > OpenStack-dev mailing list
>> > OpenStack-dev@lists.openstack.org
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> --
>
> -Dolph
>
> ___
> Ope

Re: [openstack-dev] [TripleO] mid-cycle sprint?

[Devananda van der Veen]

Neither week is possible for me. There's that thing in the desert... So
while I'd like to attend, I don't think my absence will affect the rest of
you being productive :)

Cheers,
Devananda



On Wed, Jul 10, 2013 at 8:54 PM, Robert Collins
wrote:

> Clint suggested we do a mid-cycle sprint at the weekly meeting a
> fortnight ago, but ETIME and stuff - so I'm following up.
>
> HP would be delighted to host a get-together of TripleO contributors
> [or 'I will be contributing soon, honest'] folk.
>
> We're proposing a late August / early Sept time - a couple weeks
> before H3, so we can be dealing with features that have landed //
> ensuring necessary features *do* land.
>
> That would give a start date of the 19th or 26th August. Probable
> venue of either Sunnyvale, CA or Seattle.
>
> I need a rough count of numbers to kick off the approval and final
> venue stuff w/in HP. I've cc'd some fairly obvious folk that should
> come :)
>
> So - who is interested and would come, and what constraints do you have?
>
> -Rob
>
> --
> Robert Collins 
> Distinguished Technologist
> HP Cloud Services
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] mid-cycle sprint?

[Chris Jones]

Hey

I'm away from 5th-15th August and if possible I would prefer to be at home
for 11th September. I'll start putting out feelers for family to come and
stay to help with childcare while I'm out :)
No preferences on the location.

Cheers,

Chris


On 12 July 2013 17:02, Devananda van der Veen wrote:

> Neither week is possible for me. There's that thing in the desert... So
> while I'd like to attend, I don't think my absence will affect the rest of
> you being productive :)
>
> Cheers,
> Devananda
>
>
>
> On Wed, Jul 10, 2013 at 8:54 PM, Robert Collins  > wrote:
>
>> Clint suggested we do a mid-cycle sprint at the weekly meeting a
>> fortnight ago, but ETIME and stuff - so I'm following up.
>>
>> HP would be delighted to host a get-together of TripleO contributors
>> [or 'I will be contributing soon, honest'] folk.
>>
>> We're proposing a late August / early Sept time - a couple weeks
>> before H3, so we can be dealing with features that have landed //
>> ensuring necessary features *do* land.
>>
>> That would give a start date of the 19th or 26th August. Probable
>> venue of either Sunnyvale, CA or Seattle.
>>
>> I need a rough count of numbers to kick off the approval and final
>> venue stuff w/in HP. I've cc'd some fairly obvious folk that should
>> come :)
>>
>> So - who is interested and would come, and what constraints do you have?
>>
>> -Rob
>>
>> --
>> Robert Collins 
>> Distinguished Technologist
>> HP Cloud Services
>>
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Review] Use of exception for non-exceptional cases

[Dolph Mathews]

On Fri, Jul 12, 2013 at 10:09 AM, Monty Taylor  wrote:

>
>
> On 07/11/2013 05:20 AM, Mark McLoughlin wrote:
> > On Wed, 2013-07-10 at 19:49 -0400, Monty Taylor wrote:
> >> I'd like top-post and hijack this thread for another exception related
> >> thing:
> >>
> >> a) Anyone writing code such as:
> >>
> >> try:
> >>   blah()
> >> except SomeException:
> >>   raise SomeOtherExceptionLeavingOutStackContextFromSomeException
> >>
> >> should be mocked ruthlessly.
> >
> > Ok, mock me ruthlessly then.
>
> Mock. Mock. Mock. Mock.
>
> > Part of designing any API is specifying what predictable exceptions it
> > will raise. For any predictable error condition, you don't want callers
> > to have to catch random exceptions from the underlying libraries you
> > might be calling into.
>
> Absolutely. But you don't want to go so overboard that you remove the
> ability for a developer to debug it. More importantly, INSIDE of server
> code, we're not designing fun apis for the consumption of people we
> don't know. There is clearly an API, but we are the consumers of our own
> API.
>

Lies! Write everything to be intuitive for new contributors or we won't
have any :(


>
> > Say if I was designing an image downloading API, I'd do something like
> > this:
> >
> >   https://gist.github.com/markmc/5973868
> >
> > Assume there's a tonne more stuff that the API would do. You don't want
> > callers to have to catch socket.error exceptions and whatever other
> > exceptions might be thrown.
> >
> > That works out as:
> >
> >   Traceback (most recent call last):
> > File "t.py", line 20, in 
> >   download_image('localhost', 3366, 'foobar')
> > File "t.py", line 18, in download_image
> >   raise ImageDownloadFailure(host, port, path, e.strerror)
> >   __main__.ImageDownloadFailure: Failed to download foobar from
> localhost:3366: Connection refused
> >
> > Which is a pretty clear exception.
>
> Right, but what if the message from the exception does not give you
> enough information to walk down the stack and see it...
>
> Also, I have more than once seen:
>
> class MyIOError(Exception):
> pass
>
> try:
> s = socket.create_connection((host, port))
> except socket.error:
> raise MyIOError("something went wrong!")
>
> Which is an extreme example of where my rage comes from, but not a made
> up example. I have, actually, seen code exactly like that - in Nova.
>
> BTW - I'd have download_image return None if it can't download the
> image, and I'd have it either deal with the socket.error or not locally
> at the source. But now we're nitpicking.
>
> > But I think what you're saying is missing is the stack trace from the
> > underlying exception.
>
> YES - and I'll let David's response respond to the details of the rest...
>
> But essentially, what I was really mocking earlier is throwing a new
> exception in such a way that you lose the exception propagation path. So
> if you throw an exception inside of an except block, you should be sure
> that you're passing on all of the info, because if a developer gets an
> exception, it's quite likely that they want to know how to fix it. :)
>
> > As I understood it, Python doesn't have a way of chaining exceptions
> > like this but e.g. Java does. A little bit more poking right now shows
> > up this:
> >
> >   http://www.python.org/dev/peps/pep-3134/
> >
> > i.e. we can't do the right thing until Python 3, where we'd do:
> >
> >  def download_image(host, port, path):
> >  try:
> >  s = socket.create_connection((host, port))
> >  except socket.error as e:
> >  raise ImageDownloadFailure(host, port, path, e.strerror) from e
>
> This will certainly be cleaner to write and read.
>
> > I haven't read the PEP in detail yet, though.
> >
> > Cheers,
> > Mark.
> >
> >
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 

-Dolph
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] The danger of capping python-*clients in core projects, and forbidding it in the future

[Gareth]

There're still keystone client conflict issues:
https://review.openstack.org/#/c/36684/
It seems uncapping keystone client(remove upper bound) in some else project
first is needed?


On Fri, Jul 12, 2013 at 7:25 PM, Sean Dague  wrote:

> We are working towards uncapping all the clients, with the exception of
> neutron client, because they need to make some incompatible changes on
> their next major release.
>
>
> On 07/12/2013 12:12 AM, Gareth wrote:
>
>> so, what's the final conclusion about this issue?
>>
>>
>> On Fri, Jul 12, 2013 at 12:11 PM, Gareth > > wrote:
>>
>> Thanks, Monty
>>
>> but in my review 
>> https://review.openstack.org/#**/c/36684/,
>>  Doug said
>> we will go without upper bound with those python-*clients
>> and in this one 
>> https://review.openstack.org/#**/c/36753/,
>> keystoneclient still keep '<0.4' and requirements test doesn't fail
>> in keystoneclient
>> (https://jenkins.openstack.**org/job/gate-cinder-**
>> requirements/96/console
>> it failed on glanceclient)
>>
>>
>>
>>
>> On Fri, Jul 12, 2013 at 11:54 AM, Monty Taylor > > wrote:
>>
>>
>>
>> On 07/11/2013 11:38 PM, Gareth wrote:
>>  > I heard there's a talk about this issue in #openstack-infra
>> last night
>>  > (china standard time), what's the conclusion of that?
>>  >
>>  > BTW, how to find meeting log of #openstack-infra? I didn't
>> find it
>>  > in 
>> http://eavesdrop.openstack.**org/
>>
>> We don't log it currently. There is a wider conversation going
>> on about
>> which things we should log and which things we should not log
>> ... but
>> for the time being I've submitted this:
>>
>> 
>> https://review.openstack.org/**36773
>>
>> to add -infra. I think we talk about enough things that have
>> ramifications on everyone in there that we should really capture
>> it.
>>  > On Thu, Jul 11, 2013 at 11:35 PM, Dirk Müller > 
>>  > >> wrote:
>>  >
>>  > >> See for example
>> 
>> https://bugs.launchpad.net/**horizon/+bug/1196823
>>  > > This is arguably a deficiency of mox, which
>> (apparently?) doesn't
>>  > let us mock properties automatically.
>>  >
>>  > I agree, but it is just one example. other test-only
>> issues can
>>  > happen as well.
>>  >
>>  > Similar problem: the *client packages are not
>> self-contained, they
>>  > have pretty strict dependencies on other packages. One
>> case I already
>>  > run into was a dependency on python-requests: newer
>> python-*client
>>  > packages (rightfully) require requests >= 1.x. running
>> those on a
>>  > system that has OpenStack services from Grizzly or Folsom
>> installed
>>  > cause a conflict: there are one or two that require
>> requests to be <
>>  > 1.0.
>>  >
>>  > When you run gating on this scenario, I think the same
>> flipping would
>>  > happen on e.g. requests as well, due to *client or the
>> module being
>>  > installed in varying order.
>>  >
>>  > Greetings,
>>  > Dirk
>>  >
>>  > __**_
>>  > OpenStack-dev mailing list
>>  > 
>> OpenStack-dev@lists.openstack.**org
>> 
>> > >
>>  > 
>> >
>> 
>> > >>
>>  > http://lists.openstack.org/**cgi-bin/mailman/listinfo/**
>> openstack-dev
>>  >
>>  >
>>  >
>> >
>> > --
>> > Gareth
>> >
>> > /Cloud Computing, OpenStack, Fitness, Basketball/
>> > /OpenStack contributor/
>> > /Company: UnitedStack /
>> > /My promise: if you find any spelling or grammar mistakes in my
>> email
>> > from Mar 1 2013, notify me /
>> > /and I'll donate $1 or ￥1 to an open organization you specify./
>>  >
>>  >
>>  > __**_
>>  > OpenStack-dev mailing list
>>  > 
>> OpenStack-dev@lists.openstack.**org
>> 

Re: [openstack-dev] [Review] Use of exception for non-exceptional cases

[Thomas Hervé]

On Fri, Jul 12, 2013 at 5:33 PM, Monty Taylor  wrote:

>
>
> On 07/11/2013 05:43 AM, Thomas Hervé wrote:
> >
> >
> > On Thu, Jul 11, 2013 at 1:49 AM, Monty Taylor  > > wrote:
> >
> > I'd like top-post and hijack this thread for another exception
> related
> > thing:
> >
> > a) Anyone writing code such as:
> >
> > try:
> >   blah()
> > except SomeException:
> >   raise SomeOtherExceptionLeavingOutStackContextFromSomeException
> >
> > should be mocked ruthlessly.
> >
> >
> > i don't think mocking is a good way to convey your point. Losing
> > tracebacks is not great, but having an API raising random exceptions is
> > probably worse. Can you develop?
>
> I have learned that I may have mis-read your last three words due to
> translation problems. You were not asking if I had the ability to write
> code, rather you were asking if I could elaborate.
>

Ah thanks, and sorry for the frenchism.

I think I understand your point now, which is not so much about tracebacks
but about context in the wide sense, ie enough information to understand
what's going on. I've found that we're not necessarily doing a great job at
testing the error messages: it's nice to know that FooError has been
raised, but if the message is 'Error' it leads to what you're describing,
where you need to look at the code and potentially change it to debug it. I
believe more consistent tests may help that a bit.

Cheers,

-- 
Thomas
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] SQLAlchemy-migrate needs a new maintainer

[David Ripton]

On 07/12/2013 07:31 AM, Sean Dague wrote:

On 07/12/2013 04:29 AM, Thierry Carrez wrote:

Monty Taylor wrote:

This brings us to the most important question:

Who wants to be on the core team?


That's the important question indeed. Accepting it (permanently or
temporarily) under stackforge is an easy decision. But it's useless
unless we have a set of people sufficiently interested in it not
bitrotting to volunteer to maintain it...


I'd recommend the nova-db subteam folks, like: jog0, dripton, boris-42
as good people to be +2 on this.


I'm happy to help, as long as there are at least 2 others.

--
David Ripton   Red Hat   drip...@redhat.com

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Review] Use of exception for non-exceptional cases

[Monty Taylor]

On 07/11/2013 06:22 AM, Sean Dague wrote:
> On 07/11/2013 05:43 AM, Thomas Hervé wrote:
>>
>>
>> On Thu, Jul 11, 2013 at 1:49 AM, Monty Taylor > > wrote:
>>
>> I'd like top-post and hijack this thread for another exception
>> related
>> thing:
>>
>> a) Anyone writing code such as:
>>
>> try:
>>blah()
>> except SomeException:
>>raise SomeOtherExceptionLeavingOutStackContextFromSomeException
>>
>> should be mocked ruthlessly.
>>
>>
>> i don't think mocking is a good way to convey your point. Losing
>> tracebacks is not great, but having an API raising random exceptions is
>> probably worse. Can you develop?
> 
> We have defined APIs. We have server projects that call other server
> projects through python-fooclients. We have python-fooclients that
> generate exceptions and stack traces on any non 20X return.
> 
> So we have a general exception translation issue. Because unless we want
> nova-api returns to be completely fluid based on what
> keystone/neutron/glance/cinder clients decided their exceptions are this
> week, you have to translate. And the stack trace from a 404 isn't
> something that we really care about from the client, we care that it
> happened, because that affects nova logic, but it's an expected exception.

It's an expected exception until it's not.

> And where would we even put the stack trace? take it to the user on
> their API call?

Oh gosh no! I'm CERTAINLY not suggesting that we passthrough translated
exceptions to our REST API consumers. I'm talking about actual
exceptions in server code which generate actual error logging because
they are actual errors but where you cannot find what the error is
because of masking.

> fill up the logs with stack traces for normal events
> (thus hiding real errors)? (we actually have some blueprints openned now
> to remove these because a *normal* passing tempest run generates 30+
> stack traces in nova logs, which aren't actually internal errors.)

Yes. This is actually much worse, and I fully support you in all of
these efforts!

> The bulk of these cases I've seen in Nova are exactly of that nature. We
> actually have a pretty standard pattern of 3 levels of exception
> translations for user API calls.
> 
> Underlying client exception (be it DB / python-fooclient) -> Nova
> internal exception -> Webobj exception to create our 40x / 50x returns.

I believe striping the underlying exception in the webobj translation is
perfectly reasonable. The alternative would be madness.

I honestly can't find a great example of this in our code this second (I
think concrete examples are great) but I know that I've hit it enough to
lodge an annoyance flag - so let me pull an example from the insides of
our friend d2to1:

try:
attrs = cfg_to_args(path)
except:
e = sys.exc_info()[1]
raise DistutilsSetupError(
'Error parsing %s: %s: %s' % (path, e.__class__.__name__,
  e.args[0]))

This is an attempt to give the user a meaningful error in the case that
they have a problem. So - first thing is, there's a bug, becuase
e.args[0] is the error code, so you get:

error in setup command: Error parsing setup.cfg: IOError: 2

Which is TOTALLY meaningless.

If you change the string line to:

+'Error parsing %s: %s: %s' % (path, e.__class__.__name__, e))

You at least get:

error in setup command: Error parsing setup.cfg: IOError: No such file
or directory: 'README'

Which is much more helpful. However, that's a simple error, if you have
a more complex error, such as your pbr hook threw a traceback while
loading, it's a bit bonghits and to track it down you may have to expend
some effort. BUT - for the general case, you have a typo in your
setup.cfg file, IOError: No such file or directory: 'README' is probably
sufficient and you don't probably need the traceback.

That said - when I'm having problems using this code, I tend to just
edit that file, remove that exception wrapper altogether, and let the
exceptions fly- and can usually find the problem in about a second.
Maybe I should add a debug flag which skips the wrapping...

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Review] Use of exception for non-exceptional cases

[Monty Taylor]

On 07/11/2013 05:43 AM, Thomas Hervé wrote:
> 
> 
> On Thu, Jul 11, 2013 at 1:49 AM, Monty Taylor  > wrote:
> 
> I'd like top-post and hijack this thread for another exception related
> thing:
> 
> a) Anyone writing code such as:
> 
> try:
>   blah()
> except SomeException:
>   raise SomeOtherExceptionLeavingOutStackContextFromSomeException
> 
> should be mocked ruthlessly.
> 
> 
> i don't think mocking is a good way to convey your point. Losing
> tracebacks is not great, but having an API raising random exceptions is
> probably worse. Can you develop?

I have learned that I may have mis-read your last three words due to
translation problems. You were not asking if I had the ability to write
code, rather you were asking if I could elaborate.

I will consider that I have learned something new today!

> Regards,
> 
> -- 
> Thomas
>  
> 
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Review] Use of exception for non-exceptional cases

[Monty Taylor]

On 07/11/2013 05:43 AM, Thomas Hervé wrote:
> 
> 
> On Thu, Jul 11, 2013 at 1:49 AM, Monty Taylor  > wrote:
> 
> I'd like top-post and hijack this thread for another exception related
> thing:
> 
> a) Anyone writing code such as:
> 
> try:
>   blah()
> except SomeException:
>   raise SomeOtherExceptionLeavingOutStackContextFromSomeException
> 
> should be mocked ruthlessly.
> 
> 
> i don't think mocking is a good way to convey your point. Losing
> tracebacks is not great, but having an API raising random exceptions is
> probably worse. Can you develop?

Mocking is bad at many things, and email is a bad way to express
generalizations through exaggeration. I would almost certainly not
actually mock anyone.

I doubt very seriously that any of us here working on OpenStack lack the
ability to develop.

In response to your question though- I find that when the process of
debugging an error involves editing an installed library to remove an
improperly done exception translation so that I can actually see the
traceback and find the error, then someone has made the wrong choice in
their implementation of the library.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] savanna version 0.3 - added UI mockups for EDP workflow

[Chad Roberts]

I have added some initial UI mockups for version 0.3. 
Any comments are appreciated.

https://wiki.openstack.org/wiki/Savanna/UIMockups/JobCreation

Thanks,
Chad

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Review] Use of exception for non-exceptional cases

[Monty Taylor]

On 07/11/2013 05:20 AM, Mark McLoughlin wrote:
> On Wed, 2013-07-10 at 19:49 -0400, Monty Taylor wrote:
>> I'd like top-post and hijack this thread for another exception related
>> thing:
>>
>> a) Anyone writing code such as:
>>
>> try:
>>   blah()
>> except SomeException:
>>   raise SomeOtherExceptionLeavingOutStackContextFromSomeException
>>
>> should be mocked ruthlessly.
> 
> Ok, mock me ruthlessly then.

Mock. Mock. Mock. Mock.

> Part of designing any API is specifying what predictable exceptions it
> will raise. For any predictable error condition, you don't want callers
> to have to catch random exceptions from the underlying libraries you
> might be calling into.

Absolutely. But you don't want to go so overboard that you remove the
ability for a developer to debug it. More importantly, INSIDE of server
code, we're not designing fun apis for the consumption of people we
don't know. There is clearly an API, but we are the consumers of our own
API.

> Say if I was designing an image downloading API, I'd do something like
> this:
> 
>   https://gist.github.com/markmc/5973868
> 
> Assume there's a tonne more stuff that the API would do. You don't want
> callers to have to catch socket.error exceptions and whatever other
> exceptions might be thrown.
>
> That works out as:
> 
>   Traceback (most recent call last):
> File "t.py", line 20, in 
>   download_image('localhost', 3366, 'foobar')
> File "t.py", line 18, in download_image
>   raise ImageDownloadFailure(host, port, path, e.strerror)
>   __main__.ImageDownloadFailure: Failed to download foobar from 
> localhost:3366: Connection refused
> 
> Which is a pretty clear exception.

Right, but what if the message from the exception does not give you
enough information to walk down the stack and see it...

Also, I have more than once seen:

class MyIOError(Exception):
pass

try:
s = socket.create_connection((host, port))
except socket.error:
raise MyIOError("something went wrong!")

Which is an extreme example of where my rage comes from, but not a made
up example. I have, actually, seen code exactly like that - in Nova.

BTW - I'd have download_image return None if it can't download the
image, and I'd have it either deal with the socket.error or not locally
at the source. But now we're nitpicking.

> But I think what you're saying is missing is the stack trace from the
> underlying exception.

YES - and I'll let David's response respond to the details of the rest...

But essentially, what I was really mocking earlier is throwing a new
exception in such a way that you lose the exception propagation path. So
if you throw an exception inside of an except block, you should be sure
that you're passing on all of the info, because if a developer gets an
exception, it's quite likely that they want to know how to fix it. :)

> As I understood it, Python doesn't have a way of chaining exceptions
> like this but e.g. Java does. A little bit more poking right now shows
> up this:
> 
>   http://www.python.org/dev/peps/pep-3134/
> 
> i.e. we can't do the right thing until Python 3, where we'd do:
> 
>  def download_image(host, port, path):
>  try:
>  s = socket.create_connection((host, port))
>  except socket.error as e:
>  raise ImageDownloadFailure(host, port, path, e.strerror) from e

This will certainly be cleaner to write and read.

> I haven't read the PEP in detail yet, though.
> 
> Cheers,
> Mark.
> 
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] SQLAlchemy-migrate needs a new maintainer

[Monty Taylor]

On 07/12/2013 07:37 AM, Boris Pavlovic wrote:
> Hi Sean,
> 
> I agree to help with sqlalchemy-migrate until we remove it. 
> But probably there should be one more person mikal

Done.

https://github.com/stackforge/sqlalchemy-migrate

This is up and active, and it will run unittests. Right now it only runs
them on sqlite, but making them run against mysql should be easy (we
just need to put the user/pass we use for nova unittests against mysql
into the test_db.cfg file - same with postgres)

The project is set up to publish docs to RTFD (although there is a bug
in that setup right now) and to publish packages to PyPI on tags just
like our other projects.

Have fun everyone.

> Best regards,
> Boris Pavlovic
> 
> 
> On Fri, Jul 12, 2013 at 3:31 PM, Sean Dague  > wrote:
> 
> On 07/12/2013 04:29 AM, Thierry Carrez wrote:
> 
> Monty Taylor wrote:
> 
> This brings us to the most important question:
> 
> Who wants to be on the core team?
> 
> 
> That's the important question indeed. Accepting it (permanently or
> temporarily) under stackforge is an easy decision. But it's useless
> unless we have a set of people sufficiently interested in it not
> bitrotting to volunteer to maintain it...
> 
> 
> I'd recommend the nova-db subteam folks, like: jog0, dripton,
> boris-42 as good people to be +2 on this.
> 
> -Sean
> 
> -- 
> Sean Dague
> http://dague.net
> 
> 
> _
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.__org
> 
> http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev 
> 
> 
> 
> 
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack] Improve inject network configuration

[Russell Bryant]

On 07/12/2013 04:43 AM, Thierry Carrez wrote:
> Brian Lamar wrote:
>>> Honestly, I think network injection is evil and I'd rather remove it
>>> completely. I'm certainly not too interested in trying to add more
>>> features to it.
>>
>> Can you elaborate on this a little more? Do you not like file injection
>> or dynamic network allocation?
> 
> It's an old discussion... in summary:
> 
> Nova inserting stuff pre-booting into the VM it runs = evil, brittle and
> the source of countless past vulnerabilities
> 
> VMs auto-configuring at boot-time using cloud-init based on data
> provided through generic input channels (config drive, metadata
> servers...) = good
> 
> So this is not about disliking the ability to insert files or specify
> network parameters for a VM, it's about who is in charge of actually
> creating files and network configurations. Nova shouldn't have to learn
> about the specificities of the VM image it runs, nor should it have to
> mount VM filesystems before booting them. The VM itself should take care
> of the translation based on standardized input (if it wants to).

Thank you for the nice summary.  :-)

>> Can you provide alternative strategies that could be applied to solve
>> the issue of dynamically brining up interfaces or do you think this is
>> out of the project scope (controlling the internals of VMs)?
> 
> Config-drive should pass that config to the VM, and cloud-init on the VM
> should pick it up.

Or you can use the metadata server instead of config-drive.

-- 
Russell Bryant

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Revert Pass instance host-id to Quantum using port bindings extension.

[Robert Kukura]

On 07/11/2013 04:30 PM, Aaron Rosen wrote:
> Hi, 
> 
> I think we should revert this patch that was added here
> (https://review.openstack.org/#/c/29767/). What this patch does is when
> nova-compute calls into quantum to create the port it passes in the
> hostname on which the instance was booted on. The idea of the patch was
> that providing this information would "allow hardware device vendors
> management stations to allow them to segment the network in a more
> precise manager (for example automatically trunk the vlan on the
> physical switch port connected to the compute node on which the vm
> instance was started)."
> 
> In my opinion I don't think this is the right approach. There are
> several other ways to get this information of where a specific port
> lives. For example, in the OVS plugin case the agent running on the
> nova-compute node can update the port in quantum to provide this
> information. Alternatively, quantum could query nova using the
> port.device_id to determine which server the instance is on. 
> 
> My motivation for removing this code is I now have the free cycles to
> work on
> https://blueprints.launchpad.net/nova/+spec/nova-api-quantum-create-port
>  discussed here
> (http://lists.openstack.org/pipermail/openstack-dev/2013-May/009088.html)  .
> This was about moving the quantum port creation from the nova-compute
> host to nova-api if a network-uuid is passed in. This will allow us to
> remove all the quantum logic from the nova-compute nodes and
> simplify orchestration. 
> 
> Thoughts? 

Aaron,

The ml2-portbinding BP I am currently working on depends on nova setting
the binding:host_id attribute on a port before accessing
binding:vif_type. The ml2 plugin's MechanismDrivers will use the
binding:host_id with the agents_db info to see what (if any) L2 agent is
running on that host, or what other networking mechanisms might provide
connectivity for that host. Based on this, the port's binding:vif_type
will be set to the appropriate type for that agent/mechanism.

When an L2 agent is involved, the associated ml2 MechanismDriver will
use the agent's interface or bridge mapping info to determine whether
the agent on that host can connect to any of the port's network's
segments, and select the specific segment (network_type,
physical_network, segmentation_id) to be used. If there is no
connectivity possible on the host (due to either no L2 agent or other
applicable mechanism, or no mapping for any of the network's segment's
physical_networks), the ml2 plugin will set the binding:vif_type
attribute to BINDING_FAILED. Nova will then be able to gracefully put
the instance into an error state rather than have the instance boot
without the required connectivity.

I don't see any problem with nova creating the port before scheduling it
to a specific host, but the binding:host_id needs to be set before the
binding:vif_type attribute is accessed. Note that the host needs to be
determined before the vif_type can be determined, so it is not possible
to rely on the agent discovering the VIF, which can't be created until
the vif_type is determined.

Back when the port binding extension was originally being hashed out, I
had suggested using an explicit bind() operation on port that took the
host_id as a parameter and returned the vif_type as a result. But the
current attribute-based approach was chosen instead. We could consider
adding a bind() operation for the next neutron API revision, but I don't
see any reason the current attribute-based binding approach cannot work
for now.

-Bob

> 
> Best, 
> 
> Aaron
> 
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack] Improve inject network configuration

[Russell Bryant]

On 07/12/2013 01:10 AM, Brian Lamar wrote:
> 
> 
> Russell Bryant wrote:
>> On 07/11/2013 08:53 PM, Jae Sang Lee wrote:
>>> Hi, stackers.
>>>
>>> When creating vm using multi nics, User should power up the second
>>> interface on the instance manually for use second IP.
>>> http://docs.openstack.org/trunk/openstack-compute/admin/content/using-multi-nics.html
>>>
>>>
>>>
>>> I intend to fix interfaces.template file, so It can be possible power up
>>> other interface during booting time automatically.
>>>
>>> I registered blueprint for this a month
>>> ago.(https://blueprints.launchpad.net/nova/+spec/better-network-injection)
>>>
>>>
>>> But not yet approved.
>>>
>>> If you have permission to approve who read this mail, please approve my
>>> blueprint.
>>
>> Honestly, I think network injection is evil and I'd rather remove it
>> completely. I'm certainly not too interested in trying to add more
>> features to it.
>>
> 
> Can you elaborate on this a little more? Do you not like file injection
> or dynamic network allocation?

File injection in general.

> Can you provide alternative strategies that could be applied to solve
> the issue of dynamically brining up interfaces or do you think this is
> out of the project scope (controlling the internals of VMs)?


-- 
Russell Bryant

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] SQLAlchemy-migrate needs a new maintainer

[Boris Pavlovic]

Hi Sean,

I agree to help with sqlalchemy-migrate until we remove it.
But probably there should be one more person mikal

Best regards,
Boris Pavlovic


On Fri, Jul 12, 2013 at 3:31 PM, Sean Dague  wrote:

> On 07/12/2013 04:29 AM, Thierry Carrez wrote:
>
>> Monty Taylor wrote:
>>
>>> This brings us to the most important question:
>>>
>>> Who wants to be on the core team?
>>>
>>
>> That's the important question indeed. Accepting it (permanently or
>> temporarily) under stackforge is an easy decision. But it's useless
>> unless we have a set of people sufficiently interested in it not
>> bitrotting to volunteer to maintain it...
>>
>
> I'd recommend the nova-db subteam folks, like: jog0, dripton, boris-42 as
> good people to be +2 on this.
>
> -Sean
>
> --
> Sean Dague
> http://dague.net
>
>
> __**_
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.**org 
> http://lists.openstack.org/**cgi-bin/mailman/listinfo/**openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] SQLAlchemy-migrate needs a new maintainer

[Sean Dague]

On 07/12/2013 04:29 AM, Thierry Carrez wrote:

Monty Taylor wrote:

This brings us to the most important question:

Who wants to be on the core team?


That's the important question indeed. Accepting it (permanently or
temporarily) under stackforge is an easy decision. But it's useless
unless we have a set of people sufficiently interested in it not
bitrotting to volunteer to maintain it...


I'd recommend the nova-db subteam folks, like: jog0, dripton, boris-42 
as good people to be +2 on this.


-Sean

--
Sean Dague
http://dague.net

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] The danger of capping python-*clients in core projects, and forbidding it in the future

[Sean Dague]

We are working towards uncapping all the clients, with the exception of 
neutron client, because they need to make some incompatible changes on 
their next major release.


On 07/12/2013 12:12 AM, Gareth wrote:

so, what's the final conclusion about this issue?


On Fri, Jul 12, 2013 at 12:11 PM, Gareth mailto:academicgar...@gmail.com>> wrote:

Thanks, Monty

but in my review https://review.openstack.org/#/c/36684/ , Doug said
we will go without upper bound with those python-*clients
and in this one https://review.openstack.org/#/c/36753/ ,
keystoneclient still keep '<0.4' and requirements test doesn't fail
in keystoneclient
(https://jenkins.openstack.org/job/gate-cinder-requirements/96/console
it failed on glanceclient)




On Fri, Jul 12, 2013 at 11:54 AM, Monty Taylor mailto:mord...@inaugust.com>> wrote:



On 07/11/2013 11:38 PM, Gareth wrote:
 > I heard there's a talk about this issue in #openstack-infra
last night
 > (china standard time), what's the conclusion of that?
 >
 > BTW, how to find meeting log of #openstack-infra? I didn't
find it
 > in http://eavesdrop.openstack.org/

We don't log it currently. There is a wider conversation going
on about
which things we should log and which things we should not log
... but
for the time being I've submitted this:

https://review.openstack.org/36773

to add -infra. I think we talk about enough things that have
ramifications on everyone in there that we should really capture it.
 > On Thu, Jul 11, 2013 at 11:35 PM, Dirk Müller mailto:d...@dmllr.de>
 > >> wrote:
 >
 > >> See for example
https://bugs.launchpad.net/horizon/+bug/1196823
 > > This is arguably a deficiency of mox, which
(apparently?) doesn't
 > let us mock properties automatically.
 >
 > I agree, but it is just one example. other test-only
issues can
 > happen as well.
 >
 > Similar problem: the *client packages are not
self-contained, they
 > have pretty strict dependencies on other packages. One
case I already
 > run into was a dependency on python-requests: newer
python-*client
 > packages (rightfully) require requests >= 1.x. running
those on a
 > system that has OpenStack services from Grizzly or Folsom
installed
 > cause a conflict: there are one or two that require
requests to be <
 > 1.0.
 >
 > When you run gating on this scenario, I think the same
flipping would
 > happen on e.g. requests as well, due to *client or the
module being
 > installed in varying order.
 >
 > Greetings,
 > Dirk
 >
 > ___
 > OpenStack-dev mailing list
 > OpenStack-dev@lists.openstack.org

 > >
 > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 >
 >
 >
>
> --
> Gareth
>
> /Cloud Computing, OpenStack, Fitness, Basketball/
> /OpenStack contributor/
> /Company: UnitedStack /
> /My promise: if you find any spelling or grammar mistakes in my email
> from Mar 1 2013, notify me /
> /and I'll donate $1 or ￥1 to an open organization you specify./
 >
 >
 > ___
 > OpenStack-dev mailing list
 > OpenStack-dev@lists.openstack.org

 > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 >

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




--
Gareth

/Cloud Computing, OpenStack, Fitness, Basketball/
/OpenStack contributor/
/Company: UnitedStack /
/My promise: if you find any spelling or grammar mistakes in my
email from Mar 1 2013, notify me /
/and I'll donate $1 or ￥1 to an open organization you specify./




--
Gareth

/Cloud Computing, OpenStack, Fitness, Basketball/
/OpenStack contributor/
/Company: UnitedStack /
/My promise: if you find any spelling or grammar mistakes in my email
from Mar 1 2013, notify me /
/and I'll donate $1 or ￥1 to an open orga

Re: [openstack-dev] [TripleO] mid-cycle sprint?

[Ghe Rivero]

On Thu, Jul 11, 2013 at 9:43 AM, Clint Byrum  wrote:

> Excerpts from Robert Collins's message of 2013-07-10 20:54:26 -0700:
> > Clint suggested we do a mid-cycle sprint at the weekly meeting a
> > fortnight ago, but ETIME and stuff - so I'm following up.
> >
> > HP would be delighted to host a get-together of TripleO contributors
> > [or 'I will be contributing soon, honest'] folk.
> >
> > We're proposing a late August / early Sept time - a couple weeks
> > before H3, so we can be dealing with features that have landed //
> > ensuring necessary features *do* land.
> >
> > That would give a start date of the 19th or 26th August. Probable
> > venue of either Sunnyvale, CA or Seattle.
>
> Wife booked a trip out of town August 27 - Sep 2 so that time frame is
> unavailable to me.
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>


I'll be in DebConf13 (Switzerland) the week of August 11th-17th so I prefer
the week of the 19th.

Ghe Rivero



-- 
Pinky: "Gee, Brain, what do you want to do tonight?"
The Brain: "The same thing we do every night, Pinky—try to take over the
world!"

 .''`.  Pienso, Luego Incordio
: :' :
`. `'
  `-www.debian.orgwww.openstack.com

GPG Key: 26F020F7
GPG fingerprint: 4986 39DA D152 050B 4699  9A71 66DB 5A36 26F0 20F7
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Savanna-all] installation problem

[Ruslan Kamaldinov]

That's ok. You need to specify tenant id and provide auth token in headers. 
You can find detailed how-tos here 
https://savanna.readthedocs.org/en/latest/devref/quickstart.html


btw, please use openstack-dev mail-list for Savanna-related questions. Just 
prefix mail subject with [Savanna].

Ruslan 


On Friday, July 12, 2013 at 1:59 PM, Arindam Choudhury wrote:

> Hi,
> 
> The new document works great except this command $ sudo pip install 
> savannadashboard.
> 
> I changed the savanna config to:
> [DEFAULT]
> 
> # REST API config
> port=8386
> allow_cluster_ops=true
> 
> # Address and credentials that will be used to check auth tokens
> os_auth_host=192.168.122.11
> os_auth_port=35357
> os_admin_username=admin
> os_admin_password=openstack
> os_admin_tenant_name=admin
> 
> # (Optional) Name of log file to output to. If not set,
> # logging will go to stdout. (string value)
> log_file=/home/arindam/savanna.log
> 
> [cluster_node]
> 
> # An existing user on Hadoop image (string value)
> #username=root
> 
> # User's password (string value)
> #password=swordfish
> 
> # When set to false, Savanna uses only internal IP of VMs.
> # When set to true, Savanna expects OpenStack to auto-assign
> # floating IPs to cluster nodes. Internal IPs will be used for
> # inter-cluster communication, while floating ones will be
> # used by Savanna to configure nodes. Also floating IPs will
> # be exposed in service URLs (boolean value)
> use_floating_ips=true
> 
> [sqlalchemy]
> 
> # URL for sqlalchemy database (string value)
> database_uri=sqlite:tmp/savanna-server.db
> 
> 
> and I changed config of dashboard as specified.
> 
> but I can not access the savanna dashboard:
> 
> [arindam@sl-3 ~]$ curl http://localhost:8386/v1.0
> 
>  
>   401 Unauthorized
>  
>  
>   401 Unauthorized
>   This server could not verify that you are authorized to access the document 
> you requested. Either you supplied the wrong credentials (e.g., bad 
> password), or your browser does not understand how to supply the credentials 
> required.
> Authentication required
> 
> 
>  
> 
> 
> 
> From: arin...@live.com (mailto:arin...@live.com)
> To: rkamaldi...@mirantis.com (mailto:rkamaldi...@mirantis.com)
> Subject: RE: [Savanna-all] installation problem
> Date: Fri, 12 Jul 2013 11:33:24 +0200
> 
> Thanks.
> 
> Date: Fri, 12 Jul 2013 13:25:45 +0400
> From: rkamaldi...@mirantis.com (mailto:rkamaldi...@mirantis.com)
> To: arin...@live.com (mailto:arin...@live.com)
> CC: openstack-dev@lists.openstack.org 
> (mailto:openstack-dev@lists.openstack.org); savanna-...@lists.launchpad.net 
> (mailto:savanna-...@lists.launchpad.net)
> Subject: Re: [Savanna-all] installation problem
> 
> It seems that you're using the latest code from Savanna v0.2, but following 
> instructions from Savanna v0.1.
> Please follow these docs: https://savanna.readthedocs.org/en/latest/
> 
> 
> Ruslan 
> 
> On Friday, July 12, 2013 at 1:19 PM, Arindam Choudhury wrote:
> 
> > Hi,
> > 
> > While installing savanna I am getting this error:
> > 
> > [arindam@sl-3 savanna]$ tox -evenv -- bin/savanna-db-manage --config-file 
> > etc/savanna/savanna.conf reset-db --with-gen-templates
> > GLOB sdist-make: /home/arindam/savanna/setup.py
> > venv inst-nodeps: 
> > /home/arindam/savanna/.tox/dist/savanna-0.2.a12.gf85d74f.zip
> > venv runtests: commands[0] | bin/savanna-db-manage --config-file 
> > etc/savanna/savanna.conf reset-db --with-gen-templates
> > ERROR: InvocationError: could not find executable 'bin/savanna-db-manage'
> > _ summary 
> > __
> > ERROR:   venv: commands failed
> > 
> > 
> > Regards,
> > Arindam
> > -- 
> > Mailing list: https://launchpad.net/~savanna-all
> > Post to : savanna-...@lists.launchpad.net 
> > (mailto:savanna-...@lists.launchpad.net)
> > Unsubscribe : https://launchpad.net/~savanna-all
> > More help : https://help.launchpad.net/ListHelp
> > 
> > 
> 
> 
> 
> -- 
> Mailing list: https://launchpad.net/~savanna-all
> Post to : savanna-...@lists.launchpad.net 
> (mailto:savanna-...@lists.launchpad.net)
> Unsubscribe : https://launchpad.net/~savanna-all
> More help : https://help.launchpad.net/ListHelp
> 
> 


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Campus Network Blueprint

[Zang MingJie]

Hi Filipe:

I disagree your ml2-external-port BP

It is unsuitable to connect multiple l2 networks directly, there may
be ip conflict, dhcp conflict and other problems. although neutron
dhcp agent won't respond dhcp request from unknown source, an external
dhcp may respond vm dhcp request. If we move an external port form a
network to another network, how can we ensure that the arp cache is
cleared.

And it will aslo make l2-population bp (
https://blueprints.launchpad.net/quantum/+spec/l2-population ) more
difficault. Our l2 forwarding works better if the device knows the
whole topology of the network, but the external part is totally
unknown.

So, I suggest a layer-3 solution, where the out world connects to vms
via l3 agent.


Thank you for sharing the idea
Regards

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Savanna-all] installation problem

[Ruslan Kamaldinov]

It seems that you're using the latest code from Savanna v0.2, but following 
instructions from Savanna v0.1.
Please follow these docs: https://savanna.readthedocs.org/en/latest/


Ruslan 


On Friday, July 12, 2013 at 1:19 PM, Arindam Choudhury wrote:

> Hi,
> 
> While installing savanna I am getting this error:
> 
> [arindam@sl-3 savanna]$ tox -evenv -- bin/savanna-db-manage --config-file 
> etc/savanna/savanna.conf reset-db --with-gen-templates
> GLOB sdist-make: /home/arindam/savanna/setup.py
> venv inst-nodeps: /home/arindam/savanna/.tox/dist/savanna-0.2.a12.gf85d74f.zip
> venv runtests: commands[0] | bin/savanna-db-manage --config-file 
> etc/savanna/savanna.conf reset-db --with-gen-templates
> ERROR: InvocationError: could not find executable 'bin/savanna-db-manage'
> _ summary 
> __
> ERROR:   venv: commands failed
> 
> 
> Regards,
> Arindam
> -- 
> Mailing list: https://launchpad.net/~savanna-all
> Post to : savanna-...@lists.launchpad.net 
> (mailto:savanna-...@lists.launchpad.net)
> Unsubscribe : https://launchpad.net/~savanna-all
> More help : https://help.launchpad.net/ListHelp
> 
> 


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack] Improve inject network configuration

[Robert Collins]

On 12 July 2013 20:43, Thierry Carrez  wrote:
> Brian Lamar wrote:
>>> Honestly, I think network injection is evil and I'd rather remove it
>>> completely. I'm certainly not too interested in trying to add more
>>> features to it.
>>
>> Can you elaborate on this a little more? Do you not like file injection
>> or dynamic network allocation?
>
> It's an old discussion... in summary:
>
> Nova inserting stuff pre-booting into the VM it runs = evil, brittle and
> the source of countless past vulnerabilities
>
> VMs auto-configuring at boot-time using cloud-init based on data
> provided through generic input channels (config drive, metadata
> servers...) = good
>
> So this is not about disliking the ability to insert files or specify
> network parameters for a VM, it's about who is in charge of actually
> creating files and network configurations. Nova shouldn't have to learn
> about the specificities of the VM image it runs, nor should it have to
> mount VM filesystems before booting them. The VM itself should take care
> of the translation based on standardized input (if it wants to).
>
>> Can you provide alternative strategies that could be applied to solve
>> the issue of dynamically brining up interfaces or do you think this is
>> out of the project scope (controlling the internals of VMs)?
>
> Config-drive should pass that config to the VM, and cloud-init on the VM
> should pick it up.

Or the instance should just auto-adjust. Chris Jones wrote some code
for that for tripleo, but we can't use it until we can disable file
injection... and I can't find where we stashed it.

Chris?

-Rob

-- 
Robert Collins 
Distinguished Technologist
HP Cloud Services

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] volume affinity filter for nova scheduler

[Robert Collins]

On 11 July 2013 02:39, Russell Bryant  wrote:

>> We'll probably need something like this for Ironic with persistent
>> volumes on machines - yes its a rare case, but when it matters, it
>> matters a great deal.
>
> I believe you, but I guess I'd like to better understand how this works
> to make sure what gets added actually solves your use case.  Is there
> already support for Cinder managed persistent volumes that live on
> baremetal nodes?

There isn't, but we discussed it with Cinder folk in Portland.

Basic intent is this:
 - we have a cinder 'Ironic' backend.
 - volume requests for the Ironic backend are lazy provisioned: they
just allocate a UUID on creation
 - nova-bm/Ironic will store a volume <-> node mapping
 - 'nova boot' without a volume spec will only select nodes with no
volumes associated to them.
 - 'nova boot' with a volume spec will find an existing node with
those volumes mapped to it, or if none exists create the volume
mapping just-in-time
 - the deployment ramdisk would setup the volume on the hardware
[using hardware RAID]
   - where there isn't hardware RAID we'd let the instance take care
of how to setup the persistent storage - because we don't have a
translation layer in place we can't assume lvm or windows volume
manager or veritas or.

The obvious gap between intent and implementation here is that choices
about nodes happen in the nova scheduler, so we need the scheduler to
be able to honour four cases:
 - baremetal flavor with no volumes requested, gets a baremetal node
with no volumes mapped
 - baremetal flavor with volumes requested, just one baremetal node
with any of those volumes exist -> that node
 - baremetal flavor with volumes requested, > one baremetal node with
any of those volumes exist -> error
 - baremetal flavor with volumes requested, no nodes with any of those
volumes -> pick any node that has enough disks to supply the volume
definitions

Writing this it seems like the nova scheduler may be a tricky fit;
perhaps we should - again- reevaluate just how this all glues
together?

-Rob
-- 
Robert Collins 
Distinguished Technologist
HP Cloud Services

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack] Improve inject network configuration

[Thierry Carrez]

Brian Lamar wrote:
>> Honestly, I think network injection is evil and I'd rather remove it
>> completely. I'm certainly not too interested in trying to add more
>> features to it.
> 
> Can you elaborate on this a little more? Do you not like file injection
> or dynamic network allocation?

It's an old discussion... in summary:

Nova inserting stuff pre-booting into the VM it runs = evil, brittle and
the source of countless past vulnerabilities

VMs auto-configuring at boot-time using cloud-init based on data
provided through generic input channels (config drive, metadata
servers...) = good

So this is not about disliking the ability to insert files or specify
network parameters for a VM, it's about who is in charge of actually
creating files and network configurations. Nova shouldn't have to learn
about the specificities of the VM image it runs, nor should it have to
mount VM filesystems before booting them. The VM itself should take care
of the translation based on standardized input (if it wants to).

> Can you provide alternative strategies that could be applied to solve
> the issue of dynamically brining up interfaces or do you think this is
> out of the project scope (controlling the internals of VMs)?

Config-drive should pass that config to the VM, and cloud-init on the VM
should pick it up.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Nova service(s) prolem when using Mysql behind HAProxy

[Chu Duc Minh]

Hi, when using Mysql behind Haproxy, i have a problem on reboot when some
nova services start after Haproxy service, but before Mysql service.
These service failed: (i re-checked for sure in /var/log/boot.log)
 * Starting Nova cert
[fail]
 * Starting Nova conductor
[fail]
 * Starting Nova scheduler
[fail]
 * Starting Cinder scheduler server
[fail]

I must login to server and re-start these services manually.

When check log of Nova-cert, I saw:

*2013-07-12 15:20:05.020 2490 CRITICAL nova [-] (OperationalError) (2013,
"Lost connection to MySQL server at 'reading initial communic
ation packet', system error: 0") None None*
2013-07-12 15:20:05.020 2490 TRACE nova Traceback (most recent call last):
2013-07-12 15:20:05.020 2490 TRACE nova   File "/usr/bin/nova-cert", line
51, in 
2013-07-12 15:20:05.020 2490 TRACE nova service.wait()
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/service.py", line 689, in wait
2013-07-12 15:20:05.020 2490 TRACE nova _launcher.wait()
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/service.py", line 209, in wait
2013-07-12 15:20:05.020 2490 TRACE nova super(ServiceLauncher,
self).wait()
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/service.py", line 179, in wait
2013-07-12 15:20:05.020 2490 TRACE nova service.wait()
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 168, in
wait
2013-07-12 15:20:05.020 2490 TRACE nova return self._exit_event.wait()
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait
2013-07-12 15:20:05.020 2490 TRACE nova return hubs.get_hub().switch()
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch
2013-07-12 15:20:05.020 2490 TRACE nova return self.greenlet.switch()
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 194, in
main
2013-07-12 15:20:05.020 2490 TRACE nova result = function(*args,
**kwargs)
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/service.py", line 147, in run_server
2013-07-12 15:20:05.020 2490 TRACE nova server.start()
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/service.py", line 434, in start
2013-07-12 15:20:05.020 2490 TRACE nova self.host, self.binary)
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/conductor/api.py", line 261, in
service_get_by_a
rgs
2013-07-12 15:20:05.020 2490 TRACE nova binary=binary)
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/utils.py", line 1348, in wrapper
2013-07-12 15:20:05.020 2490 TRACE nova return func(*args, **kwargs)
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/common.py",
line 424, in in
ner
2013-07-12 15:20:05.020 2490 TRACE nova return
catch_client_exception(exceptions, func, *args, **kwargs)
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/common.py",
line 407, in ca
tch_client_exception
2013-07-12 15:20:05.020 2490 TRACE nova return func(*args, **kwargs)
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 325, in
service_get_
all_by
2013-07-12 15:20:05.020 2490 TRACE nova result =
self.db.service_get_by_args(context, host, binary)
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/db/api.py", line 155, in
service_get_by_args
2013-07-12 15:20:05.020 2490 TRACE nova return
IMPL.service_get_by_args(context, host, binary)
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 96, in
wrapper
2013-07-12 15:20:05.020 2490 TRACE nova return f(*args, **kwargs)
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 409, in
service_get_by_args
2013-07-12 15:20:05.020 2490 TRACE nova result = model_query(context,
models.Service).\
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 177, in
model_query
2013-07-12 15:20:05.020 2490 TRACE nova session = kwargs.get('session')
or get_session()
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/db/sqlalchemy/session.py",
line 325, in get_session
2013-07-12 15:20:05.020 2490 TRACE nova engine = get_engine()
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/db/sqlalchemy/session.py",
line 446, in get_engine
2013-07-12 15:20:05.020 2490 TRACE nova _ENGINE =
create

Re: [openstack-dev] SQLAlchemy-migrate needs a new maintainer

[Thierry Carrez]

Monty Taylor wrote:
> This brings us to the most important question:
> 
> Who wants to be on the core team?

That's the important question indeed. Accepting it (permanently or
temporarily) under stackforge is an easy decision. But it's useless
unless we have a set of people sufficiently interested in it not
bitrotting to volunteer to maintain it...

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Bug #1194026

[Nachi Ueno]

Hi folks

I think I found possible cause of this problems.

so we expected all RPC call is executed serialized way on l3-agent
However it is executed in random order.

http://paste.openstack.org/show/40156/
line starts from  Get is RPC message log.
line starts from [ Process is when l3-agent start processing rpc messages.
(I added rpc message number for debugging)

https://bugs.launchpad.net/neutron/+bug/1194026

Here is my proposal for fixing code.

- Server side simply notifies when something updated.
- Client will update updated flag in the client when it get updated
- some looping call will check the flag,
  if the flag is true, it will full sync with servers

If this is OK, I'll start write it.

Thanks
Nachi












2013/7/11 Salvatore Orlando :
> Adding openstack-dev to this discussion thread.
> Looks like the test is going to be skipped at the moment, but we probably
> need to consider raising the priority of this issue and assign our cores
> with more experience with tempest/gating on this.
>
> salvatore
>
>
> On 9 July 2013 22:48, Maru Newby  wrote:
>>
>> My suggestion is that the quantum exercise script be disabled for now if
>> that will allow the tempest test to run, since the tempest test is more
>> useful (it does an ssh check to ensure that the metadata service has
>> configured the VM).  Doing so would allow useful gating while we look at
>> resolving the timing bug.
>>
>>
>> m.
>>
>> On Jul 9, 2013, at 5:42 PM, Nachi Ueno  wrote:
>>
>> > Hi Maru
>> >
>> > The gating test will not fail everytime. Sometimes, both of tests
>> > works, sometimes not.
>> > In this test, execise.sh works and tempest don't works.
>> > I'm still not sure is there any dependencies in this failure or not.
>> >
>> > So I'm assuming this is kind of timing issue..
>> >
>> > hmm this kind of bug is hard to fix..
>> >
>> >
>> > 2013/7/9 Maru Newby :
>> >> If there is a conflict between the exercise test and the tempest test,
>> >> does the tempest test pass if the exercise script isn't run beforehand?
>> >>
>> >>
>> >> m.
>> >>
>> >> On Jul 9, 2013, at 5:20 PM, Nachi Ueno  wrote:
>> >>
>> >>> Hi
>> >>>
>> >>> I checked briefly, and it looks some timing bug of l3-agent.
>> >>> I added note on the bug report.
>> >>> https://bugs.launchpad.net/neutron/+bug/1194026
>> >>>
>> >>> 2013/7/9 Salvatore Orlando :
>>  Sean Dague singled it out as the biggest cause for gate failures, and
>>  invited us to have a look at it.
>>  I've raised its importance to high, but I don't have the cycles to
>>  look at
>>  it in the short term.
>>  It would be really if somebody from the core team finds some time to
>>  triage
>>  it.
>> 
>>  Salvatore
>> >>
>>
>

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Neutron] please review LBaaS agent scheduling

[Oleg Bondarev]

Hi folks,
While havana 2 is approaching I can't get a review from core devs for more
than 2 weeks. That fact makes me a little nervous :) Could anyone please
review?
https://review.openstack.org/#/c/32137/

Thanks,
Oleg
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Savanna] Savanna meeting minutes

[Sergey Lukjanov]

Thanks everyone who have joined today's Savanna meeting.

Here are the logs from the last meeting (July, 11):

Minutes: 
http://eavesdrop.openstack.org/meetings/savanna/2013/savanna.2013-07-11-18.05.html
Minutes (text): 
http://eavesdrop.openstack.org/meetings/savanna/2013/savanna.2013-07-11-18.05.txt
Log: 
http://eavesdrop.openstack.org/meetings/savanna/2013/savanna.2013-07-11-18.05.log.html

Sincerely yours,
Sergey Lukjanov
Savanna Technical Lead
Mirantis Inc.


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev