Re: [Openstack] Instance IDs and Multiple Zones

[Chris Behrens]

It's early here, but I think it's closer to 200 zones? :)

On Mar 24, 2011, at 5:16 AM, Ed Leafe  wrote:

> On Mar 23, 2011, at 9:41 PM, Justin Santa Barbara wrote:
> 
>> The type of a server @id in CloudServers is xsd:int, which is a 32-bit 
>> signed integer:
>> http://docs.rackspacecloud.com/servers/api/v1.0/xsd/server.xsd
>> 
>> So if you have 1 billion integers per zone, you only get 2 zones.  You can 
>> have 4 if you're willing to go negative, but surely it's too early in the 
>> campaign.
> 
>Yes, you're correct. That always trips me up: why would anyone pick a 
> signed integer for a PK?
> 
>OK, so I'll slice the ranges down to the current Rackspace practice of 10 
> million. That will allow for around 2000 zones.
> 
> 
> 
> -- Ed Leafe
> 
> 
> 
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp


Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace. 
Any dissemination, distribution or copying of the enclosed material is 
prohibited.
If you receive this transmission in error, please notify us immediately by 
e-mail
at ab...@rackspace.com, and delete the original message. 
Your cooperation is appreciated.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Summit Talk: Information session on Zones? Any interest?

[Chris Behrens]

+2

On Apr 14, 2011, at 9:07 AM, Sandy Walsh 
mailto:sandy.wa...@rackspace.com>> wrote:

I've been getting a lot of questions about Zones lately.

How much interest is there for an informational session on Zones and, I guess, 
Distributed Scheduler and roadmap?

(pending an available slot at the summit ... things are filling up quickly I 
gather)

-S


Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace.
Any dissemination, distribution or copying of the enclosed material is 
prohibited.
If you receive this transmission in error, please notify us immediately by 
e-mail
at ab...@rackspace.com, and delete the original 
message.
Your cooperation is appreciated.


___
Mailing list:  
https://launchpad.net/~openstack
Post to :  
openstack@lists.launchpad.net
Unsubscribe :  
https://launchpad.net/~openstack
More help   :  
https://help.launchpad.net/ListHelp


Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace.
Any dissemination, distribution or copying of the enclosed material is 
prohibited.
If you receive this transmission in error, please notify us immediately by 
e-mail
at ab...@rackspace.com, and delete the original message.
Your cooperation is appreciated.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Code Reviews

[Chris Behrens]

I'm not nova-core, but this makes a heck of a lot of sense to me, too.

On May 11, 2011, at 1:16 PM, Jay Pipes wrote:

> ++ on your suggestions.
> 
> On Wed, May 11, 2011 at 3:38 PM, Vishvananda Ishaya
>  wrote:
>> Hello Everyone,
>> We have quite a large backlog of merge proposals here:
>> https://code.launchpad.net/~rlane/nova/lp773690/+merge/59565
>> I've been attempting to go through them to find some high priority ones to
>> review.  It seems like people are being pretty active in reviewing branches,
>> but there are a lot old branches that haven't been touched in a while.  So
>> first I have a general request that anyone who has old branches in for
>> review:  please update your branches or mark them Work In Progress to remove
>> them from the review queue.
>> I'd also like to propose a change to our process that will make the ready to
>> review branches easier to find. I'd like for nova-core to set branches to
>> WIP if there are two significant needs fixings or or needs information.
>>  That way everyone doesn't have to sort through branches that have already
>> been reviewed but are waiting on updates.  We may need to use our judgement
>> here, so if a large branch has a needs fixing for a minor typo or some such,
>> you could leave it under needs review so it gets viewed by more people.
>> Here is an example where i think this policy will be useful:
>> You see a branch that already has a 'Needs Fixing: this needs a failing
>> test".  If you look at the branch and reach the same conclusion, you can
>> mark it "Needs Fixing: I agree, needs a test like xxx" and then set the
>> branch to Work In Progress.  When the author has added the test or needs to
>> make more comments, he can set it back to Needs Review.
>> I think this will generally keep the review board a little cleaner and also
>> each branch will end up with a couple of people that are queued to review
>> once the changes have come in. Does this seem acceptable to everyone?  If I
>> don't here any major dissents, I will add this info to the wiki and we can
>> put it into practice.
>> Vish
>> ___
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
>> 
>> 
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Global deployment of Glance

[Chris Behrens]

Each zone should definitely have glance instances, IMO.  At least two per zone 
for redundancy and networking reasons in large OpenStack installations.  
There's some work to do to support this, though.

- Chris

On May 17, 2011, at 8:59 AM, Glen Campbell wrote:

> If we are going to deploy Glance to support a global deployment of Nova, 
> would it make sense to have replicas in different regions for better 
> performance?
> 
> Or, to put it another way, is there a recommended way to keep multiple Glance 
> installations in sync?
> 
> Users doing snapshots/backups, etc., would presumably get better performance 
> if Glance was local, but how would we keep the base/shared images in sync?
> 
> 
> Confidentiality Notice: This e-mail message (including any attached or
> embedded documents) is intended for the exclusive and confidential use of the
> individual or entity to which this message is addressed, and unless otherwise
> expressly indicated, is confidential and privileged information of Rackspace.
> Any dissemination, distribution or copying of the enclosed material is 
> prohibited.
> If you receive this transmission in error, please notify us immediately by 
> e-mail
> at ab...@rackspace.com, and delete the original message.
> Your cooperation is appreciated.
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Global deployment of Glance

[Chris Behrens]

Ignoring how it is actually implemented, I think we do want copies of base 
images in reach region.  We don't want any sort of outage in one region to 
adversely affect another region.

- Chris


On May 17, 2011, at 9:36 AM, Jay Pipes wrote:

> On Tue, May 17, 2011 at 11:59 AM, Glen Campbell
>  wrote:
>> If we are going to deploy Glance to support a global deployment of Nova, 
>> would it make sense to have replicas in different regions for better 
>> performance?
>> Or, to put it another way, is there a recommended way to keep multiple 
>> Glance installations in sync?
> 
> Hi Glen!
> 
> I think a better idea than having multiple copies of an image in
> different regions is to do two things:
> 
> a) Use a proxy caching server like Varnish or Squid to cache pieces or
> all of an image in various zones
> b) Use a highly-available storage system like Swift behind the global
> Glance server
> 
> For a) we need to complete the HTTP 1.1 Cache headers blueprint
> (https://blueprints.launchpad.net/glance/+spec/caching) and for b) you
> would simply use the Swift backend, configured appropriately for a
> large Swift cluster.
> 
>> Users doing snapshots/backups, etc., would presumably get better performance 
>> if Glance was local, but how would we keep the base/shared images in sync?
> 
> This is actually something that Rick H and Chris McG are working on.
> The basic strategy that they came up with was to add a parent ID
> attribute to the image and for any snapshot image, simply refer to the
> base image as the snapshot image's parent. The glance client would
> check for a parent_id that wasn't null and continue streaming the
> image while it found a parent URI/ID.
> 
> For example, let's say you have a "golden image" with the URI:
> http://glance.example.com/12345. A user creates an instance with this
> image and some time later, decides to do a snapshot or backup of their
> running instance. The snapshotting code in the virtualization layer
> produces what is essentially a differential snapshot, containing only
> the differing bits of the existing image with the base golden image.
> This snapshot (typically much smaller than the original image) could
> be stored in the local (zone-local) Glance server with a call to POST
> /images. When pushing this snapshot image to the local Glance server,
> we would set the parent ID to http://glance.example.com/12345.
> 
> Let's say at some later time, the user wanted to restore from this
> backup. The virtualization layer that implemented the restore call
> would need to stream the backup image from the local Glance server. In
> doing so, it would use the glance client class' get_image() method.
> When calling this method, the glance client would first return the
> snapshot image piece. Noticing the image had a parent ID, it would
> continue to stream the golden image from the global image Glance
> server in-line, essentially enabling us to store only the small diff
> of the snapshot locally while streaming the bulk of the image master
> from the global Glance server.
> 
> I'll let Rick elaborate on the above and correct any mistakes I made
> in my description. :)
> 
> -jay
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Injecting user data into instances

[Chris Behrens]

Hm, inject_file is supposed to be tied into instance creation in the API.  If 
it's not, we have some code missing from API.  Ed Leafe did this work IIRC, as 
we use this here at Rackspace.  Wonder if some code got dropped at some point 
or if it was just never completely finished.

- Chris


On Jun 8, 2011, at 8:27 PM, Mark Washenberger wrote:

> I don't know much about Cloudpipe and VPN, so I hope I don't hijack the 
> thread. However, regarding inject_file
> 
>> Another interesting situation is with inject_file compute APIs  …
>> 
>> 
>> on API level there is no even file/contents fields, only
>> def inject_file(self, context, instance_id):
>> 
>> but they exist on compute.manager level:
>> def inject_file(self, context, instance_id, path, file_contents):
> 
> I believe that the nova.compute.api inject_file function is there 
> erroneously. From my brief look, it does not appear to be called anywhere and 
> it does not appear that it would work correctly. With the current feature set 
> it seems like the only time that you can inject a file is during other api 
> actions similar to instance creation.
> 
> I take it this is a feature you would like to be able to use when the 
> instance has been running for some time?
> 
> Thanks
> 
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Cross-zone instance identifiers in EC2 API - Is it worth the effort?

[Chris Behrens]

On Jul 8, 2011, at 5:11 AM, George Reese wrote:

> I would just like to re-iterate that I think the entire UUID approach is 
> flawed and issues like this are one of the key reasons why.

The only problem I'm aware of is that developers using the EC2 API are not 
adhering to the spec.  If everyone treated them as strings, as they are 
supposed to be, then we wouldn't have to have this discussion.

That said, I do have a particular problem with the current UUID implementation 
in that I wish some sort of unique zone identifier were a part of it.  
Accompany that with some other changes to zones and we could have more 
efficient zone routing.  A side effect of that would be that it is less work to 
come up with an ID that, if truncated, would also be unique to EC2.  Taking Ed 
Leafe's approach, you could remove the recursive zone checks.

I'm not sure I'd vote for that route, though.  It implies we kludge UUID 
generation just for EC2 in the heart of nova,  which I think is completely 
wrong.  I'm pretty much with Vish on everything he's said so far in this thread.

- Chris

This email may include confidential information. If you received it in error, 
please delete it.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Cross-zone instance identifiers in EC2 API - Is it worth the effort?

[Chris Behrens]

Yeah, I'm not sure *how much* of the zone information would be sensitive, 
though.  Ie, is it okay to expose a unique identifier and nothing more?  Or do 
we want to expose _nothing_?

- Chris


On Jul 8, 2011, at 2:28 PM, Sandy Walsh wrote:

> Isn't there a concern of leaking internal Zone information to the outside 
> world (particularly in the Service Provider model)? If so, we're back to the 
> mapping table.
> 
> And, when multi-instance boot commands are more common ("provision me 10 
> servers" vs. 1), then more people will be searching by Reservation Id, 
> Project Id or Owner Id (I suspect). So, how long will this be a problem for?
> 
> Do the same quirks apply to EC2 Reservation ID's as Instance ID's?
> 
> -S
> 
> From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net 
> [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf 
> of Chris Behrens [chris.behr...@rackspace.com]
> Sent: Friday, July 08, 2011 5:43 PM
> To: George Reese
> Cc: ; Ed Leafe; Chris Behrens
> Subject: Re: [Openstack] Cross-zone instance identifiers in EC2 API - Is it 
> worth the effort?
> 
> On Jul 8, 2011, at 5:11 AM, George Reese wrote:
> 
>> I would just like to re-iterate that I think the entire UUID approach is 
>> flawed and issues like this are one of the key reasons why.
> 
> The only problem I'm aware of is that developers using the EC2 API are not 
> adhering to the spec.  If everyone treated them as strings, as they are 
> supposed to be, then we wouldn't have to have this discussion.
> 
> That said, I do have a particular problem with the current UUID 
> implementation in that I wish some sort of unique zone identifier were a part 
> of it.  Accompany that with some other changes to zones and we could have 
> more efficient zone routing.  A side effect of that would be that it is less 
> work to come up with an ID that, if truncated, would also be unique to EC2.  
> Taking Ed Leafe's approach, you could remove the recursive zone checks.
> 
> I'm not sure I'd vote for that route, though.  It implies we kludge UUID 
> generation just for EC2 in the heart of nova,  which I think is completely 
> wrong.  I'm pretty much with Vish on everything he's said so far in this 
> thread.
> 
> - Chris
> 
> This email may include confidential information. If you received it in error, 
> please delete it.
> 
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp

This email may include confidential information. If you received it in error, 
please delete it.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Cross-zone instance identifiers in EC2 API - Is it worth the effort?

[Chris Behrens]

On Jul 11, 2011, at 12:01 PM, Ed Leafe wrote:

> On Jul 11, 2011, at 2:04 PM, Eric Day wrote:
> 
>>> How is
>>> 
>>> nova--
>>> 
>>> any different than:
>>> 
>>> ----
>>> 
>>> Where // (or some subset of them) are reserved/regulated?
>> 
>> Nothing, if -- is a full UUID. If we compare to
>> swift, the account prefix is a UUID too. The account prefix could be
>> fixed for a session or passed in to every request depending on how
>> things are decided.
> 
>   
> 
>   It's a shame that the ipv6 proposal was never more fully considered. 
> That would handle the uniqueness, with the added benefit of providing simple 
> zone routing via DNS, with the exact same 128-bit/32 char size.

I don't I remember that proposal, but that's such a neat idea.  Was anything 
discussed at all in Santa Clara regarding encoding zone information in the 
instance identifier?  I apparently missed the instance identifier discussion 
somehow.

- Chris


This email may include confidential information. If you received it in error, 
please delete it.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Cross-zone instance identifiers in EC2 API - Is it worth the effort?

[Chris Behrens]

On Jul 11, 2011, at 12:37 PM, Ed Leafe wrote:

> On Jul 11, 2011, at 3:24 PM, Chris Behrens wrote:
> 
>>> It's a shame that the ipv6 proposal was never more fully considered. 
>>> That would handle the uniqueness, with the added benefit of providing 
>>> simple zone routing via DNS, with the exact same 128-bit/32 char size.
>> 
>> I don't I remember that proposal, but that's such a neat idea.  Was anything 
>> discussed at all in Santa Clara regarding encoding zone information in the 
>> instance identifier?  I apparently missed the instance identifier discussion 
>> somehow.
> 
> 
>   At the end of the instance referencing discussion, Van Lindbergh 
> brought up the idea. We discussed it with several people both in the 
> conference room and over lunch. I believe that the main objections were that 
> not everyone would have an IPv6 creation scheme, whereas UUID generators are 
> ubiquitous. The other was a vague concern

Ya, I was guessing that would be a concern.  Doesn't seem like a huge deal, 
since everyone should be moving towards ipv6 only anyway.  I could see some 
objections raised if you didn't want any sort of public network interface at 
all... but you could still assign an unused address.

> about "revealing internal structure", but since the information "revealed" 
> would be the exact same info in the instance's public network info, it didn't 
> strike me as a serious concern.

Right.  As long as there is a public network interface, people are going to be 
able to figure out at least some part of 'internal structure'.

- Chris

This email may include confidential information. If you received it in error, 
please delete it.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Cross-zone instance identifiers in EC2 API - Is it worth the effort?

[Chris Behrens]

If you're referring to encoding zone information, yes it would.  I was trying 
to ask more generally as well.  IPv6 would be a very good solution, IMO.

- Chris

On Jul 11, 2011, at 12:47 PM, Sandy Walsh wrote:

> Won't an IPv6 address do that by it's very nature?
> 
> -S
> 
> 
> From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net 
> [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf 
> of Chris Behrens [chris.behr...@rackspace.com]
> Sent: Monday, July 11, 2011 4:24 PM
> To: Ed Leafe
> Cc: openstack@lists.launchpad.net; Chris Behrens
> Subject: Re: [Openstack] Cross-zone instance identifiers in EC2 API - Is it 
> worth the effort?
> 
> On Jul 11, 2011, at 12:01 PM, Ed Leafe wrote:
> 
>> On Jul 11, 2011, at 2:04 PM, Eric Day wrote:
>> 
>>>> How is
>>>> 
>>>> nova--
>>>> 
>>>> any different than:
>>>> 
>>>> ----
>>>> 
>>>> Where // (or some subset of them) are reserved/regulated?
>>> 
>>> Nothing, if -- is a full UUID. If we compare to
>>> swift, the account prefix is a UUID too. The account prefix could be
>>> fixed for a session or passed in to every request depending on how
>>> things are decided.
>> 
>>  
>> 
>>  It's a shame that the ipv6 proposal was never more fully considered. 
>> That would handle the uniqueness, with the added benefit of providing simple 
>> zone routing via DNS, with the exact same 128-bit/32 char size.
> 
> I don't I remember that proposal, but that's such a neat idea.  Was anything 
> discussed at all in Santa Clara regarding encoding zone information in the 
> instance identifier?  I apparently missed the instance identifier discussion 
> somehow.
> 
> - Chris
> 
> 
> This email may include confidential information. If you received it in error, 
> please delete it.
> 
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp

This email may include confidential information. If you received it in error, 
please delete it.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Physical host identification

[Chris Behrens]

Nevermind.  Just found a comment in the API spec that says "hostID" is unique 
per account, not globally.  Hmmm...


On Jul 15, 2011, at 2:27 PM, Chris Behrens wrote:

> I see the v1.1 API spec talks about a 'hostId' item returned when you list 
> your instances (section 4.1.1 in the spec).  These should be the same thing, 
> IMO.
> 
> I think you're right, though.  I don't believe we have any sort of 'hostId' 
> today, since hosts just become available by attaching to AMQP.
> 
> - Chris
> 
> On Jul 15, 2011, at 1:16 PM, Glen Campbell wrote:
> 
>> I understand that we're all familiar with virtualization and its benefits. 
>> However, in the Real World, those of us who run clouds often need to work 
>> with physical devices. I've proposed a blueprint and spec for a /hosts admin 
>> API resource that would return information on physical hosts. However, I 
>> don't believe that there's any way for us to actually identify a specific 
>> server (I'm actually hoping I'm mistaken about this, because that would make 
>> my life easier). 
>> 
>> So, to get information about a specific host, you'd use /host/{id} — but 
>> what should go in the {id} slot?
>> 
>> We'd also like to include this data elsewhere; for example, in error 
>> messages, it might help to know the physical device on which a server is 
>> created. 
>> 
>> 
>> 
>> This email may include confidential information. If you received it in 
>> error, please delete it.
>> ___
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
> 

This email may include confidential information. If you received it in error, 
please delete it.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Physical host identification

[Chris Behrens]

I see the v1.1 API spec talks about a 'hostId' item returned when you list your 
instances (section 4.1.1 in the spec).  These should be the same thing, IMO.

I think you're right, though.  I don't believe we have any sort of 'hostId' 
today, since hosts just become available by attaching to AMQP.

- Chris

On Jul 15, 2011, at 1:16 PM, Glen Campbell wrote:

> I understand that we're all familiar with virtualization and its benefits. 
> However, in the Real World, those of us who run clouds often need to work 
> with physical devices. I've proposed a blueprint and spec for a /hosts admin 
> API resource that would return information on physical hosts. However, I 
> don't believe that there's any way for us to actually identify a specific 
> server (I'm actually hoping I'm mistaken about this, because that would make 
> my life easier). 
> 
> So, to get information about a specific host, you'd use /host/{id} — but what 
> should go in the {id} slot?
> 
> We'd also like to include this data elsewhere; for example, in error 
> messages, it might help to know the physical device on which a server is 
> created. 
> 
> 
> 
> This email may include confidential information. If you received it in error, 
> please delete it.
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp

This email may include confidential information. If you received it in error, 
please delete it.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Physical host identification

[Chris Behrens]

I think it's sensitive because one could figure out how many hosts a SP has 
globally... which a SP might not necessarily want to reveal.

- Chris


On Jul 15, 2011, at 3:34 PM, karim.allah.ah...@gmail.com wrote:

> On Fri, Jul 15, 2011 at 11:31 PM, Chris Behrens  
> wrote:
> Nevermind.  Just found a comment in the API spec that says "hostID" is unique 
> per account, not globally.  Hmmm...
>  
> This is weird ! I can't find anything in the code that says so !! hostID is 
> just a hashed version of the 'host' which is set as the 'hostname' of the 
> physical machine and this isn't user sensitive. So, It's supposed to be a 
> global thing !
> 
> Can somebody explain how this is a user sensitive ?
>  
> 
> 
> On Jul 15, 2011, at 2:27 PM, Chris Behrens wrote:
> 
> > I see the v1.1 API spec talks about a 'hostId' item returned when you list 
> > your instances (section 4.1.1 in the spec).  These should be the same 
> > thing, IMO.
> >
> > I think you're right, though.  I don't believe we have any sort of 'hostId' 
> > today, since hosts just become available by attaching to AMQP.
> >
> > - Chris
> >
> > On Jul 15, 2011, at 1:16 PM, Glen Campbell wrote:
> >
> >> I understand that we're all familiar with virtualization and its benefits. 
> >> However, in the Real World, those of us who run clouds often need to work 
> >> with physical devices. I've proposed a blueprint and spec for a /hosts 
> >> admin API resource that would return information on physical hosts. 
> >> However, I don't believe that there's any way for us to actually identify 
> >> a specific server (I'm actually hoping I'm mistaken about this, because 
> >> that would make my life easier).
> >>
> >> So, to get information about a specific host, you'd use /host/{id} — but 
> >> what should go in the {id} slot?
> >>
> >> We'd also like to include this data elsewhere; for example, in error 
> >> messages, it might help to know the physical device on which a server is 
> >> created.
> >>
> >>
> >> 
> >> This email may include confidential information. If you received it in 
> >> error, please delete it.
> >> ___
> >> Mailing list: https://launchpad.net/~openstack
> >> Post to : openstack@lists.launchpad.net
> >> Unsubscribe : https://launchpad.net/~openstack
> >> More help   : https://help.launchpad.net/ListHelp
> >
> 
> This email may include confidential information. If you received it in error, 
> please delete it.
> 
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
> 
> 
> 
> -- 
> Karim Allah Ahmed.
> LinkedIn
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp

This email may include confidential information. If you received it in error, 
please delete it.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [SPAM] How to connect br100 to xen bridge

[Chris Behrens]

Depending on what you're doing for networking, you should be able to shove this 
into nova.conf:

--flat_network_bridge=xenbr0

to avoid modifying the DB..

- Chris

On Aug 17, 2011, at 5:34 PM, Wayne A. Walls wrote:

> Greetings, Josh!
> 
> Just update the bridge in the db, I've done that in the past to get things 
> aligned with Xen.  Seems to do the trick...
> 
> Cheers,
> 
> Wayne 
> 
> Sent from my iPhone
> 
> On Aug 17, 2011, at 7:17 PM, Joshua Harlow  wrote:
> 
>> Hi all,
>> 
>> Just trying to continue with my xen+openstack integration.
>> 
>> Is there a recommended way to get openstack to connect to the xen bridge.
>> 
>> Apparently the xen bridge is @ xenbr0 but openstack likes to use br100.
>> 
>> Is it just a simple DB update to make this happen, or can this be changed in 
>> the nova-network config?
>> 
>> Has anyone had any experience with this?
>> 
>> Thanks
>> ___
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp

This email may include confidential information. If you received it in error, 
please delete it.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] New nova service proposal

[Chris Behrens]

I was wondering if some of this could be solved by simply using rpc.call vs 
rpc.cast so that we get appropriate responses, even if they are exceptions.

- Chris

On Aug 26, 2011, at 2:23 PM, Brian Lamar wrote:

> Hey Ed,
> 
> I absolutely agree that we need to be confident that all requests will be 
> handled by the system eventually. However, I'm unsure of the need for a new 
> service to be created to handle error cases.
> 
> I'm not saying that we can solve every case through our current architecture, 
> but with some subtle tweaks I think it can be made much more reliable.  I 
> feel like if we look at every place where errors *can* occur we can find 
> solutions not involving an external service. Are there any particular 
> bugs/situations that are happening which aren't listed as bugs in Launchpad?
> 
> Long story short I'd rather not create an external service which attempts to 
> clean up after poor exception handling / bad logic, but there might be some 
> cases I'm not considering.
> 
> Brian
> 
> 
> 
> 
> -Original Message-
> From: "Ed Leafe" 
> Sent: Friday, August 26, 2011 3:22pm
> To: openstack@lists.launchpad.net
> Subject: [Openstack] New nova service proposal
> 
>   Sorry I haven't come up with a snazzy name for it yet, but what I have 
> in mind is a new service that is essential for my employer (Rackspace), and 
> might be important for other OpenStack deployments. This new service would be 
> completely optional, of course - only those for whom it is relevant would run 
> it.
> 
>   Let me start by stating the problem: when a customer requests that we 
> create instances for them, nova casts those requests into the queue, where 
> they are eventually acted upon. That usually works great, but in cases where 
> the instance creation fails, we need to detect that failure and re-issue the 
> create request with a different host. This is currently not possible with the 
> asynchronous design of the compute-scheduler interactions.
> 
>   So what I envision is a service that scans a list of recent requests' 
> reservation IDs, and follows up to see if the request was successful or not, 
> and takes action if needed. The blueprint for this can be found at 
> https://blueprints.launchpad.net/nova/+spec/instance-creation-assurance, with 
> an Etherpad created for ongoing idea exchange at 
> http://etherpad.openstack.org/instance-creation-assurance
> 
> 
> 
> -- Ed Leafe
> 
> 
> 
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
> This email may include confidential information. If you received it in error, 
> please delete it.
> 
> 
> 
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp

This email may include confidential information. If you received it in error, 
please delete it.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] nova trunk will require a new dependency: kombu

[Chris Behrens]

This email wasn't sent sooner, because there was an assumption that since 
glance already depended on kombu.. and that nova depends on glance... that 
kombu is already a required dependency for nova.  However, this must not be 
completely true because we just got a merge failure because of kombu not 
installed for this:

https://code.launchpad.net/~cbehrens/nova/rpc-kombu/+merge/73096

That should be merged into trunk shortly barring any further issues..

python-kombu is in the ppa..  and 'kombu' is in nova/tools/pip-requires 

fyi,

- Chris


This email may include confidential information. If you received it in error, 
please delete it.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] libvirt vs. Xen driver handling of local storage

[Chris Behrens]

Vish,

I think Rackspace ozone/titan has some upcoming work to do for the resizing for 
xenserver that might close some of the gap.

I think we need some options (flags) if we are to synchronize libvirt/xen.  At 
some point, Rackspace also needs an API extension to support a couple different 
ways of handling resizes.  Until we get there, we at least need an option to 
keep the xenserver code working as-is for now.  I assume others need the 
current libvirt implementation to stay as well.

That said, I think it's probably not too difficult to do the 'libvirt way' for 
Xen, but I don't know about it making diablo.
Adding support into libvirt to do the 'xen way' should be easier, I'd think.  
But I'm the opposite of you, Vish.  I don't know the libvirt layer as well. :)

If we can FLAG the way it works... and make these options work in both 
libvirt/xen, I think we can all remain happy.

- Chris

On Aug 31, 2011, at 11:45 AM, Vishvananda Ishaya wrote:

> Hey guys,
> 
> We have a very annoying discrepancy between how local space is used in the 
> xen driver vs the libvirt driver.  I think it is vital that this is rectified 
> before the Diablo release.  We already have a few functional gaps between the 
> drivers, but the fact that disks are partitioned completely differently 
> between the two is very confusing to users.
> 
> Bug is here: https://bugs.launchpad.net/nova/+bug/834189
> 
> The libvirt driver:
> 
> * downloads the image from glance
> * resizes the image to 10G if it is < 10G
> (in the case of a separate kernel and ramdisk image it extends the filesystem 
> as well.  In the case of a whole-disk image it just resizes the file because 
> it doesn't know enough to change the filesystem)
> * attaches a second disk the size of local_gb to the image
> (when using block device mapping through the ec2 api, more swap/ephemeral 
> disks can be attached as volumes as well)
> 
> The XenServer driver (I'm less familiar with this code so please correct me 
> if i am wrong here):
> * downloads the image from glance
> * creates a vdi from the base image
> * resizes the vdi to the size of local_gb
> 
> The first method of resize to 10G and having separate local_gb is essentially 
> the strategy taken by aws.
> 
> Drawbacks of the first method:
> 
> 1) The actual space used by the image is local_gb + 10G (or more if the base 
> image is larger than 10G) which is inconsistent.
> 
> 2) The guest has to deal with the annoyance of not having one large 
> filesystem.  It is easier for the user if they can just use all the space 
> that they have without thinking about it.
> 
> Drawbacks of the second method:
> 
> 1) Limits cloud images to a particular format.  We can't always guarantee 
> that we can resize the image properly.
> 
> 
> We need to decide on a common strategy and use it for both hypervisors.
> 
> Vish
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp

This email may include confidential information. If you received it in error, 
please delete it.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] libvirt vs. Xen driver handling of local storage

[Chris Behrens]

On Sep 2, 2011, at 8:07 AM, Paul Voccio wrote:

> On Fri, Sep 2, 2011 at 8:01 AM, Soren Hansen  wrote:
> [...]
> The potential for filesystem bugs that could bring the host down gives
> me the heebie jeebies. I really, really don't want to mount people's
> filesystems.
> 
> 
> Can you explain a bit more here? I would like to understand your concerns. I 
> would advocate mounting in a utility VM if you mean to protect from mounting 
> instance with malicious data. We may have to do this to expand partitions or 
> resize down for Windows. 

Mounting someone's filesystem should not be necessary if we have certain 
restrictions on the management.  I.e., we could say we will only resize the 
last filesystem in the partition table.  That would avoid needing to know the 
filesystem layout in the image (looking at /etc/fstab or updating it).  Not 
sure that's a desired restriction, however.

That said, we'd still need to attach the VM disk somewhere and run fs resize 
utils... and it might still be best to do this in a utility VM.

- Chris

This email may include confidential information. If you received it in error, 
please delete it.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] libvirt vs. Xen driver handling of local storage

[Chris Behrens]

Yeah, I think that can be rather fair for Unix.

It's just that as you pointed out... Windows is a huge pain.   Need to make 
sure there's enough space on C: and I think there are still a lot of things 
that stupidly rely on being installed on C:


On Sep 2, 2011, at 10:32 AM, Paul Voccio wrote:

> My first thought was to do a singled fixed disk and never resize that disk at 
> all. If you need space, you have to use a volume service. 
> 
> Ultimately, I don't think this the right approach either, but it solves the 
> initial use case of needing more storage space. 
> 
> 
> 
> On Fri, Sep 2, 2011 at 11:34 AM, Chris Behrens  
> wrote:
> 
> On Sep 2, 2011, at 8:07 AM, Paul Voccio wrote:
> 
> > On Fri, Sep 2, 2011 at 8:01 AM, Soren Hansen  wrote:
> > [...]
> > The potential for filesystem bugs that could bring the host down gives
> > me the heebie jeebies. I really, really don't want to mount people's
> > filesystems.
> >
> >
> > Can you explain a bit more here? I would like to understand your concerns. 
> > I would advocate mounting in a utility VM if you mean to protect from 
> > mounting instance with malicious data. We may have to do this to expand 
> > partitions or resize down for Windows.
> 
> Mounting someone's filesystem should not be necessary if we have certain 
> restrictions on the management.  I.e., we could say we will only resize the 
> last filesystem in the partition table.  That would avoid needing to know the 
> filesystem layout in the image (looking at /etc/fstab or updating it).  Not 
> sure that's a desired restriction, however.
> 
> That said, we'd still need to attach the VM disk somewhere and run fs resize 
> utils... and it might still be best to do this in a utility VM.
> 
> - Chris
> 
> This email may include confidential information. If you received it in error, 
> please delete it.
> 
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp

This email may include confidential information. If you received it in error, 
please delete it.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] A possible alternative to Gerrit ...

[Chris Behrens]

Jay,

On Sep 4, 2011, at 10:52 AM, Jay Pipes wrote:

> I actually didn't plan on responding all that much on this
> conversation. We had months of discussion and debate about this, weeks
> upon weeks of discussion in the PPB about project autonomy and
> tooling, and the decision has been made.
> 
> I find it a bit unfortunate that all the people saying Gerrit is
> terrible and that we should just use GitHub haven't done a single
> review or change request in any of the projects that are currently
> using the Gerrit/Git toolset that has been decided will be used for
> core OpenStack projects.
[...]

I'm not sure this is true.  A few people that I've seen speaking up have used 
it.   I've been wanting to comment for a while, but I don't have a lot of 
ground to stand on, because I'm in the boat that you describe.  I've not done a 
single review or change request with Gerrit yet.  That said, I've seen a bit 
from those who have and I'm very much the opposite of excited about moving nova 
to it at some point, at least how it stands now.

But leaving aside whether I like it or dislike it, what really bothers me is 
that there was discussion about moving things to github.  And, I was 'ok' with 
that decision to do so despite preferring bzr and LP.  My 'ok' was based on 
knowing how git, github pull requests, reviews, and so forth work.  Now I feel 
like we're moving things to something to which I (and the community) never 
agreed.   I never saw any discussion about Gerrit on the mailing list as far as 
"is everyone cool with this?"  The first mention of it that I can find was July 
18th regarding moving the CI repos.  Doesn't seem like we were given much of an 
option.  That really irks me.  Above you say "that has been decided will be 
used for the core OpenStack projects".  So, I have to ask:  'Who decided?'.  I 
must have missed something.

And I want to be clear that I'm not meaning to put down anyone's efforts here.  
I'm positive a lot of hard work was put into the transition, and I do 
appreciate it.

- Chris











This email may include confidential information. If you received it in error, 
please delete it.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Reinstating Trey Morris for Nova Core

[Chris Behrens]

+1

On Jan 22, 2013, at 5:38 PM, Matt Dietz  wrote:

> All,
> 
>I think Trey Morris has been doing really well on reviews again, so I'd
> like to propose him to be reinstated for Nova core. Thoughts?
> 
> -Dietz
> 
> 
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Restart devstack errors

[Chris Behrens]

[Removed the dev list -- no need to cross-post.]

It looks like you have broken permissions on 
'/usr/local/lib/python2.7/dist-packages/httplib2-0.7.7-py2.7.egg' and/or 
subdirectories.  Make sure everything is world readable.

- Chris


On Feb 20, 2013, at 7:48 PM, harryxiyou  wrote:

> On Wed, Feb 20, 2013 at 3:14 PM, Hirendra Rathor
>  wrote:
> 
> Hi Hirendra Rathor,
> 
>> I was getting same error when I picked up devstack for the first time few
>> days ago. I could have tried troubleshooting it but I wasn't particularly
>> happy with the fact that I had to launch stack.sh every time on reboot. The
>> script updates the source code and then compiles it which means that the
>> product behavior could be different after running the script. This is not
>> necessarily a problem but I felt less in control.
>> 
>> I, therefore, modified my local copy of stack.sh to _not_ do operations that
>> should be required for the first run. Examples of such operations include
>> creating user accounts, configuration files, mysql database etc. The result
>> was a script that just launches the openstack processes. Running this lean
>> script resulted in successful launch of keystone process, so I didn't have
>> to troubleshoot the original problem!
>> 
> 
> I am not clear about how you can solve 'keystone did not start'.
> Cloud you please attach your modified stack.sh, which solved this problem.
> 
> Now, i can get some logs for my 'keystone did not start' like following.
> 
> $ screen -S stack -p key -X stuff
> $ screen -x stack
> 
> jiawei@jiawei:~/workshop1/devstack$ cd /opt/stack/keystone &&
> /opt/stack/keystone/bin/keystone-all --config-file
> /etc/keystone/keystone.conf --log-config /etc/keystone/logging.conf -d
> --debug || touch "/opt/stack/status/stack/key.failure"
> Traceback (most recent call last):
>  File "/opt/stack/keystone/bin/keystone-all", line 22, in 
>from paste import deploy
>  File 
> "/usr/local/lib/python2.7/dist-packages/Paste-1.7.5.1-py2.7.egg/paste/__init__.py",
> line 4, in 
>import pkg_resources
>  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2727,
> in 
>add_activation_listener(lambda dist: dist.activate())
>  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 700,
> in subscribe
>callback(dist)
>  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2727,
> in 
>add_activation_listener(lambda dist: dist.activate())
>  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2227,
> in activate
>self.insert_on(path)
>  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2334,
> in insert_on
>self.check_version_conflict()
>  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2373,
> in check_version_conflict
>for modname in self._get_metadata('top_level.txt'):
>  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2221,
> in _get_metadata
>for line in self.get_metadata_lines(name):
>  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 1209,
> in get_metadata_lines
>return yield_lines(self.get_metadata(name))
>  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 1201,
> in get_metadata
>return self._get(self._fn(self.egg_info,name))
>  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 1316, in _get
>stream = open(path, 'rb')
> IOError: [Errno 13] Permission denied:
> '/usr/local/lib/python2.7/dist-packages/httplib2-0.7.7-py2.7.egg/EGG-INFO/top_level.txt'
> 
> 
> Could anyone give me some suggestions to this problem? Thanks
> in advance.
> 
> 
> 
> -- 
> Thanks
> Harry Wei
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Restart devstack errors

[Chris Behrens]

Well, you probably don't want world writeable, but :)  755 on dirs and 644 
on files is probably more appropriate!  But at least you know the issue.

- Chris

On Feb 20, 2013, at 9:21 PM, harryxiyou  wrote:

> On Thu, Feb 21, 2013 at 12:14 PM, Chris Behrens  wrote:
>> [Removed the dev list -- no need to cross-post.]
>> 
>> It looks like you have broken permissions on 
>> '/usr/local/lib/python2.7/dist-packages/httplib2-0.7.7-py2.7.egg'
>> nd/or subdirectories.  Make sure everything is world readable.
> 
> You are right. After "sudo chmod -R 777 dist-packages/"
> I could solve this problem successfully.
> Thanks.
> 
> 
> 
> -- 
> Thanks
> Harry Wei

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] AggregateInstanceExtraSpecs very slow?

[Chris Behrens]

On Feb 25, 2013, at 6:39 PM, Joe Gordon  wrote:

> 
> It looks like the scheduler issues are related to the rabbitmq issues.   
> "host 'qh2-rcc77' ... is disabled or has not been heard from in a while"
> 
> What does 'nova host-list' say?   the clocks must all be synced up?

Good things to check.  It feels like something is spinning way too much within 
this filter, though.  This can also cause the above message.  The scheduler 
pulls all of the records before it starts filtering… and if there's a huge 
delay somewhere, it can start seeing a bunch of hosts as disabled.

The filter doesn't look like a problem.. unless there's a large amount of 
aggregate metadata… and/or a large amount of key/values for the instance_type's 
extra specs.   There *is* a DB call in the filter.  If that's blocking for an 
extended period of time, the whole process is blocked…  But I suspect by the 
'100% cpu' comment, that this is not the case…  So the only thing I can think 
of is that it returns a tremendous amount of metadata.

Adding some extra logging in the filter could be useful.

- Chris



___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] AggregateInstanceExtraSpecs very slow?

[Chris Behrens]

Replying from my phone, so I can't look, but I wonder if we have an index 
missing.

On Feb 25, 2013, at 8:54 PM, Sam Morrison  wrote:

> On Tue, Feb 26, 2013 at 3:15 PM, Sam Morrison  wrote:
>> 
>> On 26/02/2013, at 2:15 PM, Chris Behrens  wrote:
>> 
>>> 
>>> On Feb 25, 2013, at 6:39 PM, Joe Gordon  wrote:
>>> 
>>>> 
>>>> It looks like the scheduler issues are related to the rabbitmq issues.   
>>>> "host 'qh2-rcc77' ... is disabled or has not been heard from in a while"
>>>> 
>>>> What does 'nova host-list' say?   the clocks must all be synced up?
>>> 
>>> Good things to check.  It feels like something is spinning way too much 
>>> within this filter, though.  This can also cause the above message.  The 
>>> scheduler pulls all of the records before it starts filtering… and if 
>>> there's a huge delay somewhere, it can start seeing a bunch of hosts as 
>>> disabled.
>>> 
>>> The filter doesn't look like a problem.. unless there's a large amount of 
>>> aggregate metadata… and/or a large amount of key/values for the 
>>> instance_type's extra specs.   There *is* a DB call in the filter.  If 
>>> that's blocking for an extended period of time, the whole process is 
>>> blocked…  But I suspect by the '100% cpu' comment, that this is not the 
>>> case…  So the only thing I can think of is that it returns a tremendous 
>>> amount of metadata.
>>> 
>>> Adding some extra logging in the filter could be useful.
>>> 
>>> - Chris
>> 
>> Thanks Chris, I have 2 aggregates and 2 keys defined and each of the 80 
>> hosts has either one or the other. At the moment every flavour has either 
>> one or the other too so I don't think it's too much data.
>> 
>> I've tracked it down to this call:
>> 
>> metadata = db.aggregate_metadata_get_by_host(context, host_state.host)
> 
> More debugging has got it down to a query
> 
> In db.api.aggregate_metadata_get_by_host:
> 
>query = model_query(context, models.Aggregate).join(
>"_hosts").filter(models.AggregateHost.host == host).join(
>"_metadata")
>   ..
>   rows = query.all()
> 
> With query debug on this resolves to:
> 
> SELECT aggregates.created_at AS aggregates_created_at,
> aggregates.updated_at AS aggregates_updated_at, aggregates.deleted_at
> AS aggregates_deleted_at, aggregates.deleted AS aggregates_deleted,
> aggregates.id AS aggregates_id, aggregates.name AS aggregates_name,
> aggregates.availability_zone AS aggregates_availability_zone,
> aggregate_hosts_1.created_at AS aggregate_hosts_1_created_at,
> aggregate_hosts_1.updated_at AS aggregate_hosts_1_updated_at,
> aggregate_hosts_1.deleted_at AS aggregate_hosts_1_deleted_at,
> aggregate_hosts_1.deleted AS aggregate_hosts_1_deleted,
> aggregate_hosts_1.id AS aggregate_hosts_1_id, aggregate_hosts_1.host
> AS aggregate_hosts_1_host, aggregate_hosts_1.aggregate_id AS
> aggregate_hosts_1_aggregate_id FROM aggregates INNER JOIN
> aggregate_hosts AS aggregate_hosts_2 ON aggregates.id =
> aggregate_hosts_2.aggregate_id AND aggregate_hosts_2.deleted = 0 AND
> aggregates.deleted = 0 INNER JOIN aggregate_hosts ON
> aggregate_hosts.aggregate_id = aggregates.id AND
> aggregate_hosts.deleted = 0 AND aggregates.deleted = 0 INNER JOIN
> aggregate_metadata AS aggregate_metadata_1 ON aggregates.id =
> aggregate_metadata_1.aggregate_id AND aggregate_metadata_1.deleted = 0
> AND aggregates.deleted = 0 INNER JOIN aggregate_metadata ON
> aggregate_metadata.aggregate_id = aggregates.id AND
> aggregate_metadata.deleted = 0 AND aggregates.deleted = 0 LEFT OUTER
> JOIN aggregate_hosts AS aggregate_hosts_3 ON aggregates.id =
> aggregate_hosts_3.aggregate_id AND aggregate_hosts_3.deleted = 0 AND
> aggregates.deleted = 0 LEFT OUTER JOIN aggregate_hosts AS
> aggregate_hosts_1 ON aggregate_hosts_1.aggregate_id = aggregates.id
> AND aggregate_hosts_1.deleted = 0 AND aggregates.deleted = 0 WHERE
> aggregates.deleted = 0 AND aggregate_hosts.host = 'qh2-rcc34';
> 
> Which in our case returns 328509 rows in set (25.97 sec)
> 
> Seems a bit off considering there are 80 rows in aggregate_hosts, 2
> rows in aggregates and 2 rows in aggregate_metadata
> 
> In the code rows is only equal to 1 so it seems to be doing something
> inside to code to do this? Don't know too much how sqlalchemy works.
> 
> Seems like a bug to me? or maybe our database has something wrong in it?
> 
> Cheers,
> Sam

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] AggregateInstanceExtraSpecs very slow?

[Chris Behrens]

After thinking more, it does seem like we're doing something wrong if the query 
itself is returning 300k rows. :)  I can take a better look at it in front of 
the computer later if no one beats me to it.

On Feb 25, 2013, at 9:28 PM, Chris Behrens  wrote:

> Replying from my phone, so I can't look, but I wonder if we have an index 
> missing.
> 
> On Feb 25, 2013, at 8:54 PM, Sam Morrison  wrote:
> 
>> On Tue, Feb 26, 2013 at 3:15 PM, Sam Morrison  wrote:
>>> 
>>> On 26/02/2013, at 2:15 PM, Chris Behrens  wrote:
>>> 
>>>> 
>>>> On Feb 25, 2013, at 6:39 PM, Joe Gordon  wrote:
>>>> 
>>>>> 
>>>>> It looks like the scheduler issues are related to the rabbitmq issues.   
>>>>> "host 'qh2-rcc77' ... is disabled or has not been heard from in a while"
>>>>> 
>>>>> What does 'nova host-list' say?   the clocks must all be synced up?
>>>> 
>>>> Good things to check.  It feels like something is spinning way too much 
>>>> within this filter, though.  This can also cause the above message.  The 
>>>> scheduler pulls all of the records before it starts filtering… and if 
>>>> there's a huge delay somewhere, it can start seeing a bunch of hosts as 
>>>> disabled.
>>>> 
>>>> The filter doesn't look like a problem.. unless there's a large amount of 
>>>> aggregate metadata… and/or a large amount of key/values for the 
>>>> instance_type's extra specs.   There *is* a DB call in the filter.  If 
>>>> that's blocking for an extended period of time, the whole process is 
>>>> blocked…  But I suspect by the '100% cpu' comment, that this is not the 
>>>> case…  So the only thing I can think of is that it returns a tremendous 
>>>> amount of metadata.
>>>> 
>>>> Adding some extra logging in the filter could be useful.
>>>> 
>>>> - Chris
>>> 
>>> Thanks Chris, I have 2 aggregates and 2 keys defined and each of the 80 
>>> hosts has either one or the other. At the moment every flavour has either 
>>> one or the other too so I don't think it's too much data.
>>> 
>>> I've tracked it down to this call:
>>> 
>>> metadata = db.aggregate_metadata_get_by_host(context, host_state.host)
>> 
>> More debugging has got it down to a query
>> 
>> In db.api.aggregate_metadata_get_by_host:
>> 
>>   query = model_query(context, models.Aggregate).join(
>>   "_hosts").filter(models.AggregateHost.host == host).join(
>>   "_metadata")
>>  ..
>>  rows = query.all()
>> 
>> With query debug on this resolves to:
>> 
>> SELECT aggregates.created_at AS aggregates_created_at,
>> aggregates.updated_at AS aggregates_updated_at, aggregates.deleted_at
>> AS aggregates_deleted_at, aggregates.deleted AS aggregates_deleted,
>> aggregates.id AS aggregates_id, aggregates.name AS aggregates_name,
>> aggregates.availability_zone AS aggregates_availability_zone,
>> aggregate_hosts_1.created_at AS aggregate_hosts_1_created_at,
>> aggregate_hosts_1.updated_at AS aggregate_hosts_1_updated_at,
>> aggregate_hosts_1.deleted_at AS aggregate_hosts_1_deleted_at,
>> aggregate_hosts_1.deleted AS aggregate_hosts_1_deleted,
>> aggregate_hosts_1.id AS aggregate_hosts_1_id, aggregate_hosts_1.host
>> AS aggregate_hosts_1_host, aggregate_hosts_1.aggregate_id AS
>> aggregate_hosts_1_aggregate_id FROM aggregates INNER JOIN
>> aggregate_hosts AS aggregate_hosts_2 ON aggregates.id =
>> aggregate_hosts_2.aggregate_id AND aggregate_hosts_2.deleted = 0 AND
>> aggregates.deleted = 0 INNER JOIN aggregate_hosts ON
>> aggregate_hosts.aggregate_id = aggregates.id AND
>> aggregate_hosts.deleted = 0 AND aggregates.deleted = 0 INNER JOIN
>> aggregate_metadata AS aggregate_metadata_1 ON aggregates.id =
>> aggregate_metadata_1.aggregate_id AND aggregate_metadata_1.deleted = 0
>> AND aggregates.deleted = 0 INNER JOIN aggregate_metadata ON
>> aggregate_metadata.aggregate_id = aggregates.id AND
>> aggregate_metadata.deleted = 0 AND aggregates.deleted = 0 LEFT OUTER
>> JOIN aggregate_hosts AS aggregate_hosts_3 ON aggregates.id =
>> aggregate_hosts_3.aggregate_id AND aggregate_hosts_3.deleted = 0 AND
>> aggregates.deleted = 0 LEFT OUTER JOIN aggregate_hosts AS
>> aggregate_hosts_1 ON aggregate_hosts_1.aggregate_id = aggregates.id
>> AND aggregate_hosts_1.deleted = 0 AND aggregates.deleted = 0 WHERE
>> aggregates.deleted = 0 AND aggregate_hosts.host = 'qh2-rcc34';
>> 
>> Which in our case returns 328509 rows in set (25.97 sec)
>> 
>> Seems a bit off considering there are 80 rows in aggregate_hosts, 2
>> rows in aggregates and 2 rows in aggregate_metadata
>> 
>> In the code rows is only equal to 1 so it seems to be doing something
>> inside to code to do this? Don't know too much how sqlalchemy works.
>> 
>> Seems like a bug to me? or maybe our database has something wrong in it?
>> 
>> Cheers,
>> Sam

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] AggregateInstanceExtraSpecs very slow?

[Chris Behrens]

I am not understanding why there are secondary joins defined in the models.  I 
suspect this might break other things, but maybe you can test that this at 
least makes the scheduling faster:

http://paste.openstack.org/show/32534/

That seems to generate a much more acceptable query.

- Chris


On Feb 25, 2013, at 9:40 PM, Sam Morrison  wrote:

> 
> On 26/02/2013, at 4:31 PM, Chris Behrens  wrote:
> 
>> After thinking more, it does seem like we're doing something wrong if the 
>> query itself is returning 300k rows. :)  I can take a better look at it in 
>> front of the computer later if no one beats me to it.
> 
> Yeah I think it's more than a missing index :-)
> 
> The query does 2 INNER JOINS on aggregate_hosts then 2 INNER JOINS on 
> aggregate_metadata then does a further 2 LEFT OUTER JOINS on aggregate_hosts.
> Thanks for the help,
> Sam

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] TC candidacy

[Chris Behrens]

Hi all,

I'd like to announce my candidacy for a seat on the OpenStack
Technical Committee.

- General background -

I have over 15 years of experience designing and building distributed
systems.  I am currently a Senior Software Developer at Rackspace,
where I have been for a 2 and a half years now.  Most of my time
at Rackspace has been spent working on OpenStack as both a developer and
a technical leader.  My first week at Rackspace was spent at the very first
OpenStack Design Summit in Austin where the project was announced.

Prior to working at Rackspace, I held various roles over 14 years
at Concentric Network Corporation/XO Communications including Senior
Software Architect and eventually Director of Engineering.  My main
focus there was on an award winning web/email hosting platform which
we'd built to be extremely scalable and fault tolerant.  While my
name is not on this patent, I was heavily involved with the development
and design that led to US6611861.

- Why am I interested? -

I have strong feelings for OpenStack and I want to help take it to
the next level.  I have a lot of technical knowledge and experience
building scalable distributed systems.

During most of my past experience, I haven't had the luxury of having
access to a lot extremely fast hardware, so it's been important to
make software as performant as possible.  I've also had to put lots of
effort into having 0 downtime, meaning code can be updated seamlessly
without dropping clients.  I've also been one to lead host and software
security efforts so I have a lot of strong feelings in this area.

I am extremely interested in using this experience to make OpenStack
perform well, be secure, be more easily pluggable, and easy to use!

- OpenStack contributions -

As I mentioned above, I was at the very first design summit, so
I've been involved with the project from the beginning.  I started
the initial work for nova-scheduler shortly after the project was
opened.  I also implemented the RPC support for kombu, making sure
to properly support reconnecting and so forth which didn't work
quite so well with the carrot code.  I've contributed a number of
improvements designed to make nova-api more performant.  I've worked on
the filter scheduler as well as designing and implementing the
first version of the Zones replacement that we named 'Cells'.

I'm currently looking forward to restructuring our use of DB API to better
support upgrades w/ schema changes as well as committing an alternative
DB backend implementation for mysql that significantly reduces how long
we block on DB API calls compared to sqlalchemy.

- Summary -

I feel my years of experience contributing to and leading large scale
technical projects along with my knowledge of the OpenStack projects
will provide a good foundation for technical leadership.

Thanks,

- Chris


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Nova] Creating instances with custom UUIDs

[Chris Behrens]

I'm having a hard time understanding the original problem.  nova boot should 
return in milliseconds.  There's no blocking on provisioning.

- Chris

On Apr 3, 2013, at 8:32 PM, Rafael Rosa  wrote:

> API wise I was thinking about something like "nova boot 
> --custom-instance-uuid ABC..." or something like that. To avoid problems with 
> any current implementation I would set it to disabled by default and add a 
> config option to enable it.
> 
> As for collisions, my take is that if you're passing a custom UUID you know 
> what you're doing and is generating them in a way that won't be duplicated. 
> Just by using standard UUID generators the possibility of collisions are 
> really really small.
> 
> Thanks for the feeback :)
> 
> Rafael Rosa Fu
> 
> 
> 2013/4/3 Michael Still 
>> On Thu, Apr 4, 2013 at 9:16 AM, Rafael Rosa  wrote:
>> > Hi,
>> >
>> > In our OpenStack installation we have an issue when creating new instances,
>> > we need to execute some long running processes before calling "nova boot"
>> > and the call blocks for the end user for a while. We would like to return
>> > "immediately" to the caller with a final instance UUID and do the work on
>> > the background, but it's only generated when during actual instance
>> > creation, which is a no go in our situation.
>> 
>> The instance_create database call already accepts an instance UUID as
>> an argument, so that bit looks like it should work out well for you.
>> So, I guess this is mostly a case of working out how you want the API
>> to work.
>> 
>> Personally, I would have no problem with something like this, so long
>> as we could somehow "reserve" the instance UUID so that another caller
>> doesn't try and create an instance with the same UUID while you're
>> doing your slow thing.
>> 
>> Cheers,
>> Michael
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] nova cells minimal setup

[Chris Behrens]

On May 28, 2013, at 8:38 PM, Markus Barth  wrote:

> Hello everyone,
> 
> Nova-cells filter has been merged now, so I'd like to make use of them
> and set up an installation for a proof of concept.
> 
> My goal is to create new filters and then spawn instances in a demo.

Awesome.  I'd be happy to assist you here with your current (and future) 
questions.

> 
> Setup:
> - Global: Horizon, Keystone, Glance
> - API cell: nova-api, nova-conductor, nova-cells, nova-network (flat),
> mysql, rabbitmq
> - child cells: nova-cells, nova-conductor, nova-cert, nova-scheduler,
> nova-compute-qemu; cinder-*, mysql, rabbitmq

There's a number of limitations with cells right now, and the above won't 
*quite* work… but close.  The only issue that I see above is with cinder and 
nova-network.  Further info below.

> 
> 1. Is the above all services I need to spawn instances from Horizon
> without errors?

Generally, yes.. those are the correct services in the correct cells… *except* 
nova-network *might* work, but it would have to be in each child cell and you'd 
need different IP blocks configured in each nova-network in order to work.  
They can all be the same layer 2 network, however.  The limitation is because 
compute needs to talk to nova-network via RPC in order to assign IP addresses…  
and RPC communication between nova-compute and other services is intra-cell 
only.  And if you tried to configure the same /24 in 2 different child cells, 
since they are separate DBs, you'll get the same IPs handed out in multiple 
child cells.  More below under #3.

> 
> 2. Do I need cinder-* (or nova-volume)? Is it really optional now?

No, you do not need them if you use local storage for your VMs.  In fact, 
cinder does not currently work with cells.  I have a patch for this that I need 
to clean up and submit.

> 
> 3. Do I need nova-network (or quantum-*)? I know quantum-* does not have to 
> run when spawning instances, but I guess I need it to create a
> network for a tenant in the first place!?

Quantum is a better choice vs nova-network with or without cells...as 
nova-network will be deprecated some day (AFAIK).  If you choose Quantum, you 
can set this up as a global service to use with cells as long as all of your 
child cells are on the same layer 2 network.  IP address assignments would be 
managed globally, etc.  If you choose quantum and set it up globally, the API 
(nova-api) extensions for networking should work.  In the future, we'll need to 
find a good way to allow Quantum to work with a config such that different 
cells can be on different layer 2 networks.

Hope that helps so far,

- Chris



___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp