[Openstack] OpenStack API, Reservation ID's and Num Instances ...

2011-05-23 Thread Sandy Walsh
Hi everyone, We're deep into the Zone / Distributed Scheduler merges and stumbling onto an interesting problem. EC2 API has two important concepts that I don't see in OS API (1.0 or 1.1): - Reservation ID - Number of Instances to create Typical use case: "Create 1000 instances". The API allocat

Re: [Openstack] OpenStack API, Reservation ID's and Num Instances ...

2011-05-23 Thread Mark Washenberger
Sandy, If I understand the features correctly, their implementation in nova seems straightforward. However, I am still a little curious about their necessity. For load balancing, what is the difference between a single request for N instances and N requests for a single instance each? "Sandy W

Re: [Openstack] OpenStack API, Reservation ID's and Num Instances ...

2011-05-23 Thread Jorge Williams
Hi Sandy, My understanding (Correct me if i'm wrong here guys) is that creating multiple instances with a single call is not in scope for the 1.1 API. Same thing for changing the way in which flavors work. Both features can be brought in as extensions though. I should note that when creating

Re: [Openstack] OpenStack API, Reservation ID's and Num Instances ...

2011-05-23 Thread Jay Pipes
Hi Jorge! Comments inline :) On Mon, May 23, 2011 at 9:42 AM, Jorge Williams wrote: > Hi Sandy, > My understanding (Correct me if i'm wrong here guys) is that creating > multiple instances with a single call is not in scope for the 1.1 API. Actually, I don't think we *could* do this without issu

Re: [Openstack] OpenStack API, Reservation ID's and Num Instances ...

2011-05-23 Thread Jorge Williams
Comments inline: On May 23, 2011, at 8:59 AM, Jay Pipes wrote: > Hi Jorge! Comments inline :) > > On Mon, May 23, 2011 at 9:42 AM, Jorge Williams > wrote: >> Hi Sandy, >> My understanding (Correct me if i'm wrong here guys) is that creating >> multiple instances with a single call is not in sco

Re: [Openstack] OpenStack API, Reservation ID's and Num Instances ...

2011-05-23 Thread Soren Hansen
2011/5/23 Mark Washenberger : > If I understand the features correctly, their implementation in nova seems > straightforward. However, I am still a little curious about their necessity. > For load balancing, what is the difference between a single request for N > instances and N requests for a s

Re: [Openstack] OpenStack API, Reservation ID's and Num Instances ...

2011-05-23 Thread Soren Hansen
2011/5/23 Sandy Walsh : > Additionally, and less important at this stage, users at the summit > expressed an interest in being able to specify instances with something > richer than Flavors. We have some mockups in the current host-filter code > for doing this using a primitive little JSON grammar.

Re: [Openstack] OpenStack API, Reservation ID's and Num Instances ...

2011-05-23 Thread Jay Pipes
/me wishes you were on IRC ;) Discussing this with Mark Wash on IRC... Basically, I'm cool with using a UUID-like pregenerated instance ID and returning that as a reservation ID in the 1.X API. I was really just brainstorming about a future, request-centric 2.0 API that would allow for more atomi

Re: [Openstack] OpenStack API, Reservation ID's and Num Instances ...

2011-05-23 Thread Ed Leafe
On May 23, 2011, at 10:35 AM, Jorge Williams wrote: > If we make the instance ID a unique ID -- which we probably should. Why not > also treat it as a reservation id and generate/assign it up front? Because that precludes the 1:M relationship of a reservation to created instances.

Re: [Openstack] OpenStack API, Reservation ID's and Num Instances ...

2011-05-23 Thread Jorge Williams
On May 23, 2011, at 10:15 AM, Jay Pipes wrote: > /me wishes you were on IRC ;) > > Discussing this with Mark Wash on IRC... > I'll stop by :-) > Basically, I'm cool with using a UUID-like pregenerated instance ID > and returning that as a reservation ID in the 1.X API. Cool. > I was really

Re: [Openstack] OpenStack API, Reservation ID's and Num Instances ...

2011-05-23 Thread Jorge Williams
On May 23, 2011, at 10:15 AM, Ed Leafe wrote: > On May 23, 2011, at 10:35 AM, Jorge Williams wrote: > >> If we make the instance ID a unique ID -- which we probably should. Why >> not also treat it as a reservation id and generate/assign it up front? > > > Because that precludes the 1

Re: [Openstack] OpenStack API, Reservation ID's and Num Instances ...

2011-05-23 Thread Ed Leafe
On May 23, 2011, at 11:41 AM, Jorge Williams wrote: > I don't see how that peculates anything. Treat the instance id as the > reservation id on single instance creations -- have a separate reservation id > when launching multiple instances. End of the day even if you have the > capability to

Re: [Openstack] OpenStack API, Reservation ID's and Num Instances ...

2011-05-23 Thread Sandy Walsh
Cool, I think you all understand the concerns here: 1. We can't treat the InstanceID as a ReservationID since they do two different things. InstanceID's are unique per instance and ReservationID's might span N instances. I don't like the idea of overloading these concepts. How is the caller sup

Re: [Openstack] OpenStack API, Reservation ID's and Num Instances ...

2011-05-23 Thread Jorge Williams
I'd like to step back and denote that there won't be support for this in the 1.1 API -- unless this is an extension. So this is 2.0 API stuff -- right. Other comments inline: On May 23, 2011, at 10:53 AM, Sandy Walsh wrote: Cool, I think you all understand the concerns here: 1. We can't tre

Re: [Openstack] OpenStack API, Reservation ID's and Num Instances ...

2011-05-23 Thread Ed Leafe
On May 23, 2011, at 11:53 AM, Sandy Walsh wrote: > Likewise, we need a way to query the results of a Reservation ID request > without busting GET /servers/detail ... perhaps GET /zones/servers could do > that? GET /servers/reservation/ perhaps? Returns a list of instances similar

Re: [Openstack] OpenStack API, Reservation ID's and Num Instances ...

2011-05-23 Thread Sandy Walsh
From: Jorge Williams > So this is 2.0 API stuff -- right. Well, we need it now ... so we have to find a short term solution. > Why not simply have a request on the server list with the reservation id as a > parameter. > This can easily be supported as an extension. > > So GET /servers/detail?R

Re: [Openstack] OpenStack API, Reservation ID's and Num Instances ...

2011-05-23 Thread Jorge Williams
On May 23, 2011, at 11:25 AM, Sandy Walsh wrote: From: Jorge Williams > So this is 2.0 API stuff -- right. Well, we need it now ... so we have to find a short term solution. > Why not simply have a request on the server list with the reservation id as a > parameter. > This can easily be suppo

Re: [Openstack] OpenStack API, Reservation ID's and Num Instances ...

2011-05-23 Thread Brian Schott
Why does getting the instance id require the API to block? I can create 1 or 1000 UUIDs in order (1) time in the API server and hand back 1000 instance ids in a list of entries in the same amount of time. I'm more concerned about an external user hitting the API server 1000 times to generate

Re: [Openstack] OpenStack API, Reservation ID's and Num Instances ...

2011-05-23 Thread Jay Pipes
On Mon, May 23, 2011 at 12:33 PM, Brian Schott wrote: > Why does getting the instance id require the API to block?  I can create 1 or > 1000 UUIDs in order (1) time in the API server and hand back 1000 instance > ids in a list of entries in the same amount of time. Instance IDs aren't currentl

Re: [Openstack] OpenStack API, Reservation ID's and Num Instances ...

2011-05-23 Thread Mark Washenberger
I'm totally on board with this as a future revision of the OS api. However it sounds like we need some sort of solution for 1.1. > 1. We can't treat the InstanceID as a ReservationID since they do two > different > things. InstanceID's are unique per instance and ReservationID's might span N > i

Re: [Openstack] OpenStack API, Reservation ID's and Num Instances ...

2011-05-23 Thread Vishvananda Ishaya
So I think we've identified the real problem... :) sounds like we really need to do the UUID switchover to optimize here. Vish On May 23, 2011, at 9:42 AM, Jay Pipes wrote: > On Mon, May 23, 2011 at 12:33 PM, Brian Schott > wrote: >> Why does getting the instance id require the API to block?

Re: [Openstack] OpenStack API, Reservation ID's and Num Instances ...

2011-05-23 Thread Sandy Walsh
Changing to UUID is a great thing to do, but not sure if it solves our problem. We still need to differentiate between an Instance ID and a Reservation ID. Additionally, switching to UUID has to be a 2.0 thing, since it's going to bust all backwards compatibility. The ability to cast to int() is

Re: [Openstack] OpenStack API, Reservation ID's and Num Instances ...

2011-05-23 Thread Jorge Williams
+1 On May 23, 2011, at 11:54 AM, Vishvananda Ishaya wrote: > So I think we've identified the real problem... > > :) > > sounds like we really need to do the UUID switchover to optimize here. > > Vish > > On May 23, 2011, at 9:42 AM, Jay Pipes wrote: > >> On Mon, May 23, 2011 at 12:33 PM, Bri

[Openstack] Fwd: Chef Deployment System for Swift - a proposed design - feedback?

2011-05-23 Thread andi abes
It took a while, but finally: https://github.com/dellcloudedge/openstack-swift Jay, I've added a swift-proxy-acct and swift-proxy (without account management). This cookbook is an advanced "leak" of for swift, soon to be followed with a leak of a nova cookbook. The full crowbar that was mentioned

Re: [Openstack] OpenStack API, Reservation ID's and Num Instances ...

2011-05-23 Thread Eric Day
Also keep in mind that UUIDs alone may not be sufficient. As was discussed previously in a marathon ID rename thread, we have to handle the case of federated zones gone bad that could purposefully produce UUIDs that collide. We may want any extra namespace such as "account:uuid" or "zone:uuid", but

Re: [Openstack] Fwd: Chef Deployment System for Swift - a proposed design - feedback?

2011-05-23 Thread James W. Brinkerhoff
Andi, That's great! Taking a look right now -jwb On Mon, May 23, 2011 at 1:41 PM, andi abes wrote: > > > It took a while, but finally: > https://github.com/dellcloudedge/openstack-swift > > Jay, I've added a swift-proxy-acct and swift-proxy (without account > management). > > This cookbook is

Re: [Openstack] [NetStack] Quantum Service API extension proposal

2011-05-23 Thread Alex Neefus
Hi All - I wanted to lend support to this proposal, however I don't think we should be so quick to say this whole thing is an extension. We benefit a lot from having a standard capabilities mechanism as part of our core Quantum API. I like Ying's key value method as well. I think it's lo

Re: [Openstack] [NetStack] Quantum Service API extension proposal

2011-05-23 Thread Debo Dutta (dedutta)
Quick question: it seems we are calling one end of the virtual wire a port and the other a vif. Is there a reason to do that? Can we just call say that that a wire connects 2 ports? Also another interesting network scenario is when there is a wire connecting 2 ports and you have a tap (for all

Re: [Openstack] [NetStack] Quantum Service API extension proposal

2011-05-23 Thread Troy Toman
I think the idea was slightly different. We were equating a vif to NIC in a physical server. A port was equated to a switch port on a physical switch. Doesn't necessarily mean they have to be different. But, there was a reason we used different terminology. In particular, we felt the vif was s

Re: [Openstack] [NetStack] Quantum Service API extension proposal

2011-05-23 Thread Debo Dutta (dedutta)
Hi Troy What about a tap? Its also like a port Should that be in quantum? Regards Debo From: Troy Toman [mailto:troy.to...@rackspace.com] Sent: Monday, May 23, 2011 2:10 PM To: Debo Dutta (dedutta) Cc: Alex Neefus; Ying Liu (yinliu2); Subject: Re: [Openstack] [NetStack] Quantum

Re: [Openstack] OpenStack API, Reservation ID's and Num Instances ...

2011-05-23 Thread Sandy Walsh
Thanks to all for the input. I don't think we've really come to any conclusions for the near term. Unless someone screams, we will be proceeding along the following lines: 1. Adding PUT /zones/server/ to create an instance that will return a Reservation ID (a UUID). It will also accept a num-in

Re: [Openstack] [NetStack] Quantum Service API extension proposal

2011-05-23 Thread kmestery
Debo: A tap is nothing different than a VM vnics from a switch perspective, it still contains a portion owned by Nova (the tap itself) and it connects to a port, which is owned by Quantum. So in essence, the tap still has both a "vif" and a "port" in this terminology. Thanks, Kyle On May 23,

Re: [Openstack] [NetStack] Quantum Service API extension proposal

2011-05-23 Thread Dan Wendlandt
On Mon, May 23, 2011 at 1:05 PM, Alex Neefus wrote: > Hi All – > > > > I wanted to lend support to this proposal, however I don’t think we should > be so quick to say this whole thing is an extension. > Hi Alex, all, I'd like to try and level-set here for a minute, as I don't believe people are

Re: [Openstack] [NetStack] Quantum Service API extension proposal

2011-05-23 Thread Ying Liu (yinliu2)
Thanks, Alex. I'd like to add few points. Our proposal is not intended to provide the whole extension mechanism for Quantum Service. Instead, we are proposing an extensible construct to Quantum core API. Thus, we can have a flexible and scalable way to describe the port's configurations. For

Re: [Openstack] [NetStack] Quantum Service API extension proposal

2011-05-23 Thread Ying Liu (yinliu2)
Thanks, Alex. I'd like to add few points. Our proposal is not intended to provide the whole extension mechanism for Quantum Service. Instead, we are proposing an extensible construct to Quantum core API. Thus, we can have a flexible and scalable way to describe the port's configurations. For

[Openstack] How to limit the total virtual processors/memory for one compute node?

2011-05-23 Thread Huang Zhiteng
Hi all, In my setup of Cactus, I found Nova scheduler would place newly created instance to a compute node that is already full occupied (in terms of memory or # of virtual processors), which lead to swapping and VP overcommitting. That would cause serious performance issue on a busy environment.

Re: [Openstack] [NetStack] Quantum Service API extension proposal

2011-05-23 Thread Ying Liu (yinliu2)
Hi Dan, Totally agree. "Data Extensions" is the way we can extend the list for non-base keys. Actually, we can use this mechanism to extend the extensible construct proposed earlier, assuming that data construct is already in the name space. The extension can be something like this (pd

Re: [Openstack] How to limit the total virtual processors/memory for one compute node?

2011-05-23 Thread Lorin Hochstein
Hi Huang: You can use the simple scheduler, allocates new instances to hosts that have the fewest instances currently running. --scheduler_driver=nova.scheduler.simple.SimpleScheduler More sophisticated schedulers are currently under active development, but they haven't made it to the trunk ye

Re: [Openstack] [NetStack] Quantum Service API extension proposal

2011-05-23 Thread Dan Wendlandt
Hi Ying, Thanks for the detailed example. You are correct, this is inline with what I was thinking. A "data extension" mechanism like this would let any interested party cleanly expose additional properties for API port objects, and as Alex mentioned, potentially for API network objects as well.