[Openstack] [Nova] [Glance] diablo-4 milestone is out
Hello everyone, The last milestone of the Diablo cycle for OpenStack Cloud Compute (Nova) and OpenStack Image Registry and Delivery Service (Glance) was just released. This was a very busy milestone, which feature-wise should be very close to what to expect in the 2011.3 final release. Among other things, Nova gained final integration with Keystone, boot from volumes and a configuration drive feature, while Glance also got integration with keystone, and shared images, notifications... You can see the full list of new features and fixed bugs, as well as tarball downloads, at: https://launchpad.net/nova/diablo/diablo-4 https://launchpad.net/glance/diablo/diablo-4 Packages for Ubuntu are already available for natty/oneiric in the milestone PPAs (lucid and maverick coming up): https://launchpad.net/~nova-core/+archive/milestone https://launchpad.net/~glance-core/+archive/milestone Regards, -- Thierry Carrez (ttx) Release Manager, OpenStack ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Ubuntu run instance error + xen
Hi, Talking about cloud-init and Ubuntu instances... I have a problem related to that part which I can't really reproduce reliably. When spinning up an Ubuntu tty image, I'm often seeing this, during the cloud-init/cloud-setup phase: cloud-setup: cloudinit: getting ssh keys: [0=test] stty: /dev/console System is configured for NO sshd. and then I'm unable to ssh to my instance: I'm getting connection refused, because there's no sshd runnng on port 22. The weird thing is, when I repeat the same Openstack/nova installation procedure a few days later, I would typically not get this issue. Then after another installation attempt yet a few days later the issue would crop up again. Does any recognize this error or know what it means? Maybe cloud-init/setup fails to fetch the keys and as a result won't start up sshd ? The only thing that MIGHT be a factor is that at one point I pulled sources from Launchpad/Bazaar and next time I used the Git mirror. I could try that again and see if that's the culprit. So then it would be down to the particular version of the source code or the build of the tty image, etc. --Leo-- On Aug 25, 2011, at 10:25 PM, Scott Moser wrote: On Fri, 19 Aug 2011, Joshua Harlow wrote: Begin: Running /scripts/init-bottom ... done. [1.348832] EXT4-fs (sda1): re-mounted. Opts: (null) cloud-init start-local running: Fri, 19 Aug 2011 21:34:59 +. up 4.01 seconds no instance data found in start-local init: cloud-init-local main process (186) terminated with status 1 What would be causing that? If anyone knows. Its hung waiting for networking to come up. cloud-init has two init scripts, 'cloud-init local' and 'cloud-init start' (bad name). The second runs only when the networking has come up. That is where you're blocked. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Ubuntu run instance error + xen
On Fri, 26 Aug 2011, Leo van den Bulck wrote: Hi, Talking about cloud-init and Ubuntu instances... I have a problem related to that part which I can't really reproduce reliably. When spinning up an Ubuntu tty image, I'm often seeing this, during the cloud-init/cloud-setup phase: cloud-setup: cloudinit: getting ssh keys: [0=test] stty: /dev/console System is configured for NO sshd. Well, if you can help figure out what is going wrong, then I'll fix that in the images. Just looking through code, I can't see what would be doing it other than you passing a kernel command line option to your instances of 'nosshd' . But surely you'd know that. and then I'm unable to ssh to my instance: I'm getting connection refused, because there's no sshd runnng on port 22. The weird thing is, when I repeat the same Openstack/nova installation procedure a few days later, I would typically not get this issue. Then after another installation attempt yet a few days later the issue would crop up again. Does any recognize this error or know what it means? Maybe cloud-init/setup fails to fetch the keys and as a result won't start up sshd ? There is a bug that I should fix where the image tries to figure out if this is a slow system, and if so, it will not generate ssh keys. clearly you *always* want ssh keys generated if its your only access to the system even if it takes a long time. The only thing that MIGHT be a factor is that at one point I pulled sources from Launchpad/Bazaar and next time I used the Git mirror. I could try that again and see if that's the culprit. So then it would be down to the particular version of the source code or the build of the tty image, etc. I can't imagine that this would be an issue. Could you give a url to exactly what you're using as the ttylinux image? I've recently been working on this some, and I plan to start making those things with more sane version numbers than are present now. the recent snapshots i have at http://smoser.brickies.net/ubuntu/ttylinux-uec/dev/ support running under nova-lxc also. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] API Spec
That analysis omits a very important third party: users of the API. +1 Users of the API are one of the audiences for docs.openstack.org. Another audience is system admins standing up clouds. I'd like to revise the docs landing page to better serve users of all of the OpenStack APIs - Compute, Object Storage, Keystone, Image Service. Input is welcome. I'm tracking moving the Image Service API doc (source and presentation) via a blueprint: https://blueprints.launchpad.net/openstack-manuals/+spec/glance-api-doc. I'm also working on a consistent information architecture for all OpenStack projects, where certain information is known to reside in particular areas. For presentation of information, we have to take a bigger picture view here. Where the source is stored is less important to me, but review is very important. Just wanted to let people know we're not losing sight of users. Anne *Anne Gentle* a...@openstack.org my blog http://justwriteclick.com/ | my bookhttp://xmlpress.net/publications/conversation-community/| LinkedIn http://www.linkedin.com/in/annegentle | Delicioushttp://del.icio.us/annegentle| Twitter http://twitter.com/annegentle ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Git/Gerrit workflow changes for Glance/Keystone
On Mon, Aug 22, 2011, James E. Blair cor...@inaugust.com wrote: If you have any problems or questions, feel free to respond here, talk to mtaylor or jeblair on IRC, file a bug at URL:https://launchpad.net/openstack-ci, or submit a patch to URL:https://github.com/openstack/openstack-ci. In my experience, these changes appear to be a step backwards. I had previously been able to submit a review successfully and it was merged successfully, but now with a new set of changes I get an error. I reviewed the updated documentation and made all of the changes to my local config. However, when I run 'git review', I get this error: johannes@compute1:~/openstack/glance/johannes$ git review Successfully rebased and updated refs/heads/protection. fatal: '/jerdfelt/glance.git': not a Gerrit project fatal: The remote end hung up unexpectedly The first time I ran git review after making the documented changes, it asked me for my launchpad username, but ended up giving me the same error message in the end. Of note, my launchpad username is johannes.erdfelt and my github username is jerdfelt. What does that error message mean? How can we make that error message more useful? JE ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Why are we using github again?
I'm struggling to find a reason why the Openstack project is using github. The use of Gerrit and other tools has reduced github to being just a pretty way to view a git repository. We can't use the github workflow of forking the repository, since that confuses the tools for Gerrit. There's workarounds, but they are awkward. We can't use pull requests since we have Gerrit for that purpose. We can't use bug tracking since we have Launchpad for that purpose. So what value is github bringing to the development process? I'll tell you that it's confusing to say we're using github, except for all of the features it provides beyond a web interface to view the repository. It seems like Openstack could host it's own git repository and avoid a lot of confusion to the people expecting github style workflow. JE ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Why are we using github again?
++ Either we should use github or we shouldn't (until it meets our requirements). On Aug 26, 2011, at 12:29 PM, Johannes Erdfelt johan...@erdfelt.com wrote: I'm struggling to find a reason why the Openstack project is using github. The use of Gerrit and other tools has reduced github to being just a pretty way to view a git repository. We can't use the github workflow of forking the repository, since that confuses the tools for Gerrit. There's workarounds, but they are awkward. We can't use pull requests since we have Gerrit for that purpose. We can't use bug tracking since we have Launchpad for that purpose. So what value is github bringing to the development process? I'll tell you that it's confusing to say we're using github, except for all of the features it provides beyond a web interface to view the repository. It seems like Openstack could host it's own git repository and avoid a lot of confusion to the people expecting github style workflow. JE ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Ubuntu run instance error + xen
Hi, The weird thing is that I seem to be *almost* the only person having this problem - googling around I came across just one other person reporting exactly this same issue. Anyway thanks for the feedback, see some remarks below. On Aug 26, 2011, at 4:37 PM, Scott Moser wrote: On Fri, 26 Aug 2011, Leo van den Bulck wrote: Hi, Talking about cloud-init and Ubuntu instances... I have a problem related to that part which I can't really reproduce reliably. When spinning up an Ubuntu tty image, I'm often seeing this, during the cloud-init/cloud-setup phase: cloud-setup: cloudinit: getting ssh keys: [0=test] stty: /dev/console System is configured for NO sshd. Well, if you can help figure out what is going wrong, then I'll fix that in the images. Just looking through code, I can't see what would be doing it other than you passing a kernel command line option to your instances of 'nosshd' . But surely you'd know that. Yes - that's not the case... and then I'm unable to ssh to my instance: I'm getting connection refused, because there's no sshd runnng on port 22. The weird thing is, when I repeat the same Openstack/nova installation procedure a few days later, I would typically not get this issue. Then after another installation attempt yet a few days later the issue would crop up again. Does any recognize this error or know what it means? Maybe cloud-init/setup fails to fetch the keys and as a result won't start up sshd ? There is a bug that I should fix where the image tries to figure out if this is a slow system, and if so, it will not generate ssh keys. clearly you *always* want ssh keys generated if its your only access to the system even if it takes a long time. So, in theory that might be the explanation then... which of course would raise the question why the system would be 'slow' in one case and not in the other. In both cases it says cloud-setup finished in around 20 to 22 seconds. The only thing that MIGHT be a factor is that at one point I pulled sources from Launchpad/Bazaar and next time I used the Git mirror. I could try that again and see if that's the culprit. So then it would be down to the particular version of the source code or the build of the tty image, etc. I can't imagine that this would be an issue. Could you give a url to exactly what you're using as the ttylinux image? I've used nova.sh to install and run nova - the image downloaded by nova.sh is: http://images.ansolabs.com/tty.tgz On three successive installations with this same nova.sh script (and hence with the same image) i've seen different results: (1) everything works, (2) 'no sshd' issue, (3) everything works again. Same story with the images under http://smoser.brickies.net/ubuntu/ttylinux-uec - so, I get the impression that the issue isn't with the images per se but it's something outside of it - something to do with cloud-init/setup which queries a meta data service (mapped to the nova API service) ? I've recently been working on this some, and I plan to start making those things with more sane version numbers than are present now. the recent snapshots i have at http://smoser.brickies.net/ubuntu/ttylinux-uec/dev/ support running under nova-lxc also. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] New nova service proposal
Sorry I haven't come up with a snazzy name for it yet, but what I have in mind is a new service that is essential for my employer (Rackspace), and might be important for other OpenStack deployments. This new service would be completely optional, of course - only those for whom it is relevant would run it. Let me start by stating the problem: when a customer requests that we create instances for them, nova casts those requests into the queue, where they are eventually acted upon. That usually works great, but in cases where the instance creation fails, we need to detect that failure and re-issue the create request with a different host. This is currently not possible with the asynchronous design of the compute-scheduler interactions. So what I envision is a service that scans a list of recent requests' reservation IDs, and follows up to see if the request was successful or not, and takes action if needed. The blueprint for this can be found at https://blueprints.launchpad.net/nova/+spec/instance-creation-assurance, with an Etherpad created for ongoing idea exchange at http://etherpad.openstack.org/instance-creation-assurance -- Ed Leafe ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] API Spec
+1 On 8/26/11 1:19 PM, Devin Carlen devin.car...@gmail.com wrote: Hey all, I've been following the code vs architect debate that's been unfolding over the past week or so. Here are some of the problems I've seen from my point of view. Fundamentally, the process we have now for defining API specs is broken. I don't believe that this can be argued. The first major misstep in my opinion was forcing backwards compatibility with the Rackspace API in the OpenStack API. I believe we should have had a Rackspace API module just like we have an EC2 API module. Then the OpenStack API wouldn't have been burdened by the historical decisions around the Rackspace API. But that is ancient history at this point. But we have to look at this pragmatically, and realize that 1 year later, the OpenStack API (as spec'd) is still not even close to exposing the underlying core functionality that exists within Nova. For the most part, the OpenStack API is a subset of functionality of the EC2 API. This is a big reason why the Dashboard project used the EC2 API for its underlying communication for so long. There have been a lot of efforts lately to bring the feature set of the OpenStack API in line with the EC2 API, and this is admirable. But this has NOT been happening at the architect level. This has been happening at the developer level, and it is being done with API extensions which make it sound like the feature is somehow not complete or not supported fully, because it's not part of the core API. So the question to all in favor of architecting up front: How do you explain lacking feature parity with the underlying components for over a year now? In my opinion, this has been a big problem in gaining traction around the OpenStack API. Devin ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp This email may include confidential information. If you received it in error, please delete it. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] API Spec
2011/8/25 Jorge Williams jorge.willi...@rackspace.com: On Aug 24, 2011, at 12:02 PM, Soren Hansen wrote: 2011/8/24 Jorge Williams jorge.willi...@rackspace.com: Okay, I do remember these, but the talks focus on documenting what the implementation does. I'm all for it, docstrings are good, but that's not what I usually think of a specification. So we are having a semantic disconnect here Indeed. docstrings can have any format, language, verbosity. It's a string. You can put as much or as little stuff in them as you please (if you have one at all!). Anything you can possibly think of putting in what you refer to as an API spec, you can put in a docstring. OpenStack was created to stop every company wanting to do offer cloud computing from reinventing everything and have everyone work together. What's the use of competing implementations? We'll be right back to square one. Because whether you like it or not there are competing vendors... I don't think making our lives more difficult to make our competitor's easier is a way a strategy we'll get very far with, to be honest. -- Soren Hansen | http://linux2go.dk/ Ubuntu Developer | http://www.ubuntu.com/ OpenStack Developer | http://www.openstack.org/ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] API Spec
++ -Dolph On 08/26/2011 02:46 PM, Mark Collier wrote: +1 On 8/26/11 1:19 PM, Devin Carlendevin.car...@gmail.com wrote: Hey all, I've been following the code vs architect debate that's been unfolding over the past week or so. Here are some of the problems I've seen from my point of view. Fundamentally, the process we have now for defining API specs is broken. I don't believe that this can be argued. The first major misstep in my opinion was forcing backwards compatibility with the Rackspace API in the OpenStack API. I believe we should have had a Rackspace API module just like we have an EC2 API module. Then the OpenStack API wouldn't have been burdened by the historical decisions around the Rackspace API. But that is ancient history at this point. But we have to look at this pragmatically, and realize that 1 year later, the OpenStack API (as spec'd) is still not even close to exposing the underlying core functionality that exists within Nova. For the most part, the OpenStack API is a subset of functionality of the EC2 API. This is a big reason why the Dashboard project used the EC2 API for its underlying communication for so long. There have been a lot of efforts lately to bring the feature set of the OpenStack API in line with the EC2 API, and this is admirable. But this has NOT been happening at the architect level. This has been happening at the developer level, and it is being done with API extensions which make it sound like the feature is somehow not complete or not supported fully, because it's not part of the core API. So the question to all in favor of architecting up front: How do you explain lacking feature parity with the underlying components for over a year now? In my opinion, this has been a big problem in gaining traction around the OpenStack API. Devin ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp This email may include confidential information. If you received it in error, please delete it. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp This email may include confidential information. If you received it in error, please delete it. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] New nova service proposal
I was wondering if some of this could be solved by simply using rpc.call vs rpc.cast so that we get appropriate responses, even if they are exceptions. - Chris On Aug 26, 2011, at 2:23 PM, Brian Lamar wrote: Hey Ed, I absolutely agree that we need to be confident that all requests will be handled by the system eventually. However, I'm unsure of the need for a new service to be created to handle error cases. I'm not saying that we can solve every case through our current architecture, but with some subtle tweaks I think it can be made much more reliable. I feel like if we look at every place where errors *can* occur we can find solutions not involving an external service. Are there any particular bugs/situations that are happening which aren't listed as bugs in Launchpad? Long story short I'd rather not create an external service which attempts to clean up after poor exception handling / bad logic, but there might be some cases I'm not considering. Brian -Original Message- From: Ed Leafe e...@leafe.com Sent: Friday, August 26, 2011 3:22pm To: openstack@lists.launchpad.net Subject: [Openstack] New nova service proposal Sorry I haven't come up with a snazzy name for it yet, but what I have in mind is a new service that is essential for my employer (Rackspace), and might be important for other OpenStack deployments. This new service would be completely optional, of course - only those for whom it is relevant would run it. Let me start by stating the problem: when a customer requests that we create instances for them, nova casts those requests into the queue, where they are eventually acted upon. That usually works great, but in cases where the instance creation fails, we need to detect that failure and re-issue the create request with a different host. This is currently not possible with the asynchronous design of the compute-scheduler interactions. So what I envision is a service that scans a list of recent requests' reservation IDs, and follows up to see if the request was successful or not, and takes action if needed. The blueprint for this can be found at https://blueprints.launchpad.net/nova/+spec/instance-creation-assurance, with an Etherpad created for ongoing idea exchange at http://etherpad.openstack.org/instance-creation-assurance -- Ed Leafe ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp This email may include confidential information. If you received it in error, please delete it. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp This email may include confidential information. If you received it in error, please delete it. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] New nova service proposal
Hey Ed, Great idea. I think there are a lot of ways we can go with this. I started working on a branch called tasks that did something like this. It essentially allowed you to put every action into a tasks database and you could re-run tasks that didn't complete properly. It was based on the idea that every action should be composed of a bunch of idempotent chunks. I have since discovered that a lot of what I had done is very similar to the concept of a task in celery. In general i think that actions should become first class citizens. Action requests to the api should get back a UUID representing the task (or reservation if you prefer), and we should use the task object to keep track of the status of the action. Right now status of actions are stored in status fields in the objects that are acted on. This makes for difficult logic when tasks involve multiple objects (boot from volume for example). If we have rich information in a task object, a recovery service could be written to retry or fix failing tasks. Regardless of the approach, this seems like a great topic for the summit. So coming up with some proposals would be awesome. Vish On Aug 26, 2011, at 12:22 PM, Ed Leafe wrote: Sorry I haven't come up with a snazzy name for it yet, but what I have in mind is a new service that is essential for my employer (Rackspace), and might be important for other OpenStack deployments. This new service would be completely optional, of course - only those for whom it is relevant would run it. Let me start by stating the problem: when a customer requests that we create instances for them, nova casts those requests into the queue, where they are eventually acted upon. That usually works great, but in cases where the instance creation fails, we need to detect that failure and re-issue the create request with a different host. This is currently not possible with the asynchronous design of the compute-scheduler interactions. So what I envision is a service that scans a list of recent requests' reservation IDs, and follows up to see if the request was successful or not, and takes action if needed. The blueprint for this can be found at https://blueprints.launchpad.net/nova/+spec/instance-creation-assurance, with an Etherpad created for ongoing idea exchange at http://etherpad.openstack.org/instance-creation-assurance -- Ed Leafe ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Quantum Diablo-4 Milestone Release
Hello folks, The Quantum Diablo-4 Milestone has been released. Download details are at: https://launchpad.net/quantum/diablo/diablo-4 Major new features completed during this milestone included: - API extensions framework - Cisco plugin + associated extensions - v1.0 release of API We also made significant steps toward getting Quantum integrated with both Nova and the OpenStack Dashboard project, though more work remains with respect to Nova/Quantum integration. Congrats to the Quantum team! Dan ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] New nova service proposal
On Aug 26, 2011, at 2:22 PM, Ed Leafe wrote: Sorry I haven't come up with a snazzy name for it yet, but what I have in mind is a new service that is essential for my employer (Rackspace), and might be important for other OpenStack deployments. This new service would be completely optional, of course - only those for whom it is relevant would run it. Let me start by stating the problem: when a customer requests that we create instances for them, nova casts those requests into the queue, where they are eventually acted upon. That usually works great, but in cases where the instance creation fails, we need to detect that failure and re-issue the create request with a different host. This is currently not possible with the asynchronous design of the compute-scheduler interactions. So what I envision is a service that scans a list of recent requests' reservation IDs, and follows up to see if the request was successful or not, and takes action if needed. The blueprint for this can be found at https://blueprints.launchpad.net/nova/+spec/instance-creation-assurance, with an Etherpad created for ongoing idea exchange at http://etherpad.openstack.org/instance-creation-assurance Hmmm.. having looked over this, I agree that we need to have a way to retry failed builds, however I do not think that having another service essentially polling the builds to find failures is the right way to go. First off, I think it would be better if whatever had the failure responded by sending a request somewhere (a cast) to say Hey, this bombed. Retry it. I wouldn't try doing calls instead of casts, as some have suggested, for performance reasons. (and I could see deadlocking issues) If we step back and look at this, these requests/orders/whatever you call it amount to multi-step workflows. Even for building a single server you have things like allocate this instance on a hypervisor, Assign IP's Attach these volumes, any of which could fail for some reason. And if they do fail, there may be steps need to back-out already completed work. The proper way, IMHO, for this to work is that a request generates a workorder with a set of tasks. This gets sent to something (scheduler, probably) which looks at the first uncompleted task on the workorder, makes the decision on where to send it, and routes the whole workorder there. The service that gets it performs the task (i.e. executes the method), possibly attaching info (like id of newly created instance) to the workorder, and possibly pushing an 'undo' task to the top of a list of tasks to perform if things fail somewhere. Then the whole workorder gets sent back to the origin (again, scheduler?) This looks at the next uncompleted task, and starts the cycle again. Repeat until done. If there is a failure, the scheduler works through the 'undo' list on the workorder, and then makes whatever decisions are needed to redo the tasks elsewhere. The workorder contains the record of the failed attempt, so it doesn't, for example, try to send the server build back to the same hosts that just failed. The workorder acts as an environment for the tasks, and could be passed to tasks (rpc methods) as an attribute of the context object. Anyway, that is my notion. Flame away. -- Monsyne M. Dragon OpenStack/Nova cell 210-441-0965 work x 5014190 This email may include confidential information. If you received it in error, please delete it. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] New nova service proposal
On Aug 26, 2011, at 6:10 PM, Monsyne Dragon wrote: The proper way, IMHO, for this to work is that a request generates a workorder with a set of tasks. This gets sent to something (scheduler, probably) which looks at the first uncompleted task on the workorder, makes the decision on where to send it, and routes the whole workorder there. The service that gets it performs the task (i.e. executes the method), possibly attaching info (like id of newly created instance) to the workorder, and possibly pushing an 'undo' task to the top of a list of tasks to perform if things fail somewhere. Then the whole workorder gets sent back to the origin (again, scheduler?) This looks at the next uncompleted task, and starts the cycle again. Repeat until done. If there is a failure, the scheduler works through the 'undo' list on the workorder, and then makes whatever decisions are needed to redo the tasks elsewhere. The workorder contains the record of the failed attempt, so it doesn't, for example, try to send the server build back to the same hosts that just failed. The workorder acts as an environment for the tasks, and could be passed to tasks (rpc methods) as an attribute of the context object. This sounds very similar to Google's app engine pipeline project (http://code.google.com/p/appengine-pipeline/). ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Why are we using github again?
How is this: http://wiki.openstack.org/GerritWorkflow Better than this: http://wiki.openstack.org/LifeWithBzrAndLaunchpad At the last summit, we said it wasn't a bzr vs git issue but, rather, a workflow issue. People hated LP and loved github. But now it seems we're only using github for git. An open source project lives and dies by its contributors. I think this is like putting up a large stone wall with a sign on it that says Stay out. At least with LP, it was easy enough that people could dip their toe in. $0.02 -S From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of Josh Kearney [j...@jk0.org] Sent: Friday, August 26, 2011 2:51 PM To: Johannes Erdfelt Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] Why are we using github again? ++ Either we should use github or we shouldn't (until it meets our requirements). On Aug 26, 2011, at 12:29 PM, Johannes Erdfelt johan...@erdfelt.com wrote: I'm struggling to find a reason why the Openstack project is using github. The use of Gerrit and other tools has reduced github to being just a pretty way to view a git repository. We can't use the github workflow of forking the repository, since that confuses the tools for Gerrit. There's workarounds, but they are awkward. We can't use pull requests since we have Gerrit for that purpose. We can't use bug tracking since we have Launchpad for that purpose. So what value is github bringing to the development process? I'll tell you that it's confusing to say we're using github, except for all of the features it provides beyond a web interface to view the repository. It seems like Openstack could host it's own git repository and avoid a lot of confusion to the people expecting github style workflow. JE ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp This email may include confidential information. If you received it in error, please delete it. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Why are we using github again?
I disagree. I find lots of valuable in github even if trunk merges require gerrit. Teams can (and IMHO should) use pull requests to improve the quality before it is proposed to trunk. Pull requests to branches can be made while the work is still in progress (sending patches to others branches is something that can be done in BZR, but seems to be rarely done) Jesse On Aug 26, 2011, at 5:17 PM, Sandy Walsh wrote: How is this: http://wiki.openstack.org/GerritWorkflow Better than this: http://wiki.openstack.org/LifeWithBzrAndLaunchpad At the last summit, we said it wasn't a bzr vs git issue but, rather, a workflow issue. People hated LP and loved github. But now it seems we're only using github for git. An open source project lives and dies by its contributors. I think this is like putting up a large stone wall with a sign on it that says Stay out. At least with LP, it was easy enough that people could dip their toe in. $0.02 -S From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of Josh Kearney [j...@jk0.org] Sent: Friday, August 26, 2011 2:51 PM To: Johannes Erdfelt Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] Why are we using github again? ++ Either we should use github or we shouldn't (until it meets our requirements). On Aug 26, 2011, at 12:29 PM, Johannes Erdfelt johan...@erdfelt.com wrote: I'm struggling to find a reason why the Openstack project is using github. The use of Gerrit and other tools has reduced github to being just a pretty way to view a git repository. We can't use the github workflow of forking the repository, since that confuses the tools for Gerrit. There's workarounds, but they are awkward. We can't use pull requests since we have Gerrit for that purpose. We can't use bug tracking since we have Launchpad for that purpose. So what value is github bringing to the development process? I'll tell you that it's confusing to say we're using github, except for all of the features it provides beyond a web interface to view the repository. It seems like Openstack could host it's own git repository and avoid a lot of confusion to the people expecting github style workflow. JE ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp This email may include confidential information. If you received it in error, please delete it. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] API Spec
You couldn't be more correct. The words I would use to describe this scenario are: unacceptable and inexcusable. On Aug 26, 2011, at 7:19 PM, Devin Carlen wrote: Hey all, I've been following the code vs architect debate that's been unfolding over the past week or so. Here are some of the problems I've seen from my point of view. Fundamentally, the process we have now for defining API specs is broken. I don't believe that this can be argued. The first major misstep in my opinion was forcing backwards compatibility with the Rackspace API in the OpenStack API. I believe we should have had a Rackspace API module just like we have an EC2 API module. Then the OpenStack API wouldn't have been burdened by the historical decisions around the Rackspace API. But that is ancient history at this point. But we have to look at this pragmatically, and realize that 1 year later, the OpenStack API (as spec'd) is still not even close to exposing the underlying core functionality that exists within Nova. For the most part, the OpenStack API is a subset of functionality of the EC2 API. This is a big reason why the Dashboard project used the EC2 API for its underlying communication for so long. There have been a lot of efforts lately to bring the feature set of the OpenStack API in line with the EC2 API, and this is admirable. But this has NOT been happening at the architect level. This has been happening at the developer level, and it is being done with API extensions which make it sound like the feature is somehow not complete or not supported fully, because it's not part of the core API. So the question to all in favor of architecting up front: How do you explain lacking feature parity with the underlying components for over a year now? In my opinion, this has been a big problem in gaining traction around the OpenStack API. Devin ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- George Reese - Chief Technology Officer, enStratus e: george.re...@enstratus.comt: @GeorgeReesep: +1.207.956.0217f: +1.612.338.5041 enStratus: Governance for Public, Private, and Hybrid Clouds - @enStratus - http://www.enstratus.com To schedule a meeting with me: http://tungle.me/GeorgeReese smime.p7s Description: S/MIME cryptographic signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] API Spec
Devin, Thank you for putting this so eloquently. I cannot agree more +1000 On Aug 26, 2011 11:38 AM, Devin Carlen devin.car...@gmail.com wrote: Hey all, I've been following the code vs architect debate that's been unfolding over the past week or so. Here are some of the problems I've seen from my point of view. Fundamentally, the process we have now for defining API specs is broken. I don't believe that this can be argued. The first major misstep in my opinion was forcing backwards compatibility with the Rackspace API in the OpenStack API. I believe we should have had a Rackspace API module just like we have an EC2 API module. Then the OpenStack API wouldn't have been burdened by the historical decisions around the Rackspace API. But that is ancient history at this point. But we have to look at this pragmatically, and realize that 1 year later, the OpenStack API (as spec'd) is still not even close to exposing the underlying core functionality that exists within Nova. For the most part, the OpenStack API is a subset of functionality of the EC2 API. This is a big reason why the Dashboard project used the EC2 API for its underlying communication for so long. There have been a lot of efforts lately to bring the feature set of the OpenStack API in line with the EC2 API, and this is admirable. But this has NOT been happening at the architect level. This has been happening at the developer level, and it is being done with API extensions which make it sound like the feature is somehow not complete or not supported fully, because it's not part of the core API. So the question to all in favor of architecting up front: How do you explain lacking feature parity with the underlying components for over a year now? In my opinion, this has been a big problem in gaining traction around the OpenStack API. Devin ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Why are we using github again?
A man with one watch knows what time it is. A man with two is never sure. - Some guy From: Jesse Andrews [anotherje...@gmail.com] Sent: Friday, August 26, 2011 9:54 PM To: Sandy Walsh Cc: Josh Kearney; Johannes Erdfelt; openstack@lists.launchpad.net Subject: Re: [Openstack] Why are we using github again? I disagree. I find lots of valuable in github even if trunk merges require gerrit. Teams can (and IMHO should) use pull requests to improve the quality before it is proposed to trunk. Pull requests to branches can be made while the work is still in progress (sending patches to others branches is something that can be done in BZR, but seems to be rarely done) Jesse On Aug 26, 2011, at 5:17 PM, Sandy Walsh wrote: How is this: http://wiki.openstack.org/GerritWorkflow Better than this: http://wiki.openstack.org/LifeWithBzrAndLaunchpad At the last summit, we said it wasn't a bzr vs git issue but, rather, a workflow issue. People hated LP and loved github. But now it seems we're only using github for git. An open source project lives and dies by its contributors. I think this is like putting up a large stone wall with a sign on it that says Stay out. At least with LP, it was easy enough that people could dip their toe in. $0.02 -S From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of Josh Kearney [j...@jk0.org] Sent: Friday, August 26, 2011 2:51 PM To: Johannes Erdfelt Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] Why are we using github again? ++ Either we should use github or we shouldn't (until it meets our requirements). On Aug 26, 2011, at 12:29 PM, Johannes Erdfelt johan...@erdfelt.com wrote: I'm struggling to find a reason why the Openstack project is using github. The use of Gerrit and other tools has reduced github to being just a pretty way to view a git repository. We can't use the github workflow of forking the repository, since that confuses the tools for Gerrit. There's workarounds, but they are awkward. We can't use pull requests since we have Gerrit for that purpose. We can't use bug tracking since we have Launchpad for that purpose. So what value is github bringing to the development process? I'll tell you that it's confusing to say we're using github, except for all of the features it provides beyond a web interface to view the repository. It seems like Openstack could host it's own git repository and avoid a lot of confusion to the people expecting github style workflow. JE ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp This email may include confidential information. If you received it in error, please delete it. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp This email may include confidential information. If you received it in error, please delete it. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Netstack] Quantum Diablo-4 Milestone Release
Congrats all Lets push forward towards Diablo release Cheers Ram Sent from my iPhone On Aug 26, 2011, at 15:44, Dan Wendlandt d...@nicira.com wrote: Hello folks, The Quantum Diablo-4 Milestone has been released. Download details are at: https://launchpad.net/quantum/diablo/diablo-4 Major new features completed during this milestone included: - API extensions framework - Cisco plugin + associated extensions - v1.0 release of API We also made significant steps toward getting Quantum integrated with both Nova and the OpenStack Dashboard project, though more work remains with respect to Nova/Quantum integration. Congrats to the Quantum team! Dan -- Mailing list: https://launchpad.net/~netstack Post to : netst...@lists.launchpad.net Unsubscribe : https://launchpad.net/~netstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
