Re: [Openstack] Glance, libvirt/kvm snapshots
Ahmed El Gamil wrote: (nova.compute.manager): TRACE: Error: Unexpected error while running command. (nova.compute.manager): TRACE: Command: /usr/bin/curl --fail --silent http://ip address:/_images/2/image -H 'Date: Wed, 16 Feb 2011 14:23:23 GMT' -H 'Authorization: AWS ba97af70-24cf-4054-9185-ecf925d71385:openstack:sIAZBh7GJ7dE8urnZhvwla7upgY=' -o /var/lib/nova/instances/_base/2 (nova.compute.manager): TRACE: Exit code: 22 (nova.compute.manager): TRACE: Stdout: '' (nova.compute.manager): TRACE: Stderr: '' (nova.compute.manager): TRACE: libvir: QEMU error : Domain not found: no domain with matching name 'instance-0007' ttx on IRC told me that it shouldn't fetch a : URL, but that was fixed in a bug at https://bugs.launchpad.net/nova/+bug/708673 Right, that URL (with :/_images) should not be hit if you run with --image_service=nova.image.glance.GlanceImageService, since we have the following code in nova/virt/images.py: def image_url(image): if FLAGS.image_service == nova.image.glance.GlanceImageService: return http://%s:%s/images/%s; % (FLAGS.glance_host, FLAGS.glance_port, image) return http://%s:%s/_images/%s/image; % (FLAGS.s3_host, FLAGS.s3_port, image) So there seems to be a local issue, either you don't run the above code (Bexar release), or the flag is not set properly. To debug this you could instrument the above code to trace the value of FLAGS.image_service. Something like adding before the first if: LOG.debug(FLAGS.image_service is %s % FLAGS.image_service) -- Thierry Carrez (ttx) Release Manager, OpenStack ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Glance, libvirt/kvm snapshots
Hey Ahmed, TL;DR Glance doesn't appear to be supported with libvirt driver yet. Judging by the stack trace, it looks like the code is taking an S3 code path even though you've selected the GlanceImageService. Looking through the code, it doesn't appear that the libvirt/Glance code is there yet (the patch from the 708673https://bugs.launchpad.net/nova/+bug/708673 doesn't appear to be enough). This should be logged as a bug. I don't *think* the fix will be that hard, we just need to modify images.fetch() to respect the image_service and to stream the data using the Glance client rather than curl. Until then, I think the objectstore or local-store will be your best bet. Hope that helps, Rick From: openstack-bounces+rick.harris=rackspace@lists.launchpad.net [openstack-bounces+rick.harris=rackspace@lists.launchpad.net] on behalf of Ahmed El Gamil [ah...@manhag.org] Sent: Thursday, February 17, 2011 1:48 AM To: openstack@lists.launchpad.net Subject: Re: [Openstack] Glance, libvirt/kvm snapshots Hi Guys, So The guys over at IRC convinced me that posting that to the mailing list would be cooler to solve/discuss, so don't let me down :). Basically i'm working on launching an instance via the openstack API, using the Bexar release and the python-novatools (the novatools command), The hypervisor is KVM and Glance is used for Imaging(image_service=nova.image.glance.GlanceImageService). After using the boot subcommand in novatools, the instance is marked as build and i get the following errors in the logs: 2011-02-16 09:23:23,535 ERROR nova.compute.manager [F--YCJOZ14LY9LTFHYH1 c9er openstack] instance 7: Failed to spawn (nova.compute.manager): TRACE: Traceback (most recent call last): (nova.compute.manager): TRACE: File /usr/lib/pymodules/python2.6/nova/compute/manager.py, line 211, in run_instance (nova.compute.manager): TRACE: self.driver.spawn(instance_ref) (nova.compute.manager): TRACE: File /usr/lib/pymodules/python2.6/nova/exception.py, line 122, in _wrap (nova.compute.manager): TRACE: raise Error(str(e)) (nova.compute.manager): TRACE: Error: Unexpected error while running command. (nova.compute.manager): TRACE: Command: /usr/bin/curl --fail --silent http://ip address:/_images/2/image -H 'Date: Wed, 16 Feb 2011 14:23:23 GMT' -H 'Authorization: AWS ba97af70-24cf-4054-9185-ecf925d71385:openstack:sIAZBh7GJ7dE8urnZhvwla7upgY=' -o /var/lib/nova/instances/_base/2 (nova.compute.manager): TRACE: Exit code: 22 (nova.compute.manager): TRACE: Stdout: '' (nova.compute.manager): TRACE: Stderr: '' (nova.compute.manager): TRACE: libvir: QEMU error : Domain not found: no domain with matching name 'instance-0007' ttx on IRC told me that it shouldn't fetch a : URL, but that was fixed in a bug at https://bugs.launchpad.net/nova/+bug/708673 Any hints on what could cause that ? did anybody work with novatools against kvm/glance ? Thanks in advance. Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Review days for nova-core members
Soren Hansen wrote: That's really a corollary to this proposal: Being in nova-core means you have a review day once every N days (where N is the amount of (human) members of nova-core). As such, if you're not prepared to accept such a review day, you don't get to be part of the team. Simple. That would definitely help in getting *all* branches reviewed, and should be sufficient to clear the backlog. If the number of branches pending review go above a certain level, maybe we could switch temporarily to double mode where we get two -core members on review duty every day. For nova-core as an example, we currently have 18 people in the team, of which I'd say 9 people are regular reviewers. If we can clean up and add a few more members (~15) that would mean one review day every three weeks in normal mode and one every 1.5 week in double mode. Sounds like a reasonable commitment. That said that shouldn't prevent anyone from also reviewing stuff that they care about any day. -- Thierry Carrez (ttx) Release Manager, OpenStack ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Review days for nova-core members
On Feb 17, 2011, at 4:18 AM, Soren Hansen wrote: That's really a corollary to this proposal: Being in nova-core means you have a review day once every N days (where N is the amount of (human) members of nova-core). As such, if you're not prepared to accept such a review day, you don't get to be part of the team. Simple. It should be at *least* N/2, since two core dev approvals are needed on every merge prop. And it would be nice to be more than N/2, since additional sets of eyes always catch more potential problems. -- Ed Leafe ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Review days for nova-core members
I'll throw my hat in the ring for core if deemed worthy. Soren's point about 1 review day per # core developers was refreshing. I had previously held back from applying because I was afraid all my time would be tied up with reviews. ? From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of Thierry Carrez [thie...@openstack.org] Sent: Thursday, February 17, 2011 10:26 AM To: openstack@lists.launchpad.net Subject: Re: [Openstack] Review days for nova-core members Soren Hansen wrote: That's really a corollary to this proposal: Being in nova-core means you have a review day once every N days (where N is the amount of (human) members of nova-core). As such, if you're not prepared to accept such a review day, you don't get to be part of the team. Simple. That would definitely help in getting *all* branches reviewed, and should be sufficient to clear the backlog. If the number of branches pending review go above a certain level, maybe we could switch temporarily to double mode where we get two -core members on review duty every day. For nova-core as an example, we currently have 18 people in the team, of which I'd say 9 people are regular reviewers. If we can clean up and add a few more members (~15) that would mean one review day every three weeks in normal mode and one every 1.5 week in double mode. Sounds like a reasonable commitment. That said that shouldn't prevent anyone from also reviewing stuff that they care about any day. -- Thierry Carrez (ttx) Release Manager, OpenStack ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Review days for nova-core members
On Thu, Feb 17, 2011 at 9:51 AM, Ed Leafe e...@leafe.com wrote: On Feb 17, 2011, at 4:18 AM, Soren Hansen wrote: That's really a corollary to this proposal: Being in nova-core means you have a review day once every N days (where N is the amount of (human) members of nova-core). As such, if you're not prepared to accept such a review day, you don't get to be part of the team. Simple. It should be at *least* N/2, since two core dev approvals are needed on every merge prop. And it would be nice to be more than N/2, since additional sets of eyes always catch more potential problems. +20. -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Review days for nova-core members
On Wed, Feb 16, 2011 at 9:53 PM, Paul Voccio paul.voc...@rackspace.com wrote: Not sure what the etiquette is for removing someone. Michael Gundlach is still listed but is no longer participating. I think these sorts of things can be resolved on the mailing list just fine. It's not a big deal for a developer to leave nova-core; after all, we all move on to other things at some point in time and/or need to focus on non-review stuff. Also, another point I failed to make is that membership in nova-core *does not* mean that you must be an expert in Nova's code base or an expert in the underlying technologies in Nova (virtualization, networking, volume management, etc). If that were the case, I for one certainly would not be in nova-core. It's more about a commitment to give contributors the review time and attention they deserve. In reviews, I will be the first to acknowledge that my review will, say, focus on style or high-level design, because I may not have expertise in a certain area like networking. And part of my review will be to search out someone like Vishy who may indeed have that expertise. Just my two cents, jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Review days for nova-core members
Josh Kearney wrote: I'm with Sandy on this. I'd like to step in as well. Could you each start a new thread about your application ? That should make it easier to track the lazy consensus described in: http://wiki.openstack.org/Governance/Approved/CoreDevProcess -- Thierry Carrez (ttx) Release Manager, OpenStack ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Proposal to be a member of Nova Core ...
I'd like to help out on the review process as per http://wiki.openstack.org/Governance/Approved/CoreDevProcess I like quiet walks in the park and black and white movies. -Sandy Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Steps that can help stabilize Nova's trunk
Soren, can you clarify what you mean by Ubuntu being the primary platform? Why is that the reference? What limitations does this introduce? Jim On Feb 17, 2011, at 4:31 AM, Soren Hansen wrote: 2011/2/17 Brian Schott bfsch...@gmail.com: One thing we saw in the list and experienced first hand is that your Hudson server uses a pre-configured environment and ours pulls virtual env every time. We had failures on trunk that yours missed because of new pip pulled versions. A fresh run_tests.sh -f needs to work. It is the only guaranteed sanity test everybody can replicate. That's not technically accurate. Every part of the environment in which we run the tests can be rebuilt in a reproducible manner. Everything on that box is packaged and available in a PPA from which anyone can build an identical test system. It might pull upstream bugs, but better to be ahead of that curve than behind. I agree that the status quo is not good enough. However, I don't agree that we should pull stuff from pip directly. Ubuntu is our primary target platform, that's the reference, that's where we absolutely *must* function. I'd be *delighted* if we tested our stuff more broadly, and I welcome efforts to do so, but whether this works on Ubuntu or not is the deal breaker. That said, our tests right now are run on Maverick with a set of backported packages (all available in a PPA, though), but really ought to run on Natty. We should look into that ASAP. -- Soren Hansen Ubuntu Developerhttp://www.ubuntu.com/ OpenStack Developer http://www.openstack.org/ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Steps that can help stabilize Nova's trunk
On Thu, Feb 17, 2011 at 12:19 PM, Trey Morris trey.mor...@rackspace.com wrote: I don't like that it currently only runs on ubuntu + the ppa. If it doesn't work with existing versions I think we're doing something wrong. Even when natty comes out, I don't like the idea of having to ensure I have latest ubuntu for openstack to run. It runs on the platform that people have spent the time and effort to integrate into Hudson. This happens to be Ubuntu because that is what Soren (the person who has spent all that time and effort) is comfortable with. I encourage you to set up an integration test for the specific platform you want to ensure does not break with commits to trunk :) This could be as simple as doing the following: * Find Jordan Rinke or another Racker who has access to machines that can be linked to Hudson * SSH into the said machine and ensure that the machines have all your environment's necessary components installed. In your case, Trey, I'll presume that you want XenServer installed on the compute nodes and MySQL installed on one of the other machines to act as the main database * Find soren, mtaylor, myself or others on IRC to help install the Jenkins/Hudson agent on the machines. The Hudson agent will be responsible for pulling lp:nova and installing all the necessary pieces on the machines in your test environment * Place a /etc/nova/nova.conf file on the machines in question that matches your target environment * Create a simple functional test script that runs through a basic set of API requests that exercise the parts of the Nova API that are critical to you (XenAPI, Glance integration, etc) * Have Hudson fire said script against the test environment after starting up Nova on the relevant nodes So, we're ready to help. But we need help from yourself and others on your team, as well as good communication with folks like Jordan to make sure we're all on the same page. Together, we can get this done. As far as stability goes, i think integration testing is a great solution. Hudson should run integration tests before it allows code into trunk. I also think that code should be integration tested before it is reviewed since hardware is cheaper than core-devs. I'll review that once I'm sure it works. The only issue I can see with the hudson running integration tests is the way the testing environments are set up. There are so many options an exhaustive list would just take forever. We have to start somewhere, and a good somewhere would be the internal Rackspace Cloud Servers setup, since that's clearly a target platform. ;) Add to that NASA's environment and anyone else's environment who is willing to spend the time and effort to set up the test environments. -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Proposal to be a member of Nova Core
+1 for jk0 Jay Pipes jaypi...@gmail.com wrote: On Thu, Feb 17, 2011 at 1:25 PM, Vishvananda Ishaya vishvana...@gmail.com wrote: +1 Jk0 has been contributing a lot and doing reviews even when they don't count. All reviews count :) -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Review days for nova-core members
2011/2/17 Jay Pipes jaypi...@gmail.com: Also, good point to keep in mind: Membership to nova-core isn't a privilege or even any fun. It's a responsibility and a duty to your fellow contributors :) The first draft of my e-mail said something about it being a chore, but I decided to edit that out to not demotivate people from joining :) -- Soren Hansen Ubuntu Developer http://www.ubuntu.com/ OpenStack Developer http://www.openstack.org/ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Steps that can help stabilize Nova's trunk
2011/2/17 Jim Curry jim.cu...@rackspace.com: Soren, can you clarify what you mean by Ubuntu being the primary platform? Why is that the reference? What limitations does this introduce? It's the primary platform because it's the platform we test everything on, it's the platform we spend time integrating with, etc. It started out as the reference because that's what both the Swift devs as well as the Anso guys had chosen it as their reference platform. It's an excellent choice, so there has been no motivation to change that decision as far as I know. I can think of a long list of consequences of this choice. Attempting to phrase any of them as limitations would be contrived. Having a reference platform means we can reasonably make a lot of very useful assumptions about the environment in which Nova runs. Having that reference platform be Ubuntu means that there's a straightforward way to modify this environment if we have special needs. I've patched a number of packages in Ubuntu already to accomodate our needs, including the Linux kernel, libvirt, Eventlet, user-mode-linux, etc. -- Soren Hansen Ubuntu Developer http://www.ubuntu.com/ OpenStack Developer http://www.openstack.org/ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Proposal to be a member of Nova Core
+1 :-D From: Josh Kearney josh.kear...@rackspace.commailto:josh.kear...@rackspace.com Date: Thu, 17 Feb 2011 10:32:04 -0600 To: openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Subject: [Openstack] Proposal to be a member of Nova Core I'd like to volunteer my time to help out with reviews in Nova by becoming a member of Nova-Core. -Josh Kearney ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Monsyne Dragon requesting Core-Dev status for nova.
I think the best way to get involved is to just start doing reviews. A branch still requires 2 core reviews to be merged, but that doesn't mean other people can't jump in and give their 2 cents in the meantime. So just dive on in and you can start learning about the review process and helping out! Devin On Feb 17, 2011, at 8:25 AM, Monsyne Dragon wrote: Reposting this in a separate thread as requested by Thierry. I'm volunteering to do code reviews/ other core-dev duties, so we can get some of our review logjam through. -- -- -Monsyne Dragon work: 210-312-4190 mobile210-441-0965 google voice: 210-338-0336 Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Steps that can help stabilize Nova's trunk
On Thu, Feb 17, 2011 at 1:58 PM, Trey Morris trey.mor...@rackspace.com wrote: sounds like a good plan to me :) Awesome. I'm glad you're taking the lead on this ;) https://bugs.launchpad.net/nova/+bug/720941 Cheers! jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Monsyne Dragon requesting Core-Dev status for nova.
I would agree with Devin. Monsyne, I'd like to see some of your reviews before I give a thumbs up to nova-core. Please do participate in the review process so we've got a bit more to base a decision on. :) Of course, I may just be missing reviews you have done? If so, please don't hesitate to correct me and point me to them. Cheers! jay On Thu, Feb 17, 2011 at 3:17 PM, Devin Carlen devin.car...@gmail.com wrote: I think the best way to get involved is to just start doing reviews. A branch still requires 2 core reviews to be merged, but that doesn't mean other people can't jump in and give their 2 cents in the meantime. So just dive on in and you can start learning about the review process and helping out! Devin On Feb 17, 2011, at 8:25 AM, Monsyne Dragon wrote: Reposting this in a separate thread as requested by Thierry. I'm volunteering to do code reviews/ other core-dev duties, so we can get some of our review logjam through. -- -- -Monsyne Dragon work: 210-312-4190 mobile 210-441-0965 google voice: 210-338-0336 Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Google Summer of Code 2011
Devs: The time to decide on GSOC is fast approaching. As I indicated earlier this month, we will decide to proceed based on the projects developers would like to submit. As of today, there are no project submissions at the Wiki page at http://wiki.openstack.org/Project%20List. The schedule calls for me to submit our entry to Google between Feb 28 - March 11. If there are no projects listed by March 8th, I will remove the Wiki page and move on to other projects until next year when we can consider GSOC 2012. If there are projects listed by the 8th, I will ask for developer feedback on the projects and either submit the application or skip the process for 2011. Thanks. On Feb 7, 2011, at 10:58 AM, Stephen Spector wrote: My apologies, the link did not copy over correctly - http://wiki.openstack.org/Project%20List Thanks, On Feb 7, 2011, at 10:47 AM, Stephen Spector wrote: Thanks for everyone's feedback to the idea of supporting GSOC 2011. I have received emails from several people interested in becoming a mentor. As a next step, please list yourself on the GSOC 2011 Wiki page at http://wiki.openstack.org/Project List along with your project ideas/descriptions. From here, we as a community can review all the ideas submitted and determine if we have quality projects that make sense for the community. If so, I will go ahead and submit the application. If not, we can postpone until next year. However, there is interest in having OpenStack participate as I already have students who have contacted me wanting to review our proposals and participate. Let's see what ideas we come up with and then make a final decision on go/no-go based on the projects listed. Thanks. Rick Clark wrote: In order to be successful, quite a bit of thought and planning needs to be put into a GSOC project. I've seen other open source projects look bad by not being organized enough. This needs to be way more put together, and thought out before I would want to see openstack's name on it. In my experience, GSoC can easily backfire on unprepared organizations. And I think we are a bit young, as an organization, to successfully participate. The deadlines are just around the corner, not sure rushing our first participation is a good idea. I'd rather discuss of a full plan in one of the 2011 design summits, to have a well-organized effort for GSoC 2012. On Feb 4, 2011, at 2:37 PM, Stephen Spector wrote: OpenStack Developers: Now that you have a few days off (just kidding) to relax from the release of Bexar, I wanted to bring up a new topic for your consideration. The Google Summer of Code (http://code.google.com/soc/) will take applications from open source communities starting Feb 28 - March 11 to participate in the program. OpenStack would submit a list of projects for students to complete with a pre-determined mentor for each project to support the development effort. The project list is published by Google and any student wishing to participate with OpenStack would apply for a specific project. The mentors of each feature would select the student they wish to work with and notify Google. The student and mentor would then begin work on the feature with the expectation that the code can be written, tested, and submitted back to the community by the end of the Summer. The program is global and students and mentors may not necessarily ever meet. Google does pay the students $5,000 USD to participate and we would get $500 toward our OpenStack community. Mentors are not paid but are GREATLY appreciated in their efforts to support the students and community. All funds generated for the community would go directly to an OpenStack Developer event to cover costs. To join, we must have mentors in place who are willing to work with a student that we select in writing the code for a specific feature/portion of OpenStack. These mentors are required to put in more than just the occasional email so there is a commitment on the part of mentors to ensure the student is successful in writing the code and getting it submitted to the community. For more information about being a mentor, http://code.google.com/p/google-summer-of-code/wiki/AdviceforMentors. I believe this Google program is an excellent idea and a great way for OpenStack to get some code written by incredibly smart people who will either end up working for one of the companies involved in the project or simply become an independent developer within OpenStack. I have started the paperwork to submit in late February for OpenStack to participate but will not submit the application unless I have interest and confirmation from developers within OpenStack to be mentors. Even if we only ran 2 students this summer, I believe this would be a great way for OpenStack to reach out to a global community of developers interested in supporting open source
Re: [Openstack] Steps that can help stabilize Nova's trunk
On Wed, Feb 16, 2011 at 11:44 PM, Paul Voccio paul.voc...@rackspace.com wrote: Jay, Thanks for throwing this out. How would we build this with Hudson? What would a standard deploy of Nova even look like for integration tests? I replied with some specifics to Trey, who had a similar question, and created a bug report (that I subsequently assigned to Trey): https://bugs.launchpad.net/nova/+bug/720941 Let me know if that answered your questions and if you'd like some more explanation. We've also bounced the idea within our team of not allowing code commits if the code to test ratio decreases but I'm not sure if that would work for such a large project like this one. This is a good idea, but even if we were to add code to test ratios, the ratio would be mostly (only?) looking at unit tests. And I think we've seen that unit tests pass and functional tests blow up because of all the mocks/stubs in the unit tests that don't adequately represent a real-world deployment. Nothing wrong, of course, with exploring code to test ratio (basically, automated code coverage tests), but I think good functional and integration tests are more productive at this time. Just my two cents, though, nothing more, -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Steps that can help stabilize Nova's trunk
No disagreement with anything you say below, Matt. More testing of all kinds, including unit tests, should be a priority. Cheers, jay On Wed, Feb 16, 2011 at 11:37 PM, Matt Dietz matt.di...@rackspace.com wrote: These are all great points Jay. I'd like to re-echo the comment about unit tests. Obviously they aren't the panacea, but they can protect against some of the dumber errors that have made their way into trunk. One particular bug stopped one developer on my team dead in his tracks, and it ended up being a semi-colon in place of a colon. There's a lot of utility in simply exercising code... I think we may want to consider everyone's favorite topic of code coverage as well in all of this. Specifically, we may want to take note of code coverage on any given merge, and if subsequent merges reduce that number, we throw a fit/reject the patch. I know that won't be a popular solution, but it would definitely put a stop to the lack of unit tests. On 2/16/11 4:27 PM, Jay Pipes jaypi...@gmail.com wrote: Hey all, It's come to my attention that a number of folks are not happy that Nova's trunk branch (lp:nova) is, shall we say, less than stable. :) First, before going into some suggestions on keeping trunk more stable, I'd like to point out that trunk is, by nature, an actively developed source tree. Nobody should have an expectation that they can simply bzr branch lp:nova and everything will magically work with a) their existing installations of software packages, b) whatever code commits they have made locally, or c) whatever specific hypervisor/volume/network environment that they test their local code with. The trunk branch is, after all, in active development. That said, there's *no* reason we can't *improve* the relative stability of the trunk branch to make life less stressful for contributors. Here are a few suggestions on how to keep trunk a bit more stable for those developers who actively develop from trunk. 1) Participate fully in code reviews. If you suspect a proposed branch merge will mess everything up for you, then you should notify reviewers and developers about your concerns. Be proactive. 2) If you pull trunk and something breaks, don't just complain about it. Log a bug immediately and talk to the reviewers/approvers of the patch that broke your environment. Be constructive in your criticism, and be clear about why the patch should have been more thoroughly or carefully reviewed. If you don't, we're bound to repeat mistakes. 3) Help us to write functional and integration tests. It's become increasingly clear from the frequency of breakages in trunk (and other branches) that our unit tests are nowhere near sufficient to catch a large portion of bugs. This is to be expected. Our unit tests use mocks and stubs for virtually everything, and they only really test code interfaces, and they don't even test that very well. We're working on adding functional tests to Hudson that will run, as the unit test do, before any merge into trunk, with any failure resulting in a failed merge. However, we need your help to create functional tests and integration tests (tests that various *real* components work together properly). We also need help writing test cases that ensure software library dependencies and other packaging issues are handled properly and don't break with minor patches. 4) If you have a specific environment/setup that you use (Rackers and Anso guys, here...), then we need your assistance to set up test clusters that will pull trunk into a wiped test environment and ensure that a series of realistic calls to the Nova API are properly handled. I know some of you are working on getting hardware ready. We need help from the software teams to ensure that these environments are initialized with the exact setups you use. The more testing we fire off against each potential merge into trunk, and the more those tests are hitting real-life deployment environments, the more stable trunk will become and the easier your life as a contributor will be. Thanks in advance for your assistance, and please don't hesitate to expand on any more suggestions you might have to stabilize trunk. -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is
Re: [Openstack] Steps that can help stabilize Nova's trunk
Great thoughts Jay - I think a push to improve test coverage is a great goal for Cactus. It seems like we are getting new contributors faster than ever these days. I would like to suggest that we create a blueprint called something like improve test coverage and create a number of bug reports that reference the blueprint. Then label them as low hanging fruit, which will encourage everyone (especially new contributors) to look at them. As we all know, the best way to learn a codebase is to write tests for it. On Feb 17, 2011, at 12:57 PM, Jay Pipes wrote: On Wed, Feb 16, 2011 at 11:44 PM, Paul Voccio paul.voc...@rackspace.com wrote: Jay, Thanks for throwing this out. How would we build this with Hudson? What would a standard deploy of Nova even look like for integration tests? I replied with some specifics to Trey, who had a similar question, and created a bug report (that I subsequently assigned to Trey): https://bugs.launchpad.net/nova/+bug/720941 Let me know if that answered your questions and if you'd like some more explanation. We've also bounced the idea within our team of not allowing code commits if the code to test ratio decreases but I'm not sure if that would work for such a large project like this one. This is a good idea, but even if we were to add code to test ratios, the ratio would be mostly (only?) looking at unit tests. And I think we've seen that unit tests pass and functional tests blow up because of all the mocks/stubs in the unit tests that don't adequately represent a real-world deployment. Nothing wrong, of course, with exploring code to test ratio (basically, automated code coverage tests), but I think good functional and integration tests are more productive at this time. Just my two cents, though, nothing more, -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Steps that can help stabilize Nova's trunk
+1, this is a great suggestion. On Feb 17, 2011, at 1:06 PM, Jay Pipes wrote: Although Soren adequately explained why he thinks that running run_tests.sh on Hudson should *not* be against a pip/virtualenv setup, I would like to state that I think it would be a good idea to have Hudson do *both*. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Proposal to be a member of Nova Core ...
+1! (Yup, that was +factorial(1), for those keeping score at home) 2011/2/17 Sandy Walsh sandy.wa...@rackspace.com: I'd like to help out on the review process as per http://wiki.openstack.org/Governance/Approved/CoreDevProcess I like quiet walks in the park and black and white movies. -Sandy Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- Soren Hansen Ubuntu Developer http://www.ubuntu.com/ OpenStack Developer http://www.openstack.org/ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Steps that can help stabilize Nova's trunk
On Feb 17, 2011, at 4:21 PM, Soren Hansen wrote: I understand the motivation, I'm just not sure I want the latter to actually block a merge. As an example, the recent race condition I spotted and fixed required a patch to land in eventlet. If the latter was allowed to block a merge, we'd have to keep a known race condition in Nova until upstream decides to do a release so that pip could fetch it. That could take a *long* time. In the mean time, I stuck a fixed Eventlet package in our PPA (and in Ubuntu Natty proper) so that we could move on with our lives and get rid of the race condition. There's simply a flexibility in this approach that I don't see how we can obtain with the pip approach. For those who deploy on platforms other than Ubuntu, are these customizations recorded somewhere in the OpenStack documentation? If I wanted to deploy to SUSE, it would help to know that, say, I can use the stock version of greenlet 0.3.1, but I need to use a customized version of eventlet 0.9.12 with the patches that correspond to python-eventlet in the nova-core/release PPA. Having access to a comprehensive set of the dependency packages versions for the reference implementation platform would be very helpful for debugging problems on other platforms. Is it possible automatically pull this info from the PPA, extract it from the package metadata, and then add it to the admin guide docs? Lorin -- Lorin Hochstein, Computer Scientist USC Information Sciences Institute 703.812.3710 http://www.east.isi.edu/~lorin smime.p7s Description: S/MIME cryptographic signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Steps that can help stabilize Nova's trunk
Got it. But the primary tradeoff is simply that we need to make sure any changes we make to another platforms don't break the Ubuntu integration? And in general that should not be a major issue...? Jim On Feb 17, 2011, at 1:32 PM, Soren Hansen wrote: 2011/2/17 Jim Curry jim.cu...@rackspace.com: Soren, can you clarify what you mean by Ubuntu being the primary platform? Why is that the reference? What limitations does this introduce? It's the primary platform because it's the platform we test everything on, it's the platform we spend time integrating with, etc. It started out as the reference because that's what both the Swift devs as well as the Anso guys had chosen it as their reference platform. It's an excellent choice, so there has been no motivation to change that decision as far as I know. I can think of a long list of consequences of this choice. Attempting to phrase any of them as limitations would be contrived. Having a reference platform means we can reasonably make a lot of very useful assumptions about the environment in which Nova runs. Having that reference platform be Ubuntu means that there's a straightforward way to modify this environment if we have special needs. I've patched a number of packages in Ubuntu already to accomodate our needs, including the Linux kernel, libvirt, Eventlet, user-mode-linux, etc. -- Soren Hansen Ubuntu Developerhttp://www.ubuntu.com/ OpenStack Developer http://www.openstack.org/ Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Queue Service, next steps
Thanks to everyone who gave great feedback on the first queue service thread. I've updated the wiki page to include the suggestions. http://wiki.openstack.org/QueueService With a decent vision of what we want to build, the next step is figuring out how. In a previous thread it was suggested that the preferred languages for OpenStack projects are Python, C, and C++. Since there is an emphasis on speed and efficiency for the queue service, I suggest we use C++. I expect this service to be CPU bound and would benefit being able to leverage multiple cores efficiently (within the same process), so I don't think Python is a good fit. I think C++ is a better fit than C due to the need for modular interfaces. While this can obviously be done in C, C++ APIs are more concise and much less error prone. The OO style will also make it easier for Python developers who also want to learn and assist with C++ projects. Erlang is not on the preferred lists, but I would also put it out there as an option. While it may be a great fit for a project like this, I worry it won't attract the developer resources since Erlang isn't really a first-class language yet. If we decide to take the C++ path, I propose using a modular application framework I've been working on over the past year (mostly in my spare time). It provides a simple module programming interface with dependency tracking (kind of like Linux kernel modules). It already provides a multi-threaded event module (currently based on libevent, but this is pluggable) with simple networking abstractions built on top of it. We should be able to dive in an start writing the HTTP protocol module and queue processing modules. You can check out the current project at: https://launchpad.net/scalestack http://scalestack.org/ The intention of using a framework like this is so we can easily reuse the other modules (auth, HTTP, logging, ...) for other OpenStack services in the future. Much like we use Eventlet, WSGI, etc. (and eventually openstack-common) for Python, we could prefer using the modules in Scale Stack for lower level projects. Thoughts? -Eric ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Steps that can help stabilize Nova's trunk
By defining an unbreakable reference platform, are we necessarily limiting its ability to integrate on other platforms? That is my underlying question. I understand the need for a reference platform but am trying to understand to what extent that results in us not being able to easily support other platforms (or not). Jim On Feb 17, 2011, at 4:46 PM, Soren Hansen wrote: 2011/2/17 Jim Curry jim.cu...@rackspace.com: Got it. But the primary tradeoff is simply that we need to make sure any changes we make to another platforms don't break the Ubuntu integration? And in general that should not be a major issue...? I don't think I understand the question, I'm afraid. :( From where I see it, there's no trade-off involved. We've decided to define a reference platform. Support for other platforms is very welcome indeed, but there's just the one blessed platform, where we can't break. It's the platform we test everything on. -- Soren Hansen Ubuntu Developerhttp://www.ubuntu.com/ OpenStack Developer http://www.openstack.org/ Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Steps that can help stabilize Nova's trunk
2011/2/17 Jim Curry jim.cu...@rackspace.com: By defining an unbreakable reference platform, are we necessarily limiting its ability to integrate on other platforms? That is my underlying question. I understand the need for a reference platform but am trying to understand to what extent that results in us not being able to easily support other platforms (or not). Oh, I see. Thanks for clarifying. I don't think this limits us in that respect at all. I don't think we've ever rejected patches that would allow us to run on other platforms (and I can't imagine we ever would). That said, Ubuntu offers a very up-to-date selection of software that we can build upon. Not being limited by a static, out-of-date set of dependencies is very much the reason we can move forward at the pace that we do. Other distros might not have all the dependencies we need. Yet, at least. It's possible they will in their next release. If we take the example from my previous e-mail in this thread.. There was a race condition that was caused by a bug in Eventlet. There were two[1] ways to fix it: 1) Fix it in Eventlet or 2) accept Eventlet's brokenness and attempt to work around it. Once the problem was identified and I was at a point where I knew I had to make that decision, option 1) took probably about an hour or so, includiing filing the bug with eventlet, providing them with a test case, writing a patch, uploading it to Ubuntu, and backporting it to multiple version of Ubuntu in our PPA's. It's not entirely inconceivable that I'd still be working on option 2 right now (we rely pretty heavily on Eventlet). If someone wants to put Nova into some other distribution they just need to make sure that the eventlet in said distro has this patch applied. [1]: Well, there's a third: Monkey patch eventlet. I'm just afraid I'd be caught in an infinite loop of irony. -- Soren Hansen Ubuntu Developer http://www.ubuntu.com/ OpenStack Developer http://www.openstack.org/ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] OpenStack Compute API 1.1
I wanted to put out into the open where we think the evolution of the apis will go over the next few releases. This is by no means the only way to do this, but I thought it would be a start of conversation. http://wiki.openstack.org/api_transition I also wanted to clear up some confusion that I think came out of our email thread the other day. With the OpenStack 1.1 API proposal, this is really a OpenStack Compute 1.1 proposal. While volumes and images are currently in, I think longer term they would be pulled out. The network and volume services should be able to scale independently of each other. If you look at the diagram, the changes would entail moving from an amqp protocol to a http protocol that a worker would hit on the public/admin interfaces to accomplish the same work as before. Lets keep the thread going. Pvo From: Justin Santa Barbara jus...@fathomdb.commailto:jus...@fathomdb.com Date: Tue, 15 Feb 2011 11:38:37 -0800 To: Troy Toman troy.to...@rackspace.commailto:troy.to...@rackspace.com Cc: openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Subject: Re: [Openstack] OpenStack Compute API 1.1 Sounds great - when the patch comes in we can discuss whether this should be an extension or whether scheduled snapshots / generic tasks have broader applicability across OpenStack (and thus would be better in the core API) Is there a blueprint? On Tue, Feb 15, 2011 at 11:32 AM, Troy Toman troy.to...@rackspace.commailto:troy.to...@rackspace.com wrote: On Feb 15, 2011, at 1:06 PM, Justin Santa Barbara wrote: OK - so it sounds like volumes are going to be in the core API (?) - good. Let's get that into the API spec. It also sounds like extensions (swift / glance?) are not going to be in the same API long-term. So why do we have the extensions mechanism? Until we have an implemented use case (i.e. a patch) that uses the extensions element, I don't see how we can spec it out or approve it. So if you want it in v1.1, we better find a team that wants to use it and write code. If there is such a patch, I stand corrected and let's get it reviewed and merged. I would actually expect that the majority of the use cases that we want in the API but don't _want_ to go through core would be more simply addressed by well-known metadata (e.g. RAID-5, multi-continent replication, HPC, HIPAA). I'm don't agree that the lack of a coded patch means we can't discuss an extension mechanism. But, if you want a specific use case, we have at least one we intend to deliver. It may be more of a one-off than a general case because it is required to give us a reasonable transition path from our current codebase to Nova. But, it is not an imagined need. In the Rackspace Cloud Servers 1.0 API, we support a concept of backup schedules with a series of API calls to manage them. In drafting the OpenStack compute API, this was something that didn't feel generally applicable or useful in the core API. So, you don't see it as part of the CORE API spec. That said, for transition purposes, we will need a way to provide this capability to our customers when we move to Nova. Our current plan is to do this using the extension mechanism in the proposed API. If there is a better way to handle this need, then let's discuss further. But, I didn't want the lack of a specific example to squash the idea of extensions. Troy Toman Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.commailto:ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Steps that can help stabilize Nova's trunk
Believe, me we know. Our team is bringing up OpenStack on our heterogeneous testbed, a 1TB 256core SGI UltraViolet machine, three Nvidia S2050 boards connected to three SGI XE500 boxes (exposing the GPUs to new instance types using a modified gVirtuS driver), and a nova-compute proxy managing 10 Tilera TILEmpower systems with 64-core TilePro64 processors (that are bare-metal PXE booting the images from an x86 box). They all have dual 10GbE interfaces and it's all SUSE Enterprise. Configuration management is a joy. I was under the impression that runtest.sh -V -f was what committers should strive to achieve before submitting a branch for review. That means you might win the booby prize and hit python_migrate bug totally unrelated to your patch. Brian Schott bfsch...@gmail.com On Feb 17, 2011, at 2:18 PM, Soren Hansen wrote: One other thing I forgot to point out is that there's a world outside of Python, too. You can pull all the stuff you want from pip, but there's still a whole bunch of things that aren't managed by pip that you need to rely on your distro for anyways. Like, say, a kernel, Python itself, libvirt, iscsi tools, etc., etc. -- Soren Hansen Ubuntu Developerhttp://www.ubuntu.com/ OpenStack Developer http://www.openstack.org/ PGP.sig Description: This is a digitally signed message part ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] OpenStack Compute API 1.1
Pulling volumes images out into separate services (and moving from AMQP to REST) sounds like a huge breaking change, so if that is indeed the plan, let's do that asap (i.e. Cactus). On Thu, Feb 17, 2011 at 3:44 PM, Paul Voccio paul.voc...@rackspace.comwrote: I wanted to put out into the open where we think the evolution of the apis will go over the next few releases. This is by no means the only way to do this, but I thought it would be a start of conversation. http://wiki.openstack.org/api_transition I also wanted to clear up some confusion that I think came out of our email thread the other day. With the OpenStack 1.1 API proposal, this is really a OpenStack Compute 1.1 proposal. While volumes and images are currently in, I think longer term they would be pulled out. The network and volume services should be able to scale independently of each other. If you look at the diagram, the changes would entail moving from an amqp protocol to a http protocol that a worker would hit on the public/admin interfaces to accomplish the same work as before. Lets keep the thread going. Pvo From: Justin Santa Barbara jus...@fathomdb.com Date: Tue, 15 Feb 2011 11:38:37 -0800 To: Troy Toman troy.to...@rackspace.com Cc: openstack@lists.launchpad.net openstack@lists.launchpad.net Subject: Re: [Openstack] OpenStack Compute API 1.1 Sounds great - when the patch comes in we can discuss whether this should be an extension or whether scheduled snapshots / generic tasks have broader applicability across OpenStack (and thus would be better in the core API) Is there a blueprint? On Tue, Feb 15, 2011 at 11:32 AM, Troy Toman troy.to...@rackspace.comwrote: On Feb 15, 2011, at 1:06 PM, Justin Santa Barbara wrote: OK - so it sounds like volumes are going to be in the core API (?) - good. Let's get that into the API spec. It also sounds like extensions (swift / glance?) are not going to be in the same API long-term. So why do we have the extensions mechanism? Until we have an implemented use case (i.e. a patch) that uses the extensions element, I don't see how we can spec it out or approve it. So if you want it in v1.1, we better find a team that wants to use it and write code. If there is such a patch, I stand corrected and let's get it reviewed and merged. I would actually expect that the majority of the use cases that we want in the API but don't _want_ to go through core would be more simply addressed by well-known metadata (e.g. RAID-5, multi-continent replication, HPC, HIPAA). I'm don't agree that the lack of a coded patch means we can't discuss an extension mechanism. But, if you want a specific use case, we have at least one we intend to deliver. It may be more of a one-off than a general case because it is required to give us a reasonable transition path from our current codebase to Nova. But, it is not an imagined need. In the Rackspace Cloud Servers 1.0 API, we support a concept of backup schedules with a series of API calls to manage them. In drafting the OpenStack compute API, this was something that didn't feel generally applicable or useful in the core API. So, you don't see it as part of the CORE API spec. That said, for transition purposes, we will need a way to provide this capability to our customers when we move to Nova. Our current plan is to do this using the extension mechanism in the proposed API. If there is a better way to handle this need, then let's discuss further. But, I didn't want the lack of a specific example to squash the idea of extensions. Troy Toman Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netUnsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Steps that can help stabilize Nova's trunk
Agreed. I don't think bleeding-edge pip should block a merge either. It could cause a lot of false-positives. I do think up-to-date Natty, Maverick, and Lucid with build-depends from PPA should be tested, probably CentOS and SUSE if that can be automated. You want apt-get dist-upgrade ; run_test.sh -N to work. How about some automation to Score your Branch? Do that BEFORE a branch gets proposed for review. Encourage committers to post their branch score in the bug. At least we when see what is broken. Brian Schott bfsch...@gmail.com On Feb 17, 2011, at 4:21 PM, Soren Hansen wrote: 2011/2/17 Jay Pipes jaypi...@gmail.com: One thing we saw in the list and experienced first hand is that your Hudson server uses a pre-configured environment and ours pulls virtual env every time. We had failures on trunk that yours missed because of new pip pulled versions. A fresh run_tests.sh -f needs to work. It is the only guaranteed sanity test everybody can replicate. It might pull upstream bugs, but better to be ahead of that curve than behind. Although Soren adequately explained why he thinks that running run_tests.sh on Hudson should *not* be against a pip/virtualenv setup, I would like to state that I think it would be a good idea to have Hudson do *both*. In other words, for each pre-merge into trunk, Hudson would run two things: * run_tests.sh -N * run_tests.sh -V -f I understand the motivation, I'm just not sure I want the latter to actually block a merge. As an example, the recent race condition I spotted and fixed required a patch to land in eventlet. If the latter was allowed to block a merge, we'd have to keep a known race condition in Nova until upstream decides to do a release so that pip could fetch it. That could take a *long* time. In the mean time, I stuck a fixed Eventlet package in our PPA (and in Ubuntu Natty proper) so that we could move on with our lives and get rid of the race condition. There's simply a flexibility in this approach that I don't see how we can obtain with the pip approach. Again, though, I would be *delighted* to have automated testing on a load of different platforms. I just don't believe they should all be allowed to block us from moving forward. -- Soren Hansen Ubuntu Developerhttp://www.ubuntu.com/ OpenStack Developer http://www.openstack.org/ PGP.sig Description: This is a digitally signed message part ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] OpenStack Compute API 1.1
On Thu, Feb 17, 2011 at 6:57 PM, Justin Santa Barbara jus...@fathomdb.com wrote: Pulling volumes images out into separate services (and moving from AMQP to REST) sounds like a huge breaking change, so if that is indeed the plan, let's do that asap (i.e. Cactus). Sorry, I have to disagree with you here, Justin :) The Cactus release is supposed to be about stability and the only features going into Cactus should be to achieve API parity of the OpenStack Compute API with the Rackspace Cloud Servers API. Doing such a huge change like moving communication from AMQP to HTTP for volume and network would be a change that would likely undermine the stability of the Cactus release severely. -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] OpenStack Compute API 1.1
An API is for life, not just for Cactus. I agree that stability is important. I don't see how we can claim to deliver 'stability' when the plan is then immediately to destablize everything with a very disruptive change soon after, including customer facing API changes and massive internal re-architecting. On Thu, Feb 17, 2011 at 4:18 PM, Jay Pipes jaypi...@gmail.com wrote: On Thu, Feb 17, 2011 at 6:57 PM, Justin Santa Barbara jus...@fathomdb.com wrote: Pulling volumes images out into separate services (and moving from AMQP to REST) sounds like a huge breaking change, so if that is indeed the plan, let's do that asap (i.e. Cactus). Sorry, I have to disagree with you here, Justin :) The Cactus release is supposed to be about stability and the only features going into Cactus should be to achieve API parity of the OpenStack Compute API with the Rackspace Cloud Servers API. Doing such a huge change like moving communication from AMQP to HTTP for volume and network would be a change that would likely undermine the stability of the Cactus release severely. -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Steps that can help stabilize Nova's trunk
On Feb 17, 2011, at 5:55 PM, Soren Hansen wrote: 2011/2/17 Lorin Hochstein lo...@isi.edu: For those who deploy on platforms other than Ubuntu, are these customizations recorded somewhere in the OpenStack documentation? Hmm.. No. I submitted it upstream, applied it in Ubuntu, and uploaded backported packages to the PPA's. I guess we could also add a patches/ directory to nova and stick stuff in there. Would that address your concerns? Yeah, I think that should do. Having access to a comprehensive set of the dependency packages versions for the reference implementation platform would be very helpful for debugging problems on other platforms. Is it possible automatically pull this info from the PPA, extract it from the package metadata, and then add it to the admin guide docs? I'd hate to have to update admin docs for this sort of stuff. A general pointer to the (not-yet-existing) patches/ directory would be a good idea, though. I think that's fine, something like Some dependencies contain bugs that prevent nova working correctly. In those cases nova relies on a customized version of the dependency with a bugfix. These bugfixes to dependencies are located in the patches/ directory. Take care, Lorin -- Lorin Hochstein, Computer Scientist USC Information Sciences Institute 703.812.3710 http://www.east.isi.edu/~lorin smime.p7s Description: S/MIME cryptographic signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] OpenStack Compute API 1.1
Sorry, I don't view the proposed changes from AMQP to REST as being customer facing API changes. Could you explain? These are internal interfaces, no? -jay On Thu, Feb 17, 2011 at 8:13 PM, Justin Santa Barbara jus...@fathomdb.com wrote: An API is for life, not just for Cactus. I agree that stability is important. I don't see how we can claim to deliver 'stability' when the plan is then immediately to destablize everything with a very disruptive change soon after, including customer facing API changes and massive internal re-architecting. On Thu, Feb 17, 2011 at 4:18 PM, Jay Pipes jaypi...@gmail.com wrote: On Thu, Feb 17, 2011 at 6:57 PM, Justin Santa Barbara jus...@fathomdb.com wrote: Pulling volumes images out into separate services (and moving from AMQP to REST) sounds like a huge breaking change, so if that is indeed the plan, let's do that asap (i.e. Cactus). Sorry, I have to disagree with you here, Justin :) The Cactus release is supposed to be about stability and the only features going into Cactus should be to achieve API parity of the OpenStack Compute API with the Rackspace Cloud Servers API. Doing such a huge change like moving communication from AMQP to HTTP for volume and network would be a change that would likely undermine the stability of the Cactus release severely. -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Steps that can help stabilize Nova's trunk
Trey Morris wrote: I don't like that it currently only runs on ubuntu + the ppa. If it doesn't work with existing versions I think we're doing something wrong. Even when natty comes out, I don't like the idea of having to ensure I have latest ubuntu for openstack to run. Maybe with my Ubuntu core developer hat on, I can try to clarify. Once an Ubuntu version is released, you can't push new versions of libraries or new features in it, only critical bugfixes. So when for openstack we happen to need to new version of a library, it won't run on the current stable version. We have a choice between avoiding new versions of libraries altogether, or use a PPA to add the required version. The route we follow is to provide the new version of the library in a PPA for the current stable releases (Lucid/Maverick), and update the development release (Natty) proper. The end goal being, when Natty goes out, the corresponding Openstack release (Cactus) runs on it without the need for a PPA. For those that prefer to run it on an older stable release (or an LTS), the release PPA can be used. Regards, -- Thierry Carrez (ttx) Release Manager, OpenStack Core developer, Ubuntu ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack-poc] Meeting this week
Reminder that we've got a meeting today at 2100UTC/3:00 PM central. On Feb 15, 2011, at 11:02 AM, Jonathan Bryce wrote: In the meeting last week, we approved the 2011 Scope and Charter and discussed the image format proposal and intermediate releases without any final decision. John Purrier and others are discussing the Image Format proposal for changes based off the feedback from Ewan. You can read the minutes and logs from the meeting here: http://wiki.openstack.org/Governance/POC/PocMeetingLogs Since we didn't get through everything, we wanted to meet again at the same time this week--Thursday 2/17 at 2100 UTC/3:00 PM central. Here's the agenda I have so far. Send me anything else you'd like to get on there. 1) Continue release discussion - Current thought was to allow projects to decide to do intermediate releases between major releases if needed and work with the release manager to backport appropriate fixes and package a point release. 2) Google Summer of Code 3) Incubation/new project process - We had an idea thrown out last week to create a community of related projects that allowed developers to add themselves and their code to a system like userscripts.org. The projects are then voted up and down by the community and the relevant projects that are the highest rated are candidates for inclusion as full OpenStack projects. Let me know if the time will not work for any of you. Thanks! Jonathan. ___ Mailing list: https://launchpad.net/~openstack-poc Post to : openstack-poc@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack-poc More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack-poc Post to : openstack-poc@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack-poc More help : https://help.launchpad.net/ListHelp
[Openstack-xenapi] Requesting input on vm-params for Nova
Hi,
I'm working to provide a way in Nova to pass configurable boot
parameters to XenServer instances based on the os type of the image.
We (Rackspace) have been doing this in our older code base for some
time. Linux instances are provided certain parameters and Windows
instances get others. I'd like to get some input from people at
Citrix about which combinations of vm-params we should be using in
Nova. The latter part of this email details the differences I've
found between Rackspace's vm-params and Nova's.
First, Nova already has code for switching based on whether a kernel
image is available, or if the guest is PV or HVM, but not yet by os
type. In nova:nova/virt/xenapi/vm_utils.py
if instance.kernel_id:
rec['PV_args'] = 'root=/dev/xvda1'
rec['PV_kernel'] = kernel
rec['PV_ramdisk'] = ramdisk
else:
if pv_kernel:
rec['PV_args'] = 'noninteractive'
rec['PV_bootloader'] = 'pygrub'
else:
rec['HVM_boot_policy'] = 'BIOS order'
rec['HVM_boot_params'] = {'order': 'dc'}
rec['platform'] = {'acpi': 'true', 'apic': 'true',
'pae': 'true', 'viridian': 'true'}
All of Rackspace's images, including Linux ones, will include a kernel
inside the image. Thus we will always fall inside the first 'else'
block in the above code.
However, based on what is already in Nova I'm not entirely sure which
combination of vm-params should be used as there are some
discrepancies. Below are the vm-params we have been using in our
current infrastructure:
Rackspace:
Linux:
PV_bootloader: pygrub
PV_args: clocksource=jiffies
platform: {viridian: true, pae: true,
apic: true, acpi: true,
nx : false, timeoffset: 0}
HVM_boot_policy:
HVM_boot_params: {}
HVM_shadow_multiplier:
Windows:
PV_bootloader:
PV_args:
platform: {viridian: true, pae: true,
apic: true, acpi: true,
nx : true}
HVM_boot_policy: BIOS order
HVM_boot_params: {order: dc}
HVM_shadow_multiplier: 1.000
Both:
other_config: {allowvssprovider: false}
user_version: 1
Some of these were chosen because they were used in the Linux and
Windows templates.
Below are the vm-params from Nova that are different from what we
(Rackspace) currently use:
Nova:
Linux:
platform: {}
PV_args: 'noninteractive'
Windows:
platform: {'acpi': 'true', 'apic': 'true',
'pae': 'true', 'viridian': 'true'}
Both:
other_config: {}
user_version: '0'
Also, we supply the following vm-params to all instances when booting
but they are not currently used in Nova.
blocked_operations: {}
ha_always_run: true,
ha_restart_priority:
tags: []
xenstore_data: {},
Questions for the Citrix folks:
1) Do you see any problems with our existing vm-params? Are any
unnecessary?
2) Do you have any suggestions for resolving the differences found in Nova?
(PV_args, platform, other_config, user_version)
3) Should our five extra vm-params be added to Nova?
Thanks,
Cory
--
Cory Wright
Software Developer
cory.wri...@rackspace.com
___
Mailing list: https://launchpad.net/~openstack-xenapi
Post to : openstack-xenapi@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack-xenapi
More help : https://help.launchpad.net/ListHelp
