Re: [Openstack] error while creating instance on nfs share
On Tue, Jul 9, 2013 at 3:27 AM, Chathura M. Sarathchandra Magurawalage 77.chath...@gmail.com wrote: Hello al, I followed the Openstack instructions for enabling vm migration ( http://docs.openstack.org/diablo/openstack-compute/admin/content/configuring-live-migrations.html ) But I can not get it working, since when I create an instance I get the following error on the compute node. [snip] 2013-07-08 17:01:34 9329 TRACE nova.openstack.common.rpc.amqp File /usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py, line 1310, in _create_image 2013-07-08 17:01:34 9329 TRACE nova.openstack.common.rpc.amqp utils.ensure_tree(basepath(suffix='')) 2013-07-08 17:01:34 9329 TRACE nova.openstack.common.rpc.amqp File /usr/lib/python2.7/dist-packages/nova/utils.py, line 1367, in ensure_tree 2013-07-08 17:01:34 9329 TRACE nova.openstack.common.rpc.amqp os.makedirs(path) 2013-07-08 17:01:34 9329 TRACE nova.openstack.common.rpc.amqp File /usr/lib/python2.7/os.py, line 157, in makedirs Thanks heaps for including the traceback in your message, it makes these things much easier to debug. This looks to me like a permissions problem. The code has tried to make a directory on your NFS share, and that has failed. Specifically, this looks like the directory the instance files go into. What happens if you try to create a directory on the NFS share as the user that nova is running as? Cheers, Michael -- Rackspace Australia ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Download a specific image from Glance to local hard drive
On Thu, Jun 20, 2013 at 7:33 PM, Narayanan, Krishnaprasad naray...@uni-mainz.de wrote: Hallo All, I am using OpenStack ESSEX and want to download an image from glance to my local hard drive. Can I get to know any commands in glance (in ESSEX) using which I can perform this operation? Take a look at the image-download command to the glance command line client. Cheers, Michael ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Download a specific image from Glance to local hard drive
I'm not sure. Essex is very old now. Try running glance help and see what happens? Michael On Thu, Jun 20, 2013 at 7:49 PM, Narayanan, Krishnaprasad naray...@uni-mainz.de wrote: Hi Michael, Is this command valid in OpenStack ESSEX? Thanks Krishnaprasad -Original Message- From: mikalst...@gmail.com [mailto:mikalst...@gmail.com] On Behalf Of Michael Still Sent: Donnerstag, 20. Juni 2013 11:46 To: Narayanan, Krishnaprasad Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] Download a specific image from Glance to local hard drive On Thu, Jun 20, 2013 at 7:33 PM, Narayanan, Krishnaprasad naray...@uni-mainz.de wrote: Hallo All, I am using OpenStack ESSEX and want to download an image from glance to my local hard drive. Can I get to know any commands in glance (in ESSEX) using which I can perform this operation? Take a look at the image-download command to the glance command line client. Cheers, Michael ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] To run Openstack, stack.sh is required to execute after every reboot ?
devstack is only intended for developers to test changes. If your building a persistent cloud, then you should be using one of the various sets of operating system packages. Cheers, Michael On Tue, Jun 18, 2013 at 5:15 AM, Vivek Satpute vivekonlin...@gmail.com wrote: Hi, I recently setup a openstack on one node, which is hosting all the services like keystone, nova, glance, cinder, etc. Using Dashboard, I could create the VM instances and volumes too. But if I rebooted my host machine where openstack is configured, once machine comes up, horizon cannot retrieve any data. I realised that, there is no services running on host setup. (Checked using ps -ef command). I again did ./stack.sh and then I could create new instances, volumes, etc. How can I configure openstack such a that it would get started after reboot automatically ? Or Am I missing something here ? Thanks in advance, Vivek Satpute ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Benefits for moving live migration/resize/code migration/provision to conductor
IIRC the discussion from the summit, there was concern about compute nodes talking directly to each other. The way live migration works in libvirt is that the libvirt on one node opens up a connection to the other and then streams the instance across. If this is bounced off a conductor, then it makes firewall rules much easier to construct. Cheers, Michael On Sat, Jun 1, 2013 at 2:53 PM, Lau Jay jay.lau@gmail.com wrote: Hi Stackers, I noticed that there are some blueprints trying to move the logic of live migration/resize/code migration/provision from nova scheduler to nova conductor, but the blueprint did not describe clearly the benefits of doing so, can some experts give some explanation on this? I know the original design for nova conductor is for a non-db nova compute, but what's the reason of moving scheduling logic to nova conductor? Thanks, Jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OVS] Loopingcall and high load on nova comp
Hi! This means that this loopingcall will be running in a hard loop (no delay between runs) because it is taking longer than the interval it has requested to be called at. Either we need to change the interval (if that's ok, no idea with Quantum), or we need to make the work being done in a loop faster. Hope that helps, Michael On Fri, May 17, 2013 at 8:41 PM, Heiko Krämer i...@honeybutcher.de wrote: Hi guys, i seen on each nova compute node a ressource overhead caused by quantum ovs agent. This means if only runs the agent on the compute node (+ nova-compute) i'll have sometimes a load by 2. The agent will have a many cpu hours and if i take a look into the log file i see manytimes this one: 2013-05-17 12:35:46 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 0.195565 sec 2013-05-17 12:35:50 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 0.24272 sec 2013-05-17 12:35:54 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 0.254162 sec 2013-05-17 12:35:58 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 0.131815 sec 2013-05-17 12:36:02 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 0.213518 sec 2013-05-17 12:36:07 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 0.212293 sec 2013-05-17 12:36:11 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 0.223576 sec I've searched to this warning but without success. Anyone have an idea or the same problem ? Greetings Heiko ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [OSSA 2013-012] Nova fails to verify image virtual size (CVE-2013-2096)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 OpenStack Security Advisory: 2013-012 CVE: CVE-2013-2096 Date: May 16, 2013 Title: Nova fails to verify image virtual size Reporter: Loganathan Parthipan Products: Nova Affects: All versions Description: Loganathan Parthipan publicly reported a vulnerability in Nova. Nova did not implement checking for the virtual size of a qcow2 image used as ephemeral storage for instances. It is therefore possible for a user to create an image which has a large virtual size, but little data. Once the instance is created, the user can then proceed to fill the virtual disk, and consume all available disk on the host node file system. Havana (development branch) fix: https://review.openstack.org/28717 Grizzly fix: https://review.openstack.org/28901 Folsom fix: https://review.openstack.org/29192 References: https://bugs.launchpad.net/nova/+bug/1177830 http://www.cve.mitre.org/cgi-bin/cvename.cgi?name=2013-2096 Thanks, Michael Still OpenStack Vulnerability Management Team -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlGUoRQACgkQlhS32Mrx3702BgCeKZUDDA/W6Nj/xgC1a1n9vHvP vvoAnRfIOXnuvJ01c7IxGyXON7LIh5kt =YfoG -END PGP SIGNATURE- ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] nova-compute : nbd module not loaded
You get this error if there are no nbd devices. Specifically /sys/block/nbd0 doesn't exist. I'd start by seeing if the kernel module is loaded. Cheers, Michael On Fri, May 3, 2013 at 10:57 AM, Mohammed Amine SAYA asaya.openst...@gmail.com wrote: Hi All, I have had time to investigate why booting from volume doesn't work and I found the following trace in /var/log/nova/nova-compute.log. I took a look at nova.conf but I couldn't find any relevent entry for nova.virt.disk.mount.disk. Is it normal to have this trace? It occurs every time I try to launch a new instance whether nova uses a standalone image or tries to boot from a volume. Has nova.virt.disk.mount.nbd module been integrated to nova? BTW, I deployed grizzly 2013.1. Any hint on this issue guys? Thanks for you help. Amine. 2013-05-03 02:04:03.447 ERROR nova.virt.disk.mount.nbd [req-afe5f7ba-ac22-4d6c-9fad-283903de0193 f037cb17ddba4a92b1429e11b0c3200e 28be43a159854b43ae3b3bdcec3f266a] nbd module not loaded 2013-05-03 02:04:05.449 ERROR nova.virt.disk.mount.nbd [req-afe5f7ba-ac22-4d6c-9fad-283903de0193 f037cb17ddba4a92b1429e11b0c3200e 28be43a159854b43ae3b3bdcec3f266a] nbd module not loaded 2013-05-03 02:04:07.451 ERROR nova.virt.disk.mount.nbd [req-afe5f7ba-ac22-4d6c-9fad-283903de0193 f037cb17ddba4a92b1429e11b0c3200e 28be43a159854b43ae3b3bdcec3f266a] nbd module not loaded 2013-05-03 02:04:09.452 ERROR nova.virt.disk.mount.nbd [req-afe5f7ba-ac22-4d6c-9fad-283903de0193 f037cb17ddba4a92b1429e11b0c3200e 28be43a159854b43ae3b3bdcec3f266a] nbd module not loaded 2013-05-03 02:04:11.454 ERROR nova.virt.disk.mount.nbd [req-afe5f7ba-ac22-4d6c-9fad-283903de0193 f037cb17ddba4a92b1429e11b0c3200e 28be43a159854b43ae3b3bdcec3f266a] nbd module not loaded 2013-05-03 02:04:13.475 ERROR nova.virt.disk.mount.nbd [req-afe5f7ba-ac22-4d6c-9fad-283903de0193 f037cb17ddba4a92b1429e11b0c3200e 28be43a159854b43ae3b3bdcec3f266a] nbd module not loaded 2013-05-03 02:04:15.477 ERROR nova.virt.disk.mount.nbd [req-afe5f7ba-ac22-4d6c-9fad-283903de0193 f037cb17ddba4a92b1429e11b0c3200e 28be43a159854b43ae3b3bdcec3f266a] nbd module not loaded 2013-05-03 02:04:17.479 ERROR nova.virt.disk.mount.nbd [req-afe5f7ba-ac22-4d6c-9fad-283903de0193 f037cb17ddba4a92b1429e11b0c3200e 28be43a159854b43ae3b3bdcec3f266a] nbd module not loaded 2013-05-03 02:04:19.481 ERROR nova.virt.disk.mount.nbd [req-afe5f7ba-ac22-4d6c-9fad-283903de0193 f037cb17ddba4a92b1429e11b0c3200e 28be43a159854b43ae3b3bdcec3f266a] nbd module not loaded 2013-05-03 02:04:21.483 ERROR nova.virt.disk.mount.nbd [req-afe5f7ba-ac22-4d6c-9fad-283903de0193 f037cb17ddba4a92b1429e11b0c3200e 28be43a159854b43ae3b3bdcec3f266a] nbd module not loaded 2013-05-03 02:04:23.551 ERROR nova.virt.disk.mount.nbd [req-afe5f7ba-ac22-4d6c-9fad-283903de0193 f037cb17ddba4a92b1429e11b0c3200e 28be43a159854b43ae3b3bdcec3f266a] nbd module not loaded 2013-05-03 02:04:25.554 ERROR nova.virt.disk.mount.nbd [req-afe5f7ba-ac22-4d6c-9fad-283903de0193 f037cb17ddba4a92b1429e11b0c3200e 28be43a159854b43ae3b3bdcec3f266a] nbd module not loaded 2013-05-03 02:04:27.556 ERROR nova.virt.disk.mount.nbd [req-afe5f7ba-ac22-4d6c-9fad-283903de0193 f037cb17ddba4a92b1429e11b0c3200e 28be43a159854b43ae3b3bdcec3f266a] nbd module not loaded 2013-05-03 02:04:29.556 ERROR nova.virt.disk.mount.nbd [req-afe5f7ba-ac22-4d6c-9fad-283903de0193 f037cb17ddba4a92b1429e11b0c3200e 28be43a159854b43ae3b3bdcec3f266a] nbd module not loaded 2013-05-03 02:04:31.558 ERROR nova.virt.disk.mount.nbd [req-afe5f7ba-ac22-4d6c-9fad-283903de0193 f037cb17ddba4a92b1429e11b0c3200e 28be43a159854b43ae3b3bdcec3f266a] nbd module not loaded 2013-05-03 02:04:33.636 WARNING nova.virt.disk.mount.api [req-afe5f7ba-ac22-4d6c-9fad-283903de0193 f037cb17ddba4a92b1429e11b0c3200e 28be43a159854b43ae3b3bdcec3f266a] Device allocation failed after repeated retries. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Vm provision code flow in openstack
So, there are a few things which happen here. VM provisioning is handled by the compute driver (checkout run_instance in nova/compute/manager.py). Scheduling however is handled by the scheduler, which has a plugin system (checkout nova/scheduler/*). Also, you'd be better off browing git for the source code ( http://github.com/openstack/nova.git) instead of the system installed files. Cheers, Michael On Mon, Apr 29, 2013 at 7:14 PM, Deepak Jeswani1 dejes...@in.ibm.comwrote: Hello everyone, I am trying to understand the flow of control in code in openstack for provisioning a VM. I want to understand it in order to do modify scheduler module as per our requirements. I have found the code base which, if I am not wrong, is kept in /usr/share/pyshared/nova/* directory but I can't see .py files getting accessed or executed if I provision a VM. Can someone please help me to understand the flow of code or give me pointers from where I can understand it. I hope even if I know the starting point (REST handler for VM provisioning call) in code then I will be able to trace it. Thanks Regards Deepak Deepak Jeswani Software Engineer, Distributed Systems Group, IBM Research-India ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Grizzly release notes and the never-ending image-cache-manager issue
On Fri, Apr 5, 2013 at 7:20 AM, Lorin Hochstein lo...@nimbisservices.com wrote: Michael (et al): The Grizzly release notes https://wiki.openstack.org/wiki/ReleaseNotes/Grizzly say: The image-cache-manager has been turned on by default. This may have potential issues for users who are using a shared filesystem for their instances_path. Set remove_unused_base_images=false in your nova.conf file on your compute nodes to revert this behaviour. My understanding was that in Grizzly, this wasn't an issue since shared storage was automatically detected (e.g. https://bugs.launchpad.net/nova/+bug/1075018) This is my understanding as well -- that we now detect shared storage and do the right thing. I don't have any data on how much real world testing that code has experienced though. Is it safe to zap this from the release notes? Safe is a relative thing. I'd be more comfortable if I knew that someone had deployed the code and had a good experience, but when I ask on the operators list I get puzzled stares... Michael ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Nova] Creating instances with custom UUIDs
On Thu, Apr 4, 2013 at 9:16 AM, Rafael Rosa rafaelros...@gmail.com wrote: Hi, In our OpenStack installation we have an issue when creating new instances, we need to execute some long running processes before calling nova boot and the call blocks for the end user for a while. We would like to return immediately to the caller with a final instance UUID and do the work on the background, but it's only generated when during actual instance creation, which is a no go in our situation. The instance_create database call already accepts an instance UUID as an argument, so that bit looks like it should work out well for you. So, I guess this is mostly a case of working out how you want the API to work. Personally, I would have no problem with something like this, so long as we could somehow reserve the instance UUID so that another caller doesn't try and create an instance with the same UUID while you're doing your slow thing. Cheers, Michael ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] DNS options for nova
Hi. I've been playing with the DNS drivers in nova recently, and I'm trying to decide if anyone actually uses it. The only driver in the codebase that looks useful a the moment is the LDAP backed DNS driver that Wikimedia wrote. There's also stackforge project called Moniker which implements DNS as a service, but its a lot more complicated that the nova DNS driver (multiple daemons, a database, message queues). So... Do you use the nova DNS drivers? Would you be filled with rage if I deleted them? On the flip side, would you use a TSIG dyndns driver if I added it to nova? What if I moved the DNS driver layer to the compute manger in nova so that it worked as well with Quantum? Or are you using Moniker? If so, what do you think of it? Or does Quantum have a solution to this that I should be looking at? Thanks, Michael ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] TC candidacy
Hi. I'd like to run for the TC Spring 2013 election. I am a senior software engineer at Rackspace in their OpenStack group, and have worked in a variety of cloud devops roles for the last seven years. I think my operations experience gives me an interesting perspective into where OpenStack should be going in the next few years. My basic platform is the same as for the Nova PTL election [1] -- I think we need to get better at closing bugs, and selecting defaults which work out of the box for OpenStack deployers. We are blessed with a very engaged user community, and we need to focus us much as possible on giving new users a good experience. I am an active Nova and Oslo core reviewer and frequently appear in the top ten contributors for Nova in a given month. I am a very active code reviewer, especially for Nova. I am also serve on the OpenStack Vulnerability Management Team. I strongly believe in the future of OpenStack and want to be part of that success in any way I can. For the last two years my non-OpenStack open source contributions have mainly been as Director for linux.conf.au 2013, which was the largest OpenStack event to be run in Australia so far. Thanks, Michael 1: http://lists.openstack.org/pipermail/openstack-dev/2013-March/006417.html ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] TC candidacy
On Fri, Mar 15, 2013 at 11:27 AM, Michael Still mi...@stillhq.com wrote: Hi. I'd like to run for the TC Spring 2013 election. I am a senior software engineer at Rackspace in their OpenStack group, and have worked in a variety of cloud devops roles for the last seven years. I think my operations experience gives me an interesting perspective into where OpenStack should be going in the next few years. I was just asked what my thoughts on integrated projects are, so I figured I should address that in public. First off, the TC has oversight on all technical matters [1], so while project incubation and integration is a topic that the TC handles, it isn't the only thing I'd expect a well running TC to handle. Examples of other things that the TC should be caring about are: - what languages we develop in - what libraries we standardise on and if the existing standards are still the best choices (for example, is sqlalchemy still the best way of handling database abstraction?) - how our CI infrastructure helps us run the project To specifically address the incubation / integration question, I think that OpenStack runs the risk of becoming distracted with too many projects. We need to remember that the foundation is still quite young, as is our community in general. I personally think that OpenStack should be focussing on Infrastructure as a Service problems for now, with the intention of expanding that scope later when we have a solid stable base of infrastructure projects. I can see why it is attractive for projects to want to become incubated, especially because it means they get a bunch of (mainly CI) infrastructure for free, as well as a lot of mind share from the association. We need to remember that our CI resources are limited however, and we need to be careful not to overload that team. Additionally, each project we add makes it hard for us to define who is distributing OpenStack. Do they have to ship every integrated project? Even if its not relevant to the product they're trying to build? We live in a complicated world though. I think each of these decisions should be made on its individual merits based on the facts that are true at the time. I also think the TC is on the right track here at the moment, so I don't think we need to wrap a bunch of super formal policy around this area at the moment. Hopefully that helps. Cheers, Michael 1: https://wiki.openstack.org/wiki/Governance/TechnicalCommittee ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [Packagers] Adding psutils as a dependency for nova
Hi. https://review.openstack.org/#/c/17980/ adds psutils (packaged on Ubuntu as python-psutils) as a dependency for nova. This (or something very much like it) is required to traverse the process tree in the is_parent_process() method in nova.utils. I need that functionality to eliminate a race condition in the nbd mounter code. Hopefully this isn't too painful for packagers. That review hasn't landed yet, so its possible that things will change before it does, but I wanted to provide as much notice as possible. Michael ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Packagers] Adding psutils as a dependency for nova
Stand down. Padraig has suggested a better way. Michael On Thu, Dec 13, 2012 at 10:02 AM, Michael Still mi...@stillhq.com wrote: Hi. https://review.openstack.org/#/c/17980/ adds psutils (packaged on Ubuntu as python-psutils) as a dependency for nova. This (or something very much like it) is required to traverse the process tree in the is_parent_process() method in nova.utils. I need that functionality to eliminate a race condition in the nbd mounter code. Hopefully this isn't too painful for packagers. That review hasn't landed yet, so its possible that things will change before it does, but I wanted to provide as much notice as possible. Michael ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Upcoming wiki migration to Mediawiki
On 12/08/2012 07:15 AM, Anne Gentle wrote: Hi all - tl;dr: Migration of wiki.openstack.org http://wiki.openstack.org from MoinMoin to Mediawiki commences 12/17. Service will continue with Mediawiki as the wiki engine after 12/21. While the MoinMoin engine served its purpose to get started, it makes sense for us to move to Mediawiki and bring the wiki instances into the CI umbrella for better maintenance and ongoing work. Here's the timeline for this move: - migration test Monday 12/17 - cleaning up and testing new wiki site 12/18-20 - go-live 12/21 Is there a period where we shouldn't edit the wiki at all? I ask because this will block new contributor signups. Michael ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Understanding flavors of VM
On 12/05/2012 06:59 PM, Marco CONSONNI wrote: To be honest it seems like I missed something because, from your investigation, the storage is kept under _base. Strange. I didn't know that. Hi! The following description is libvirt specific. Bearing in mind that this code is a moving target and has been re-written at least three times in the last year [1], it works a bit like this... - you request an instance using a given image - that image is fetched to _base from glance (if its not already there) - that image is format converted if required, and then resized to the requested size - the instance disk image is a copy-on-write layer on top of that resized image in _base and is stored in the instance's directory - the instance is booted using the COW layer If another instance with the same image / disk size starts, it can short circuit the process and use the already existing image, which is cool. When an instance is terminated, the instance directory is removed, but files in _base remain. If you have image cache management turned on, then the files in _base are periodically cleaned up. The files in _base are also checksummed to try and detect file corruption, although that hasn't been the most loved feature ever implemented. Hope this helps, Michael 1: Several times I have gone to write a blog post about how this works, and then realized the code has changed again. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Wikipedia page
Hi. I just noticed that the wikipedia page for OpenStack seems pretty badly out of date: http://en.wikipedia.org/wiki/Openstack I wonder if someone feels called to improve it? Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Openstack-operators] Base images removed in upgrade essex - folsom and other stories
On 11/14/2012 04:03 PM, Sam Morrison wrote: After the upgrade which went relatively smoothly (a lot easier than diablo - essex) almost all our base images were deleted by the image cache clean up. I can't explain how this happened. We lost a total of about 70 images that affected ~200 running instances. We have since disabled this flag until we can find out what went wrong. I can see it in the logs and if this flag is enabled it would delete a lot of in use base files still. We have an nfs mounted /var/lib/nova/instances directory where the _base dir is located so I'm wondering if this had something to do with it? Is the image cache cleanup meant to work in a shared instance storage environment? This sounds familiar, so I went spelunking through bugs. Bug 1075018 is relevant, but not actually what you're talking about. The closest bug I can find is 1014227 which is about starting instances using shared storage. So, I think this is a newly reported bug, but I have a pretty strong idea where the problem is (the code doesn't handle shared storage for base files at all). For now I'd recommend that all operators using shared storage and libvirt disable image cache cleanup. I've filed bug 1078594 for this issue, and I apologize for the pain caused. Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Is there any documents explain nova source code in detail?
On 11/01/2012 08:07 AM, xiaohua liu wrote: Want to read nova code but don't know how should I begin. Sorry for disturb. The nova code is a pretty fast moving target. Any documentation which tried to provide a code overview is probably wrong by now. I'd start by deciding what part of nova you find interesting, and then asking where that is in the code base if you can't find it yourself. Then start reading... Cheers, Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Tracking triage statistics
On 10/27/2012 12:08 AM, Dolph Mathews wrote: How many cookies for a self-triaged bug that was subsequently closed as invalid? This rulebook is going to get really complicated really fast ;) Would love to see a report for more projects on a longer timeline! Well, the tool was aimed at nova's weekly meetings, but it will work for any project that uses launchpad for its bug tracking. The problem with a longer report timeline is that tool takes a really long time to run, I guess if you're super patient then that's ok... I've given into you crazy hipsters and moved the code to github. Its at: https://github.com/mikalstill/openstack-triage-tools Cheers, Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Tracking triage statistics
Hi. nova-core wanted a tool to help us track how we were going with triage of incoming bugs, with the intention that we would discuss the statistics at our weekly meetings. I therefore whipped up a quick script to do this. The business rules are as follows: - report on triage events in the last 14 days - a triage is when a user changes status from New and importance from Undecided. A single user needs to perform both actions to earn a cookie The results for the last 14 days are as follows: Report: Chuck Short: 6 (1065211, 1065848, 1066213, 1066254, 1066845, 1068539) Dan Prince: 1 (1070509) Michael Still: 6 (1062474, 1070349, 1064854, 1065728, 1065430, 1070452) Mauro Sergio Martins Rodrigues: 16 (1070155, 1070156, 1070157, 1070158, 1070160, 1070161, 1070162, 1070163, 1070164, 1070165, 1070167, 1070169, 1070170, 1070171, 1070172, 1070173) Matthew Treinish: 1 (1071338) Russell Bryant: 1 (1067858) Vish Ishaya: 8 (1071017, 1053814, 1066887, 1067638, 1067744, 1068154, 1071069, 1071462) I'd be interested in comments people might have. The code is at http://bazaar.launchpad.net/~mikalstill/+junk/openstack-lp-scripts/view/head:/triage-stats.py Cheers, Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Tracking triage statistics
On 10/26/2012 12:24 PM, Russell Bryant wrote: On 10/25/2012 08:18 PM, Michael Still wrote: I'd be interested in comments people might have. The code is at http://bazaar.launchpad.net/~mikalstill/+junk/openstack-lp-scripts/view/head:/triage-stats.py Awesome, thanks! One thing I think we should do for these stats is filter out cases where the reporter == triager. Developers filing bugs and triaging them for their own patches shouldn't be counted. I thought about this... Surely any triage is better than none? If we don't reward self triage, then someone else will still have to triage the bug, right? I'd be interested in other people's thoughts on this. You should move this to github so I can send you a pull request. :-) Heh. The code is on LP mainly because that's where the existing launchpadlib code for openstack resides. I can move it to github if people feel strongly about it. Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Tracking triage statistics
On 10/26/2012 12:41 PM, Russell Bryant wrote: On 10/25/2012 09:30 PM, Michael Still wrote: I thought about this... Surely any triage is better than none? If we don't reward self triage, then someone else will still have to triage the bug, right? I'd be interested in other people's thoughts on this. I guess I figured it's just expected that you triage your own bugs that you filed for patches you're submitting. You're getting credit for that in other places (a merged patch, a fixed bug). If you're lazy and make someone else do it, we should certainly give them triage credit for it. But I see your point. It's still real work that was done. I'm fine with leaving it how you have it. Ok, I've tweaked this a little. Report (a * after the bug id indicates self triage): Chuck Short: 6 (1065211, 1065848, 1066213, 1066254, 1066845, 1068539) Dan Prince: 1 (1070509*) Michael Still: 6 (1062474, 1070349, 1064854, 1065728*, 1065430, 1070452) Mauro Sergio Martins Rodrigues: 16 (1070155*, 1070156*, 1070157*, 1070158*, 1070160*, 1070161*, 1070162*, 1070163*, 1070164*, 1070165*, 1070167*, 1070169*, 1070170*, 1070171*, 1070172*, 1070173*) Matthew Treinish: 1 (1071338*) Russell Bryant: 1 (1067858*) Vish Ishaya: 9 (1071017, 1071547, 1053814, 1066887, 1067638, 1067744, 1068154, 1071069, 1071462) So, you get the cookie, but people know about it too. Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] CLA Contributor list is missing from the wiki?
On 10/17/2012 09:41 AM, Pádraig Brady wrote: I tried to approve a few new CLA signers and noticed that the http://wiki.openstack.org/Contributors page is missing. Trying to restore a previous version resulted in an error. Could someone with appropriate admin rights look into this? The list appears to be back now. Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] OpenStack Performance
On 10/14/2012 03:06 PM, Sriram Subramanian wrote: Frans: Does the report you are talking about has any info on OpenStack performance? Is it sharable? One of the sessions during the summit next week aims at brainstorming ways to do performance and scalability testing for OpenStack. http://summit.openstack.org/cfp/details/37 This sounds like an interesting session. I will try and make it. I was chasing a performance problem the other day that turned out to be a locking issue. In the process I had to add a bunch of instrumentation to nova, and I'd like to see some of that land in the code base. Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [openstack-dev] Nova PTL candidacy
On 09/05/2012 06:03 AM, Matt Joyce wrote: Vish is also a pretty cool guy and doesn't afraid of anything. Vish does a great job -- many hours a day of code review and mentoring, puts up with criticism much more calmly than I think many would, and is a pleasure to work with. Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] A plea from an OpenStack user
On 08/29/2012 07:26 AM, Ryan Lane wrote: My plea is for the developers to think about how their changes are going to affect production deployments when upgrade time comes. I for one would like to see the ops bug tag used more to try and track these issues. If an upgrade makes something harder for operations people, developers at the very least should create an ops bug to fix that so that its at least tracked. Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Nova] How common is user_data for instances?
On 14/08/12 01:24, Jay Pipes wrote: Or just set the column to the LONGTEXT type and both MySQL and PostgreSQL will be just as happy. This is what I was originally aiming at -- will large deployers be angry if I change this column to longtext? Will the migration be a significant problem for them? Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Nova] How common is user_data for instances?
On 14/08/12 08:54, Jay Pipes wrote:
I was *going* to create a random-data table with the same average row
size as the instances table in Nova to see how long the migration would
take, and then I realized something... The user_data column is already
of column type MEDIUMTEXT, not TEXT:
jpipes@uberbox:~$ mysql -uroot nova -e DESC instances | grep user_data
user_data mediumtext YES NULL
So the column can already store data up to 2^24 bytes long, or 16MB of
data. So this might be a moot issue already? Do we expect user data to
be more than 16MB?
The bug reports truncation at 64kb. The last schema change I can see for
that column is Essex version 82, which has:
$ grep user_data *.py
082_essex.py:Column('user_data', Text),
http://docs.sqlalchemy.org/en/latest/dialects/mysql.html says that Text
is MySQL TEXT type, for text up to 2^16 characters.
Am I misunderstanding something here?
Mikal
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
[Openstack] [Nova] How common is user_data for instances?
Greetings. I'm seeking information about how common user_data is for instances in nova. Specifically for large deployments (rackspace and HP, here's looking at you). What sort of costs would be associated with changing the data type of the user_data column in the nova database? Bug 1035055 [1] requests that we allow user_data of more than 65,535 bytes per instance. Note that this size is a base64 encoded version of the data, so that's only a bit under 50k of data. This is because the data is a sqlalchemy Text column. We could convert to a LongText column, which allows 2^32 worth of data, but I want to understand the cost to operators of that change some more. Is user_data really common? Do you think people would start uploading much bigger user_data? Do you care? Mikal 1: https://bugs.launchpad.net/nova/+bug/1035055 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OSSA 2012-011] Compute node filesystem injection/corruption (CVE-2012-3447)
On 08/08/12 14:33, Eric Windisch wrote: The solution here may be to use libguestfs, which seems to be a modern alternative to mtools, but to use it as a non-privileged user and to forego any illusions of mounting the filesystem anywhere via the kernel or FUSE. Looking at the docs for the libguestfs python bindings, I think it provides everything I need for config drive out of the box, so that's certainly an option... Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OSSA 2012-011] Compute node filesystem injection/corruption (CVE-2012-3447)
On 08/08/12 07:38, Eric Windisch wrote: Pádraig Brady from Red Hat discovered that the fix implemented for CVE-2012-3361 (OSSA-2012-008) was not covering all attack scenarios. By crafting a malicious image with root-readable-only symlinks and requesting a server based on it, an authenticated user could still corrupt arbitrary files (all setups affected) or inject arbitrary files (Essex and later setups with OpenStack API enabled and a libvirt-based hypervisor) on the host filesystem, potentially resulting in full compromise of that compute node. Unfortunately, this won't be the end of vulnerabilities coming from this feature. Even if all the edge-cases around safely writing files are handled (and I'm not sure they are), simply mounting a filesystem is a very dangerous operation for the host. The idea had been suggested early-on to supporting ISO9660 filesystems created with mkisofs, which can be created in userspace, are read-only, and fairly safe to produce, even as root on compute host. I am in the process of re-writing the config drive code as we speak. The re-write supports (and defaults to) providing the config drive as an iso9660 image. There are two places that mounting occurs with the new code: - if the user wants a vfat config drive, as I couldn't find a way to create a vfat filesystem from a directory using userspace code. This should be relatively safe though because the filesystem which is mounted is created by the code just before the mount. [1] - if the user specifies an image from glance for the injection to occur to. This is almost certainly functionality that you're not going to like for the reasons stated above. Its there because v1 did it, and I'm willing to remove it if there is a consensus that's the right thing to do. However, file IO on this image mount is done as the nova user, not root, so that's a tiny bit safer (I hope). https://review.openstack.org/#/c/10934/ Mikal 1: I am not a security researcher. I would not play a good one on TV. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OSSA 2012-011] Compute node filesystem injection/corruption (CVE-2012-3447)
On 08/08/12 10:58, Eric Windisch wrote: This might be kind-of okay if it uses libguestfs, but I'd need to look more closely at libguestfs before considering it safe. If it is only updating vfat, another option is mtools which is entirely userspace and can be run with some safety on the host. I just realized you said glance… I'm assuming these are probably ext2/3/4 or other Linux filesystems. Libguestfs might be the best option, besides simply not having that feature. Yeah, my reading of the code is that any image format the compute node knows how to mount could be used in glance, and will then be transcoded to vfat or iso9660 before being handed to the guest. Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] compute_nodes DB table
On 06/08/12 16:47, Trinath Somanchi wrote: Hi- Can any one guide me on understanding the process that updates the compute_nodes table in nova database. nova-compute on each compute node sends a regular message to the nova-scheduler on the scheduler node with an update of its status. These are saved into that table. You can see these updates in your log: 2012-08-06 17:25:59 DEBUG nova.scheduler.host_manager [req-4549b72e-83e5-4617-9a6b-81c17609c83e None None] Received compute service update from molokai. from (pid=5003) update_service_capabilities /usr/lib/python2.7/dist-packages/nova/scheduler/host_manager.py:273 Cheers, Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Example horizon customization module?
Hi. I've been poking around trying to find an example of a horizon customization module. I haven't had a lot of luck. Does anyone have one they can point me to? Thanks heaps, Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [glance] legacy client removal and python-glanceclient
On 01/08/12 15:09, Brian Waldon wrote: I do agree that this current upgrade story could be better, but how much better do you expect it to be and in what specific ways? If the command-line interface were backwards-compatibile, would that solve all your problems? I'm fine doing that for the v1.0.0 release as long as we can deprecate all the old behavior out for a v2.0.0 release. Do you expect the legacy python library to get installed with folsom? As a side note, I do want to point out that the python module paths do not conflict. What about providing a legacy API version with the new client? Something a little like the (very not finished) example at https://review.openstack.org/#/c/10699/ ? I know it would be some work, but I don't think its more than a couple of days worth. An example usage of that proof of concept code: $ ../glance-client --os-image-api-version legacy details URI: http://192.168.1.14:9292/v1/images/6d55dd55-053a-4765-b7bc-b30df0ea3861 Id: 6d55dd55-053a-4765-b7bc-b30df0ea3861 Public: No Protected: No Name: ubuntu-bucket/oneiric-server-cloudimg-amd64-vmlinuz-generic Status: active Size: 4660272 Disk format: aki Container format: aki Minimum Ram Required (MB): 0 Minimum Disk Required (GB): 0 Owner: 8aef75b5c0074a59aa99188fdb4b9e90 Property 'image_location': ubuntu-bucket/oneiric-server-cloudimg-amd64-vmlinuz-generic.manifest.xml Property 'image_state': available Property 'architecture': x86_64 Created at: 2012-06-25T02:10:32 Updated at: 2012-06-25T02:10:36 Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Example horizon customization module?
On 01/08/12 17:43, Atul Jha wrote: Mikal, If you meant customizing the UI http://docs.openstack.org/developer/horizon/topics/customizing.html The link will help. Yep, that's how I ended up deciding I wanted a customization module. From that page: If you wish to alter dashboards or panels which are not part of your codebase, you can specify a custom python module which will be loaded after the entire Horizon site has been initialized, but prior to the URLconf construction. This allows for common site-customization requirements such as: Registering or unregistering panels from an existing dashboard. Changing the names of dashboards and panels. Re-ordering panels within a dashboard or panel group. However, the page only shows you how to pass such a module in the config file -- I can't find an example of a module upon which to base mine. So, has anyone else done this? Do you have an example customization module you'd be willing to share? Thanks, Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Example horizon customization module?
On 01/08/12 19:11, Erwan Gallen wrote:
Hi,
here you have a customization example of Gabriel :
https://github.com/gabrielhurley/horizon_demo
and docs of this demo :
http://gabrielhurley.github.com/slides/openstack/building_on_horizon/index.html
Thanks for this. I can't see an implementation of a customization module
(as specified with the following flag):
HORIZON_CONFIG = {
customization_module: my_project.overrides
}
In this demo. Am I missing it somewhere?
Thanks,
Mikal
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Qcow2 Details on base images
On 28/07/12 05:42, Gaurab Basu wrote: Hi, I am trying to figure out the technology that openstack uses when multiple VM's having the*same *base image (OS) are provisioned on a physical server. Does it use as many copy as the number of VM's or does it use the same base image and then copy on write. I need to understand the complete details. Can anybody share some details or point me to some place where I can find the details. Its pretty hard to provide a complete description of what happens, because the code keeps changing. However, assuming you have copy on write turned on (which is the default IIRC), and assuming that all of the instances have the same disk size, then you end up with: - the image as fetched from glance, with possible format conversion - that image resized to the size the instance requested - a cow on write layer for each instance that is using that sized image The first should be smallish, the second can be quite large, and the third will really depend on how much writing the instances are doing. Note that this all falls apart if instances are migrated, because as part of the migration the copy on write layer is transformed into a full disk image, which is what is shipped over to the new machine. Hope this helps, Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Individual Nominations for Foundation Board of Directors
On 26/07/12 09:44, Jonathan Bryce wrote: Elections for the Individual Directors will take place August 20-24. To vote in the initial Board election, you must join as an Individual Member by August 15. To appear on the ballot, an Individual Member must receive nominations from 10 other Individual Members. The deadline to nominate an Individual Member is August 6, and all nominations are submitted via email to secret...@openstack.org. Is there a page somewhere that details what you see these Directors doing? That would help us evaluate the candidates, as well as helping people decide if they think they'd be a good Director. Thanks, Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Improving logging
On 23/07/12 19:54, Thierry Carrez wrote: Michael Still wrote: On 21/07/12 00:08, Jay Pipes wrote: Not that I've seen, but I think it would be good to standardize on one. How about just ops? Works for me. Added to http://wiki.openstack.org/BugTags and as official tag for all core projects. Thanks! I didn't even know that wiki page existed... Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Improving logging
On 21/07/12 00:08, Jay Pipes wrote: Not that I've seen, but I think it would be good to standardize on one. How about just ops? Works for me. Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Improving logging
On 19/07/12 23:16, Gabe Westmaas wrote: There is definitely support for logging only patches that help us find problems easier. I just added openstack operators, because I think it would be good to get as much input as possible over what problems are difficult to track down. Some thoughts off the top of my head: When downloads from glance fail after multiple retries, the glance server that is failing is not in the output anywhere. Request IDs just landed in glance and it would be cool if those were logged in nova I know there are more things! I'd encourage these sorts of things to be filed as bugs when you find them. They're quick and easy to fix, but still need to be tracked. Is there a tag people are using to flag bugs as ops related? Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [nova] Proposal to add Padraig Brady to nova-core
On 19/07/12 09:09, Vishvananda Ishaya wrote: Hello Everyone! Padraig has been contributing a lot of code to all parts of nova, and has been contributing a lot to reviews[1]. I think he would make a great addition to nova-core. +1 Padraig has been a pleasure to work with for the entire time I've been associated with openstack. Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Improving logging
On 19/07/12 09:32, Eugene Kirpichov wrote: Hi, I noticed that there are quite a few cases when I found my OpenStack installation is broken and logging was not verbose enough for me to understand what exactly was broken, so I had to add more logging statements to the code and relaunch. Most of the people I know are running with debug logging turned on for this reason, which is clearly bonkers. There was discussion at the design summit about trying to promote meaningful debug messages to info messages so that deployers wouldn't need to do that. I don't think its progressed much though, as we've all be working on other stuff. However, to answer you actual question -- I'd support patches with only added logging if that logging was helpful. Cheers, Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Hashing image files
On 03/07/12 02:33, Alexey Ababilov wrote: Hi, Michael! I am curious to know: is it necessary for nova to save images in file which names are sha1 hashes from image IDs (UUIDs)? Nova saves root image under its sha1 hash: https://github.com/openstack/nova/blob/master/nova/virt/libvirt/connection.py#L1214 but it saves kernel and ramdisk under their original IDs: https://github.com/openstack/nova/blob/master/nova/virt/libvirt/connection.py#L1199 https://github.com/openstack/nova/blob/master/nova/virt/libvirt/connection.py#L1206 Is it a security measure? That code pre-dates my involvement with openstack. I'm not really sure why the hash is used, given its the hash of the id not of the contents of the file. We could rework all that code if you wanted, but it would be the third or fourth rewrite in two releases, so I feel a little like we should let it sit for a bit before changing it again. Cheers, Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Problems with sqlalchemy migrate and foreign key constraints
Hi. I've spent ages on this and can't get myself unstuck. I have a DB
schema update like this:
def upgrade(migrate_engine):
meta = MetaData()
meta.bind = migrate_engine
instances = Table('instances', meta, autoload=True)
instance_info_caches = Table('instance_info_caches', meta,
autoload=True)
# We need to remove the foreign key constraint or the column rename
# will fail
fkeys = list(instance_info_caches.c.instance_id.foreign_keys)
try:
fkey_name = fkeys[0].constraint.name
ForeignKeyConstraint(
columns=[instance_info_caches.c.instance_id],
refcolumns=[instances.c.uuid],
name=fkey_name).drop()
except Exception:
LOG.error(_(foreign key constraint couldn't be removed))
raise
instance_info_caches.c.instance_id.alter(name='instance_uuid')
# Add the foreign key constraint back
try:
ForeignKeyConstraint(
columns=[instance_info_caches.c.instance_uuid],
refcolumns=[instances.c.uuid]).create()
except Exception:
LOG.error(_(foreign key constraint couldn't be created))
raise
This gives me this error:
[snip]
2012-06-22 11:18:33 TRACE nova File
/opt/stack/nova/nova/db/sqlalchemy/migrate_repo/versions/105_instance_info_caches_uses_uuid.py,
line 53, in upgrade
2012-06-22 11:18:33 TRACE nova
columns=[instance_info_caches.c['instance_uuid']],
2012-06-22 11:18:33 TRACE nova File
/usr/lib/python2.7/dist-packages/sqlalchemy/util.py, line 797, in
__getitem__
2012-06-22 11:18:33 TRACE nova return self._data[key]
2012-06-22 11:18:33 TRACE nova KeyError: 'instance_uuid'
2012-06-22 11:18:33 TRACE nova
It looks to me like the old column name is being left in
instance_info_caches.c, although trying to use the old name doesn't work
as well. I suspect this is because the SQL for the alter hasn't actually
run yet.
Is there some way to do this that I can't think of, or should I put the
new foreign key constraint in a separate DB migration? The problem with
that path is I'm not sure how to do the downgrade() step, as it would
have the same problem in reverse.
Thanks,
Mikal
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
[Openstack] Examples of authentication using the glance REST API?
Hi. I am trying to find an example of how to do authentication using the glance REST API. I assume there are some HTTP headers that I need to set, but a pointer to some existing code which does this would be appreciated. This is part of porting the glance replicator to use the REST API instead of the client library. Thanks heaps, Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] nova.projects table problems
Has anyone else experienced something like this? Its happened to me twice now (on the same machine) and I have no idea what is causing it: mysql show tables like 'projects'; +---+ | Tables_in_nova (projects) | +---+ | projects | +---+ 1 row in set (0.00 sec) mysql describe projects; ERROR 1146 (42S02): Table 'nova.projects' doesn't exist mysql repair table projects; +---++--+-+ | Table | Op | Msg_type | Msg_text| +---++--+-+ | nova.projects | repair | Error| Table 'nova.projects' doesn't exist | | nova.projects | repair | status | Operation failed| +---++--+-+ 2 rows in set (0.00 sec) I'm a bit confused. Thanks, Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] nova.projects table problems
On 01/06/12 09:54, Michael Still wrote: Has anyone else experienced something like this? Its happened to me twice now (on the same machine) and I have no idea what is causing it: mysql show tables like 'projects'; +---+ | Tables_in_nova (projects) | +---+ | projects | +---+ 1 row in set (0.00 sec) mysql describe projects; ERROR 1146 (42S02): Table 'nova.projects' doesn't exist mysql repair table projects; +---++--+-+ | Table | Op | Msg_type | Msg_text| +---++--+-+ | nova.projects | repair | Error| Table 'nova.projects' doesn't exist | | nova.projects | repair | status | Operation failed| +---++--+-+ 2 rows in set (0.00 sec) I'm a bit confused. This turns out to be https://bugs.launchpad.net/ubuntu/+source/nova/+bug/975085 / https://bugs.launchpad.net/nova/+bug/993663 I think. Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Glance] Replication implementations
On 04/05/12 20:31, Eoghan Glynn wrote: Sorry for the slow reply, I've been trapped in meetings. [snip] So the way things currently stand, the EC2 image ID isn't really capable of migration. I was thinking however that we should change the EC2 image generation logic, so that there is a reproducible glance UUID - EC2 mapping (with a small chance of collision). This change would allow the same EC2 ID to be generated in multiple regions for a given glance UUID (modulo collisions). Would that be helpful in your migration use-case? I do think this is a good idea. Or even if the column wasn't auto-increment, but just picked a random number or something (because that would be marginally less likely to clash). Without somehow making these ec2 ids more global, replication between regions is going to suffer from ec2 api users having to somehow perform a lookup out of band. Now, my use case is a bit special, because I can enforce that images are only ever uploaded to one master region, and then copied to all others. I think that's probably not true for other users though. Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Glance] Replication implementations
On 04/05/12 04:07, Brian Waldon wrote: I think the path forward is first to gather information on what people are already doing w.r.t. replication, which you have helped trigger with this email. I'm definitely interested in seeing your solution. Once we get this information out in the open, we need to explore these existing solutions can be included with Glance. I'd prefer to go with an external tool that handles replication rather than baking it directly into the code right now, but if the work has already been done, I'm more than happy to consider it. My code is a separate tool, and I agree that's the way to go. It makes it much easier for systems integrators to tweak how it works to meet their needs, and if we find something that you can't do via the API that probably indicates a gap in the API more than anything else. I will try and make some time early next week to clean up the code enough to show it to you without you laughing at me. Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [Glance] Replication implementations
Hi. I'd be interested in hearing from people who have implemented some form of replication with glance. I'm especially interested in how you went about it. I attended the session at the dev summit, but that was forward looking, and I am pretty sure that there wasn't any mention of current implementations. I'll get the ball rolling -- I have just implemented API level replication for glance for an internal project. It uses the glance API to replicate images from a master glance to slave glances. Its not very smart at the moment, it just copies everything. Current warts: - no support for metadata update - no support for filtering of what is replicated - maintaining amazon ec2 ids across regions requires twiddling the nova database where this mapping is stored Advantages of this approach: - API level replication means that I can run different versions of glance at each end of the replication relationship - it really wasn't that hard to do Would other people be interested in getting this code into glance, probably as a stand alone command line tool? Or has someone done something much better that I should be using instead? Cheers, Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Pending reviews
On 26/04/12 02:28, Mandar Vaze / मंदार वझे wrote: It would be great if someone can spare some time to have a look at these : https://review.openstack.org/#/c/6451/ : I've addressed comments from first review cycle - Second patch set needs to be reviewed and approved https://review.openstack.org/#/c/6452/ : Brad Hall reviewed - But more review and/or approval needed. https://review.openstack.org/#/c/6076/ : Chuck Short and Kevin Mitchell said Looks good - But approval still needed. I'm in the same state, with many reviews which need eyeballs. Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Spelling error on www.openstack.org
I'm not sure who to report this to, but I just noticed that the yellow graphic at the top left of www.openstack.org proudly announces that essex was relased recently. I hope it develops laser based super powers. That would be cool. Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Spelling error on www.openstack.org
On 09/04/12 11:07, Mark Collier wrote: Todd morey (t...@openstack.org) is your friendly neighborhood webmaster and laser operator. Todd, can I please have a laser too? Michael Still michael.st...@canonical.com wrote: I'm not sure who to report this to, but I just noticed that the yellow graphic at the top left of www.openstack.org proudly announces that essex was relased recently. I hope it develops laser based super powers. That would be cool. Mikal Thanks, Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Spelling error on www.openstack.org
On 09/04/12 14:18, Todd Morey wrote: Mikal, are you running Essex? It already has them. (See release notes under World Domination Enhancements.) Careful now. The release notes are a wiki, and you might end up with a World Domination section... Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack-poc] [Bug 930270] Re: flagfile interpolation breaks instance_name_template
** Summary changed:
- flagfile interpolation breaks instance_name_tempalte
+ flagfile interpolation breaks instance_name_template
--
You received this bug notification because you are a member of OpenStack
Common Drivers, which is the registrant for openstack-common.
https://bugs.launchpad.net/bugs/930270
Title:
flagfile interpolation breaks instance_name_template
Status in OpenStack Compute (Nova):
Fix Committed
Status in openstack-common:
Fix Committed
Bug description:
When --volume_name_template=volume-%08x and
--instance_name_template=instance-%08x are present in nova.conf,
ConfigParser attempts to interpolate the '%' strings. There are meant
for nova to interpolate.
2012-02-10 10:54:49,089 DEBUG nova.service [-] osapi_compute_link_prefix :
None from (pid=22867) wait /opt/stack/nova/nova/service.py:409
2012-02-10 10:54:49,089 CRITICAL nova [-] '%' must be followed by '%' or '(',
found: '%08x'
(nova): TRACE: Traceback (most recent call last):
(nova): TRACE: File /opt/stack/nova/bin/nova-api, line 53, in module
(nova): TRACE: service.wait()
(nova): TRACE: File /opt/stack/nova/nova/service.py, line 402, in wait
(nova): TRACE: flag_get = FLAGS.get(flag, None)
(nova): TRACE: File /opt/stack/nova/nova/flags.py, line 73, in get
(nova): TRACE: value = getattr(self, name)
(nova): TRACE: File /opt/stack/nova/nova/flags.py, line 70, in __getattr__
(nova): TRACE: return getattr(self._conf, name)
(nova): TRACE: File /opt/stack/nova/nova/openstack/common/cfg.py, line
784, in __getattr__
(nova): TRACE: return self._substitute(self._get(name))
(nova): TRACE: File /opt/stack/nova/nova/openstack/common/cfg.py, line
985, in _get
(nova): TRACE: return opt._get_from_config_parser(self._cparser, section)
(nova): TRACE: File /opt/stack/nova/nova/openstack/common/cfg.py, line
433, in _get_from_config_parser
(nova): TRACE: return cparser.get(section, self.dest)
(nova): TRACE: File /usr/lib/python2.7/ConfigParser.py, line 615, in get
(nova): TRACE: return self._interpolate(section, option, value, d)
(nova): TRACE: File /usr/lib/python2.7/ConfigParser.py, line 683, in
_interpolate
(nova): TRACE: self._interpolate_some(option, L, rawval, section, vars, 1)
(nova): TRACE: File /usr/lib/python2.7/ConfigParser.py, line 724, in
_interpolate_some
(nova): TRACE: '%%' must be followed by '%%' or '(', found: %r %
(rest,))
(nova): TRACE: InterpolationSyntaxError: '%' must be followed by '%' or '(',
found: '%08x'
(nova): TRACE:
stack@356591-essex-k1:/opt/stack/nova$
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/930270/+subscriptions
___
Mailing list: https://launchpad.net/~openstack-poc
Post to : openstack-poc@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack-poc
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Google Summer of Code-2012
On 10/02/12 02:59, Armaan wrote: Hi folks, Google has announced Google Summer of Code -2012, /Google Summer of Code/ is a global program that offers student developers stipends to write code for various open source software projects. More info is available here http://code.google.com/soc/. I wanted to know if there is any possibility of Openstack participating in /Google Summer of Code/. I'd be happy to mentor as well, although it sounds like there is no shortage of mentors. Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Image cache management - FFE
Hi. https://blueprints.launchpad.net/nova/+spec/nova-image-cache-management This change has been in code review for a while (https://review.openstack.org/#change,2902). It didn't make it in before the feature freeze because I got called away and didn't have a chance to address the last few review comments. The change stops the unbounded growth of libvirt image caches on compute nodes, which operators currently have to manually clean up out of band. Thanks, Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Nova] Essex dead wood cutting
On 28/01/12 02:23, Thierry Carrez wrote: Just as Nova enters feature freeze, it sounds like a good moment to consider removing deprecated, known-buggy-and-unmaintained or useless feature code from the Essex tree. Here are my suggestions for removal: - Hyper-V support (known broken and unmaintained) Given that hyper-v appears to be supported by libvirt, wouldn't it be ok to drop the direct support in return for that? I've taken a look at the spawn() code and I can't immediately see anything which would stop that from working. Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
