Re: [Openstack] Image prep. cloud-init user configuration help.
On Wed, 24 Jul 2013, Jake G. wrote: Thanks Scott! I have removed cloud-init from my instance and enabled config-drive in the nova.conf file. I don't think I suggested you remove cloud-init. Just enable config drive and cloud-init will use that instead of the ec2 metadata service. You really do not want to remove cloud-init. Anything else I should take notice of? As others have pointed out, the fact that your instances can't reach the metadata service shows that at least *something* is not functioning correctly on your installation. From: Scott Moser smo...@ubuntu.com To: Jake G. dj_dark_jungl...@yahoo.com Cc: Dean Troyer dtro...@gmail.com; openstack@lists.launchpad.net openstack@lists.launchpad.net Sent: Thursday, July 25, 2013 10:58 AM Subject: Re: [Openstack] Image prep. cloud-init user configuration help. On Wed, 24 Jul 2013, Jake G. wrote: I got the user correct now I am wondering why this is nessasary for my environment. I am not using Amazon EC2 services so why would I need to get metadata from there API? Can someone explain? Cloud-init uses the datasource of the EC2 Metadata service, that Openstack provides a workalike for. That metadata service has things like instance-id and hostname that are useful. cloud-init uses instance-id for run-once-per-instance functionality. cloud-init also gets user-data from the datasource, and that is very useful for initializing images with information provided at launch time by the user. If you enable config-drive in openstack, then cloud-init will find the same data above from the config-drive, and wont bother looking for the ec2 metadata service. Scott___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Image prep. cloud-init user configuration help.
On Thu, 25 Jul 2013, Brian Schott wrote: When you bundle the image, make sure you remove the /var/lib/cloud directory. It contains lock files and a local cache from the first time it was run. This is not actually necessary. cloud-init runs some things per-instance, and other things per-boot. The per-instance is handled using the instance-id as a marker. So, unless you hit an instance-id (uuid) collision there is no need to specifically clear out /var/lib/cloud. If you have some other reason that you think this needs to be done, please open a bug on cloud-init. Scott - Brian Schott, CTO Nimbis Services, Inc. brian.sch...@nimbisservices.com ph: 443-274-6064 fx: 443-274-6060 On Jul 24, 2013, at 11:00 PM, Jake G. dj_dark_jungl...@yahoo.com wrote: Thanks Scott! I have removed cloud-init from my instance and enabled config-drive in the nova.conf file. Anything else I should take notice of? From: Scott Moser smo...@ubuntu.com To: Jake G. dj_dark_jungl...@yahoo.com Cc: Dean Troyer dtro...@gmail.com; openstack@lists.launchpad.net openstack@lists.launchpad.net Sent: Thursday, July 25, 2013 10:58 AM Subject: Re: [Openstack] Image prep. cloud-init user configuration help. On Wed, 24 Jul 2013, Jake G. wrote: I got the user correct now I am wondering why this is nessasary for my environment. I am not using Amazon EC2 services so why would I need to get metadata from there API? Can someone explain? Cloud-init uses the datasource of the EC2 Metadata service, that Openstack provides a workalike for. That metadata service has things like instance-id and hostname that are useful. cloud-init uses instance-id for run-once-per-instance functionality. cloud-init also gets user-data from the datasource, and that is very useful for initializing images with information provided at launch time by the user. If you enable config-drive in openstack, then cloud-init will find the same data above from the config-drive, and wont bother looking for the ec2 metadata service. Scott ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Image prep. cloud-init user configuration help.
On Tue, 23 Jul 2013, Dean Troyer wrote: On Tue, Jul 23, 2013 at 1:49 AM, Jake G. dj_dark_jungl...@yahoo.com wrote: I am following the part on how to set up cloud-init from the below guide, but I was wondering how to change the user? I do not have the default ec2-user on my instance image. Do I just create this user in the instance? What permissions do I give the user? About half way down this page (http://hackstack.org/x/blog/2013/04/25/a-centos-6-image-for-openstack/) is a link to a centos 6 kickstart file that configures cloud-init during the kickstart %POST phase to behave similar to how it works on Ubuntu images by default: create a 'centos' user with sudo permissions to root. Josh was right, though. Recent versions of cloud-init will create the user if it is not there. If the user is present, it will not modify it. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Image prep. cloud-init user configuration help.
On Wed, 24 Jul 2013, Jake G. wrote: I got the user correct now I am wondering why this is nessasary for my environment. I am not using Amazon EC2 services so why would I need to get metadata from there API? Can someone explain? Cloud-init uses the datasource of the EC2 Metadata service, that Openstack provides a workalike for. That metadata service has things like instance-id and hostname that are useful. cloud-init uses instance-id for run-once-per-instance functionality. cloud-init also gets user-data from the datasource, and that is very useful for initializing images with information provided at launch time by the user. If you enable config-drive in openstack, then cloud-init will find the same data above from the config-drive, and wont bother looking for the ec2 metadata service. Scott ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Can I run qemu-kvm in an OpenStack Instance
On Fri, 14 Jun 2013, laclasse wrote: Yes, the Ubuntu cloud image is made to run as a guest atop an hypervisor and it makes sense to optimize it by removing the unlikely needed modules that usually require hardware to run (nested virt is not yet common). Scott Moser may confirm/infirm. Maybe you can try installing the 'normal' kernel on your guest that has the vmx flag exported in its vm config and try loading it again. On that Ubuntu guest, you can also install cpu-checker that gives you a 'kvm-ok' cli: # kvm-ok INFO: /dev/kvm exists KVM acceleration can be used Hope this helps. I saw this thread, and was about to respond, but Robert Collin's response was correct: install linux-image-generic, it will bring in linux-image-extra-$version-generic which has kvm. Alternatively, just: apt-get install linux-image-virtual-extra I think that then you have to do a 'modprobe kvm_intel ; modprobe kvm_amd' also, but a reboot would also fix it. The reason you have to modprobe is that the udev events that would fire to automatically load those modules have already fired on boot. Then, happily use kvm. I use kvm all the time in nested virt on openstack. Now, if I could only get an instance that supported nested virt from a public cloud provider that would ROCK ... I'd happily pay additional $/hr for 'm1.large-nested-virt' type. Robert / HP / dreamhost / Rackspace are you listening? ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] KVM on Ubuntu server 13.04
On Thu, 25 Apr 2013, skible.openst...@gmail.com wrote: Hi, Was anyone able to install kvm package on ubuntu server 13.04. It keeps on switching to another package named qemu-system-x86 Message returned when i do apt-get install kvm = Note, selecting 'qemu-system-x86' instead of 'kvm' Without KVM, i can't create VMs ! Did you *actually* find a problem or just notice that 'apt-get install kvm' installs qemu-system-x86. There should not be any fallout of this. If you see any functional issues, then please open a bug in launchpad with 'ubuntu-bug qemu-system-x86'. Scott ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] How to create a Image which can extend the partition automatically.
On Wed, 6 Mar 2013, Pádraig Brady wrote: On 03/05/2013 02:04 PM, Scott Moser wrote: On Tue, 5 Mar 2013, Sylvain Bauza wrote: If you look at http://www.pixelbeat.org/docs/openstack_libvirt_images/#1361764412, you'll see that resize2fs is performed. But there is a caveat with RHEL6 which is Linux 2.6 (contrary to Ubuntu 12.04 which is Linux 3.0). If you look at man resize2fs : Please don't do this, or rely on this. Having the hypervisor do this for your guest is simply wrong. Hypervisors should know little to nothing about the internals of the instances they're launching. Just to point out the alternative for when the VM doesn't have the smarts to resize itself, is to use something like virt-resize: https://blueprints.launchpad.net/nova/+spec/resize-no-raw Using libguestfs is worlds better than having the host directly do it. But really whats happening here is *still* something with very little information mucking with (and possibly breaking) the inside of a disk image. The more we consider that a bucket of bytes the better. If you are using libguestfs, chances are you're using it from your distro, and patching it to deal with a new filesystem (or otherwise get the update to your hostOS) is still problematic. Just let/expect the guest do it, and we'll avoid a whole silly game of cat and mouse that doesn't even have to be played. When was the last time you updated your bios on your laptop so you could use a new linux filesystem? That sounds silly, doesn't it. Having openstack reach inside the contents of the disk is just as silly. One of the major benefits of having cloud-init direct the partition resize *and* subsequent filesystem resize is that the user can (in user-data) disable this! (currently the initramfs doesn't take any instruction from user-data, so disabling it isn't really a possibility). perhaps they didn't want that extra instance-store space to be part of the root filesystem. If you put that function inside the hypervisor, you either can't do it, of you have to expose some silly api-launch parameter of do_not_modify_disk. It complicates the API and complicates the host.___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] initramfs-growroot or LVM
On Sat, 9 Feb 2013, Davide Guerri wrote: Brady, thanks for the infos and for the bugzilla link. I made some tests and some some other researches about potential performance penalties of LVM. These seem not to be noticeable especially with recent linux versions. (please see for instance [1]). Now I'm even more curious about the choice of canonical: there must be a good reason for using such rough (pass me the term) filesystem resize technique over a more clean and practical lvm resize. LVM is a wonderfully useful technology for making multiple physical volumes appear as a single block device. LVM could definitely work here and arguably results in a more feature-laiden selection for the block layer of a root filesystem. However, for a cloud-guest, the environment is just massively different than on a physical server. You're no longer limited by the fact that your server has 3 physical disks each 360G in size. You're not exposed to that. You have a root device that can be anywhere from 1G to some large number. You can attach more of those same block devices later on with an API call (as opposed to a sys-admin or call to a storage vendor). To be clear, I'm *not* saying LVM is not suited for the task of being the block layer. I'm saying it is less necessary in a cloud world. Thanks to Juerg, kernel, and util-linux, With the next version of cloud-init and growpart, we'll be able to resize the root filesystem on a partition outside of the initramfs. I'll gladly accept patches or bugs to cloud-init if that doesn't work for root on LVM disks. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] cirros 0.3.1 released
Hi, I just thought I'd announce cirros 0.3.1 here. The biggest feature is config-drive-v2 support, but a more complete list is below. Download images at http://download.cirros-cloud.net/0.3.1 I'm interested in feedback on how they work. - move to buildroot 2012.05 (busybox 1.20.1) - support https via curl/openssl (LP: #918702) - ssh-import-id uses https directly to landscape.net - support mounting of vfat filesystems (LP: #929841) - support acpi shutdown (LP: #944151) - ec2metadata: remove double '/' in --public-keys requests (LP: #992492) - upgrade kernel to Ubuntu 12.04 LTS kernel (3.2.0-25.40) - mount first ephemeral device to /mnt in openstack instances (LP: #1022619) - ec2 data source has socket timeouts (10 second) when looking for metadata service (LP: #1022600) - wait longer than 3 seconds (60) for dhcp response on eth0 (LP: #918699) - support openstack config-drive-v2 data source (LP: #1097488) including insertion of interfaces(5) - support NoCloud data source, as supported by cloud-init (LP: #918375) Scott Moser ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Configuring instances during first boot
On Tue, 29 Jan 2013, Juerg Haefliger wrote: On Tue, Jan 29, 2013 at 3:03 PM, Pádraig Brady p...@draigbrady.com wrote: On 01/29/2013 01:08 PM, JuanFra Rodriguez Cardoso wrote: Thanks for your replies!! Has anyone tried to resize root disk of Centos/Fedora instances with cloud-init? I assume you're referring to resizing the root partition? Juerg Haefliger is working on cloud-utils and cloud-initramfs-tools packages for Fedora/Red Hat, which support the growroot feature. He might have some packages for you to test. Yes working on packaging for RHEL and adding support for GPT partition tables for Fedora. I could give you preliminary RHEL6 packages but Fedora is not quite there yet. Let me know. Juerg, I'm interested/willing to help with this. I'd like for growpart to have support for GPT partitions also. Please let me know how I can help. One exciting thing is that newer kernels now support 'update' of a partition table, that should be a good fit for simplistic growing like we're interested in. util-linux has recently gained the ability to tell the kernel to update its partition table view at [1]. What that means is that going forward we won't have to do this stuff inside the initramfs, and I'd like for cloud-init to be able to drive it itself. -- [1] http://git.kernel.org/?p=utils/util-linux/util-linux.git;a=commitdiff;h=3b905b794e93609af7e42459d32b27e7c18ce02e ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Key injection failure on boot
On Fri, 11 Jan 2013, David Kranz wrote: Thanks Vish, but I am still a little confused. I am using an ubuntu precise cloudimg and normally when I pass a keyname to boot, the public key shows up in ~ubuntu/.ssh/authorized_keys. If you've installed a newer version of python-boto into the image, it would be possible that you're hitting bug 1068801 (http://pad.lv/1068801). But the stock Ubuntu 12.04 images should be using a suitably old version of boto. Other than that, I've not seen any issue like this in cloud-init. Looking at the console log, I presume it is the guest cloud-init that is doing that. But sometimes not. This has to be a bug some where even if it is not in nova. There is a lot of mechanism here that I don't understand. If there is documentation some where about exactly how to use metadata to install an ssh key I can't find it. Do you have any more advice? -David On 1/11/2013 1:32 PM, Vishvananda Ishaya wrote: Key name is the recommended method, but injecting it into the guest is not. The key should be downloaded from the metadata server using a guest process like cloud-init. Vish On Jan 11, 2013, at 10:20 AM, David Kranz david.kr...@qrclab.com wrote: Sometimes when I boot a bunch of vms seconds apart, using the key_name argument, some instance will not have its key injected. I found a bug ticket marked won't fix with a comment from Vish that key injection was for developer convenience[1]. Of course the personality argument could also be used to inject the file. This is odd because key_name is a documented part of nova client, as the files mechanism. So what is the recommended way to do what the key_name argument is documented to do? I think if key_name is not intended to work it should be removed from nova client. -David [1] https://bugs.launchpad.net/nova/+bug/967994 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Instance flavor root disk partition problem
On Tue, 11 Dec 2012, João Soares wrote: Hi, I have set up a CentOS VM according to this tutorial https://www.ibm.com/developerworks/mydeveloperworks/wikis/home/wiki/OpenStac k/page/Creating%20qcow2%20CentOS%20Image%20for%20OpenStack?lang=en I uploaded it to glance and can boot up a nice instance (1cpu, 1GB RAM). The disk size is 10GB because that is the size I set up for the image. However my problem comes when I want to set a root disk, e.g. 15GB. The Instance boots normally, if I do an fdisk ?l I can see that the size of the disk is 15GB however the partition still has the original size, i.e. 10GB, and if a do a df ?h it in fact only says I have 10GB. This does not happen for example with the cirros image. How can I make the 15GB available on the VM when it boots? When an instance's disk is resized up, zeros are just added to the end. As a result the new instance will still have 2 issues to address. a.) have a partition table that only covers the original size To address this, both Ubuntu images and cirros use 'growpart' from cloud-utils. Due to a limitation in the linux kernel, you cannot modify a partition table of a disk that has partition currently mounted (you can for lvm, but not for raw disks). So, this has to happen inside the initramfs before the root volume is mounted, or a reboot has to be done to free the mount of /. (Note, I've been told that newer kernels have a mechanism for growing a partition while its mounted, but I've not really looked into it. Also growpart doesn't/wont work for GPT partitions... patches welcome) b.) have a filesystem that fills that original size. cirros runs resize2fs on the root partition once per instance. In Ubuntu images, that is handled by cloud-init, but in order for resize2fs to do anything, 'a' must have been addressed first.___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Nova] Got 'qemu-img: error while writing sector 634880: Input/output error' in _create_image
On Mon, 24 Sep 2012, Wangpan wrote: Thanks for your attention! - disk full # I believe there is enough space, because I have tried to boot serveral instances after this error with the same image, they are all booted successfully. - permission issues# Not this reason, too. explain as above. I still believe this is a race problem, if we convert to image from qcow2 to raw at a short interval, one of the convertion may fail, I have verified this by running two same commands continuously such as: qemu-img convert -f qcow2 -O raw test.qcow2 test.part and one of them give me the same error: qemu-img: error while writing sector 1052672: Input/output error I would expect the above. Or at least it surprises me very little. I doubt that qemu-img expects some other process to write to its output file while it is. Surely nothing in openstack is expecting to be able to have 2 instances of 'qemu-img convert' write to the same file concurrently, right? ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Keyring support in openstack
On Sat, 28 Jul 2012, Bhuvaneswaran A wrote: Team, As per patch https://review.openstack.org/#/c/9497/ we are adding keyring support for openstack client. If password is not specified in command line or environment variable, the user is prompted to enter password. During this time, the password is stored in keyring. During next time, the password is read from keyring, instead of prompt. It is true, if password is not specified in command line or environment variable. This behavior is documented in this wiki page: http://wiki.openstack.org/KeyringSupport I haven' tried this specifically for the openstack client, but when this went into nova, it annoyed me, as I started having to give a password on remote systems every time. For the devstack instances I was working on, I honestly couldn't care less about security, and wanted to not be bothered. For others looking for something similar, here is the 'keyringrc.cfg' file that you need. Put it either in ~/ or in the current working directory (strange). --- keyringrc.cfg --- # This is an example keyringrc.cfg file that allows python-keyring # to use the UncryptedFileKeyring. # See /usr/share/doc/python-keyring/README.gz for more info. # Note, this works, but I see the following significant issues with it: # * python-keyring goes looking for 'keyringrc.cfg' in the current working # directory and user's home directory. Note, specifically it does *not* # look for ~/.keyringrc.cfg (which would be more common) # * no environment variable can affect the path read for ~/keyringrc.cfg # this means you're stuck with one of the following options if you # wanted to somehow maintain that one app (that you do not care about) # use UncryptedFileKeyring while all other apps use a different: #* keep 'HOME' environment variable set when that app is used to # something other than your home. and maintain $HOME/keyringrc.cfg #* keep the current working directory when that app is used to a given # working directory. # I think it'd be much nicer if I could affect this with # 'PYTHON_KEYRING_CFG' # or if the config file had some apt specific stuff. # * UncryptedFileKeyring writes its data to a file named keyring_pass.cfg # in the home directory, and does not pay attention to 'keyring-path' # * UncryptedFileKeyring laughably creates keyring_pass.cfg its password # file with default umask meaning in most cases it is world readable # I've opened bug http://pad.lv/1023433 to track these. [backend] default-keyring=keyring.backend.UncryptedFileKeyring keyring-path=/home/ubuntu/xxx # this is ignored ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [openstack-dev] [nova] Disk attachment consistency
On Mon, 13 Aug 2012, Vishvananda Ishaya wrote: Hey Everyone, Resulting Issues a) The device name only makes sense for linux. FreeBSD will select different device names, and windows doesn't even use device names. In addition xen uses /dev/xvda and kvm uses /dev/vda b) The device sent in kvm will not match where it actually shows up. We can consistently guess where it will show up if the guest kernel is = 3.2, otherwise we are likely to be wrong, and it may change on a reboot anyway Long term solutions -- We probably shouldn't expose a device path, it should be a device number. This is probably the right change long term, but short term we need to make the device name make sense somehow. I want to delay the long term until after the summit, and come up with something that works short-term with our existing parameters and usage. The first proposal I have is to make the device parameter optional. The system will automatically generate a valid device name that will be accurate for xen and kvm with guest kernel 3.2, but will likely be wrong for old kvm guests in some situations. I think this is definitely an improvement and only a very minor change to an extension api (making a parameter optional, and returning the generated value of the parameter). (review at https://review.openstack.org/#/c/10908/) The second proposal I have is to use a feature of kvm attach and set the device serial number. We can set it to the same value as the device parameter. This means that a device attached to /dev/vdb may not always be at /dev/vdb (with old kvm guests), but it will at least show up at /dev/disk/by-id/virtio-vdb consistently. This is the right way to do this. Expose 'serial-number' (or some other name for it) in the API, attach the device with that serial number and get out of the way. If the user doesn't provide you one, then create a unique one (at least for that guest) and return it. For many use cases, a user attaches a disk, ssh's in, finds the new disk, and uses it. Don't burden them with coming up with a naming/uuid scheme for this parameter if they dont want to. Does xen have anything like this? Can you set the serial number of the xen block device? (review coming soon) First question: should we return this magic path somewhere via the api? It would be pretty easy to have horizon generate it but it might be nice to have it show up. If we do return it, do we mangle the device to always show the consistent one, or do we return it as another parameter? guest_device perhaps? From the api perspective, I think it makes most sense to call it what it is. Don't make any promises or allusions to what the guest OS will do with it. Second question: what should happen if someone specifies /dev/xvda against a kvm cloud or /dev/vda against a xen cloud? I see two options: a) automatically convert it to the right value and return it b) fail with an error message In EC2, this fails with an error message. I think this is more correct. The one issue here is that you really cannot, and should not attempt to guess or know what the guest has named devices. Thats why we're we have this problem in the first place. So, I dont have strong feelings either way on this. Its broken to pass 'device=' and assume that means something. Third question: what do we do if someone specifies a device value to a kvm cloud that we know will not work. For example the vm has /dev/vda and /dev/vdb and they request an attach at /dev/vdf. In this case we know that it will likely show up at /dev/vdc. I see a few options here and none of them are amazing: a) let the attach go through as is. advantages: it will allow scripts to work without having to manually find the next device. disadvantages: the device name will never be correct in the guest b) automatically modify the request to attach at /dev/vdc and return it advantages: the device name will be correct some of the time (kvm guests with newer kernels) disadvantages: sometimes the name is wrong anyway. The user may not expect the device number to change c) fail and say, the next disk must be attached at /dev/vdc: advantages: explicit disadvantages: painful, incompatible, and the place we say to attach may be incorrect anyway (kvm guests with old kernels) I vote 'a'. Just be stupid. Play it simple, don't believe that you can understand what device naming convention the guest kernel and udev have decided upon. Heres an example. Do you know what happens if I attach 26 devices? /dev/vd[a-z], then what? I'm pretty sure it goes to /dev/vd[a-z][a-z], but its not worth you trying to know that. That convention may not be followed for xen block devices. At one point (maybe only with scsi attached disks, letters were never re-used, so an attach, detach, attach would end up going /dev/vdb, dev/vdc, /dev/vdd). There is no binary api that linux and udev promise on this, so
Re: [Openstack] [OSSA 2012-011] Compute node filesystem injection/corruption (CVE-2012-3447)
On Wed, 8 Aug 2012, Michael Still wrote: On 08/08/12 07:38, Eric Windisch wrote: Pádraig Brady from Red Hat discovered that the fix implemented for CVE-2012-3361 (OSSA-2012-008) was not covering all attack scenarios. By crafting a malicious image with root-readable-only symlinks and requesting a server based on it, an authenticated user could still corrupt arbitrary files (all setups affected) or inject arbitrary files (Essex and later setups with OpenStack API enabled and a libvirt-based hypervisor) on the host filesystem, potentially resulting in full compromise of that compute node. Unfortunately, this won't be the end of vulnerabilities coming from this feature. Even if all the edge-cases around safely writing files are handled (and I'm not sure they are), simply mounting a filesystem is a very dangerous operation for the host. The idea had been suggested early-on to supporting ISO9660 filesystems created with mkisofs, which can be created in userspace, are read-only, and fairly safe to produce, even as root on compute host. I am in the process of re-writing the config drive code as we speak. The re-write supports (and defaults to) providing the config drive as an iso9660 image. There are two places that mounting occurs with the new code: - if the user wants a vfat config drive, as I couldn't find a way to create a vfat filesystem from a directory using userspace code. This should be relatively safe though because the filesystem which is mounted is created by the code just before the mount. [1] For sufficiently archaic filesystems, there are user-space tools that understand how to manipulate them. vfat has reached such a state. mtools can read and write vfat filesystem without any help from the kernel. Heres an example of how to use it as I've been confused by the documentation before. # get a directory to copy $ wget https://launchpad.net/python-novaclient/trunk/2.6.10/+download/python-novaclient-2.6.10.tar.gz $ tar -xvzf python-novaclient-2.6.10.tar.gz $ dir=python-novaclient-2.6.10 # create a 64M file and put vfat filesystem on it $ img=my-vfat.img $ rm -f $img; $ truncate --size 64M $img; $ mkfs.vfat -n MY-LABEL $img # copy contents to image and then off that to a new dir $ mcopy -ospmi $img $dir/* :: $ mcopy -ospmi $img :: $dir.new $ du -hs $dir.new $dir 736K python-novaclient-2.6.10.new 736K python-novaclient-2.6.10 $ diff -Naur $dir $dir.new echo the same the same Outside of the dependency on mtools, this seems like a much better solution for limited vfat filesystem writing than libguestfs or kernel mount. I carried this example on to actually boot a kvm with my-vfat.img attached and mounted it and it looked good there also. the only issue I saw in this was the timestamp on the directory python-novaclient-2.6.10.new. - if the user specifies an image from glance for the injection to occur to. This is almost certainly functionality that you're not going to like for the reasons stated above. Its there because v1 did it, and I'm Either you or I are reading the existing nova/virt/libvirt/driver.py 'config_drive_id' code incorrectly. That code and nova/compute/api.py:_create_instance seem to me to indicate if config_drive_id is true than config_drive is not true. the mount and add path is only taken if config_drive is true. willing to remove it if there is a consensus that's the right thing to do. However, file IO on this image mount is done as the nova user, not root, so that's a tiny bit safer (I hope). https://review.openstack.org/#/c/10934/ As I've said in the review, I think this function should not exist in config-drive-v2. I do think that attach volume by volume id at boot function should be added to nova (it may already be), but it has no relevance to config-drive. Scott___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [nova] a proposal to change metadata API data
On Sat, 21 Jul 2012, Matt Joyce wrote: Preamble: Until now, all data that is made available by the metadata server has been data that cannot be found anywhere else at the time it may be needed. In short, an instance can't be passed it's instance id before it's instance id has been allocated so a user cannot pass it to an instance that is being started up. So whether a user wants to jump through a few hoops or not to pass their instance the instance id of itself... they simply cannot without metadata api being there to provide it at creation time. This means that the metadata server holds an uneasy place as a necessary clearing house ( evil? ) of data that just doesn't have another place to be. It's not secure, it's not authenticated, and it's a little scary that it exists at all. I wish to add some data to the metadata server that can be found somewhere else. That a user could jump through a hoop or two to add to their instances. Esteemed personages are concerned that I would be crossing the Thank you for using 'Esteemed' rather than 'no good @#@$! %!$@$ %#%#%#%', which I'm sure you considered. rubicon in terms of opening up the metadata api for wanton abuse. They are not without a right or reason to be concerned. And that is why I am going to attempt to explicitly classify a new category of data that we might wish to allow into the metadata server. If we can be clear about what we are allowing we can avoid abuse. As Jay was confused, I think we really need to separate the notion of EC2 API and EC2 Instance Metadata API. Right now, nova is *very* usable without use of the external EC2 API. It is much more dependent on the Instance Metadata API. Config drive is one alternative to the EC2 metadata service. Matt's mail discusses only the Instance Metadata API I had a blueprint at the last summit that covered making config-drive and the ec2 metadata service basically contain the same data. http://etherpad.openstack.org/FolsomNovaConfigDriveImprovements The general idea there was to replace the current config-drive format with basically a rendering of the data found in the metadata service. This proposal met with surprisingly little complaint. I planned on then also extending that by adding to both the metadata service and the config-drive openstack metadata content. This bit of information that Matt is interested in would fit under the openstack metadata content. But would not be available under the EC2 data. My plan for backwards compatibility was basically that we would make: * http://169.254.169.254/ : look just like the ec2 metadata like thing that we have now in nova. A request here would dump the same content it currently does, which is a list of -MM-DD versions. * http://169.254.169.254/ec2 point to http://169.254.169.254/ * http://169.254.169.254/openstack/ be the well known location of the openstack metadata service. The config drive basically then looks just like the metadata service. We then have to define the content that we have at that exists at /openstack, basically creating the first version of the openstack metadata service api. I personally see nothing wrong with -MM-DD for a version, as it sorts so well, and is generally immediately understood what it means. I planned to simply have the top level entry list versions available just like the EC2 one does. Ie, It would have something like: 2012-07-25 latest A request to http://169.254.169.254/openstack/2012-07-25 would return JSON formatted data. That data would have things like: instance-id ami-id keystone-endpoint -- for Matt What I said to Matt was that I think we need to consider (and scrutinize) changes to the data available in the openstack metadata service the same way we would an API change. My choice of the keyname 'keystone-endpoint' is probably bad, but in making a bad choice, it shows the value in making good choices, which aren't made without thought and deliberation. I simply haven't gotten around to doing the work above. If someone else wants to take a stab at it, I'd love to help and work together on it. My current work effort in regards to this is related to passing keystone credentials to instances via pam authentication. So I can do a number of API related queries into openstack because I have credentials available to the OS that are dynamically allocated. But to make my image portable I need to not be baking in the keystone API URI. I promised Matt I would not complain, so my comments here have been deleted. :) ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] How do I stop image-create from using /tmp?
On Mon, 2 Jul 2012, Jay Pipes wrote: On 07/02/2012 01:32 PM, Mark Lehrer wrote: just did an ln -s /some/dir/with/space /tmp and that does solve I added an option to /etc/init/nova_compute.conf to specify the tmp space, so the start line looks like this: exec su -s /bin/sh -c export TMPDIR=/var/tmp; exec nova-compute --flagfile Not a great solution either since package updates over-write this setting. Yeah, that's exactly what we found as well -- Chef would overwrite the upstart script, and we couldn't for the life of us figure out how to get Chef to set the TMPDIR environment variable for *user running nova-compute*. I'm not familiar with Chef, or how you were using it, so I can't help there. I wont argue against a specific flag, but I do think its mostly useless. I think this is largely bikeshed, and for some reason I feel compelled to argue in favor of blue. However, going the flag route then seems more dangerous to me. Any code would then to know that it should reference this flag for temp file creation. That problem is already solved with TMPDIR and mkdtemp and friends. Using the standard path, you get that for free. If things are *not* respecting TMPDIR, they should. And, for what its worth (not much), a better way of modifying the /etc/init/nova_compute.conf file above would be to add: env TMPDIR=/var/tmp (see man 5 init). ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Sending userdata during server create via api's
On Mon, 2 Jul 2012, Ed Shaw wrote: Hi, I've posted on this previously but have yet to be pointed in the right direction - so I'm posting again. Examples or docs appreciated. I'm trying to pass user_data on server create using the xml (or JSON) api. My userdata looks like... #!/bin/bash # #Purpose : Setup the initial image Other than the leading , that looks fine. I suspect you did not mean the leading quote. set -e -x export DEBIAN_FRONTEND=noninteractive apt-get update apt-get upgrade -y ... I am base64 UTF-8 encoding the string and I've tried sending it as a message part, a query string on the url and as a post parameter. This works from the Horizon UI, but I get... 2012-06-18 19:58:18,610 - __init__.py[WARNING]: Unhandled non-multipart userdata '' Well, the above is indicating that there is no user-data available to the instance. Ie, it is . I actually just committed a *removal* of this warning in cloud-init as I had previously thought it just an annoying message. You can verify from inside the instance that there is no user-data, with 'ec2metadata --user-data' or 'wget http://169.254.169.254/latest/user-data' when I try to pass via xml. The only thing I haven't tried is a different extension namespace on the user_data element if passing it that way, but I can't see any docs on this. Sorry, without digging/learning more, I can't help more. Have you tried the python-novaclient: nova boot --user_data my.user.data.file.txt ... Ie, try verifying that it is working with known working tools on your openstack first. Alternatively, if the ec2 api is available, that looks like: euca-run-instances --user-data-file my.user.data.file.txt ... Here is an example of one of the configurations I tried... ?xml version=1.0 encoding=UTF-8? server xmlns=http://docs.openstack.org/compute/api/v1.1; name=server8 imageRef=http://192.168.75.70:8774/30ddcb35079f406eae98857515cbf1d2/images/57443c48-eb29-48f6-853a-b8bc7d5dde05; flavorRef=1 user_data=IyEvYmluL2Jhc2gNCiMNCiNBdXRob3IgOiBFZCBzaGF3DQojRGF0ZSA6IDE0IEp1biAxMg0KI1B1cnBvc2UgOiBTZXR1cCB0aGUgaW5pdGlhbCBpbWFnZQ0KI0NvbW1lbnRzIDoNCiMNCiMgTGFzdCBFZGl0dGVkIGJ5OiBlZHNoYXcNCg0Kc2V0IC1lIC14DQpleHBvcnQgREVCSUFOX0ZST05URU5EPW5vbmludGVyYWN0aXZlDQoNCmFwdC1nZXQgdXBkYXRlICYmIGFwdC1nZXQgdXBncmFkZSAteQ0KYXB0LWdldCAteSBpbnN0YWxsIGFwYWNoZTINCg0KI1NlbmQgdG8gY29uc29sZSB3ZSBmaW5pc2hlZCBydW5uaW5nLg0KZWNobyAiTkVYSkNPTkZJRzogSW5zdGFuY2Ugc2V0dXAgc3VjY2Vzc2Z1bHkgZXhlY3V0ZWQuIiA/ I dont see anything obviously wrong, but I'd try using the knonw orking tools first. I think python-nova client even has a --debug to dump what it is sending. Scott ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Setting VM passwords when not running on Xen
On Tue, 3 Jul 2012, Day, Phil wrote: Hi Folks, Is anyone else looking at how to support images that need a password rather than an ssh key (windows) on hypervisors that don't support set_admin_password (e.g. libvirt) ? I'm completely ignorant about windows. Please forgive me. Is it for some reason not possible to have code that runs on first instance boot that reads the metadata service (or config drive) and sets the password appropriately? Or, is there something that makes this actually a nova problem, something that cannot reasonably be solved in the same way such things are solved in other operating systems. Is there no way that you could pass in a public key that would be used for authentication to RDP or whatever you'd do? (grave ignorance of windows). ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Anyone using instance metadata?
Hi, I'm looking at nova, and the compute API has 3 methods: delete_instance_metadata update_instance_metadata get_instance_metadata I know that * python nova client has * the ability to specify --meta=KEY=VALUE on instance creation * a top level subcommand 'meta' which allows set and delete of metadata keys (but no support for querying current value). * content specified on instance creation is injected into the instance's root filesystem at '/meta.js'. Or, if config_drive is given the config drive will have /meta.js What I'm missing is: * There is an 'update' for this content, implying that it is at least partially dynamic in intent, but the filesystem data passing mechanism is clearly *not* dynamic. * This data is not made available in the metadata server (http://169.254.169.254) where it *could* be dynamic. So, I'm confused on the intent of this metadata. I can't decide if it is really just supposed to be tags or more of a user-data replacement. In EC2, you can store arbitrary key/value pairs on an instance-id, ami-id, or anything else, but those values are not readable without credentials. Here, they're readable inside the instance from the filesystem. Anyone able to comment on the original intent of 'metadata'? It seems to me that if we expose this in the metadata server, then it will be a very useful feature, but one that overlaps confusingly with user-data. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Anyone using instance metadata?
On Tue, 3 Jul 2012, Vishvananda Ishaya wrote: Metadata is supposed to be user tags that are associated with a guest that are available via the api. We discussed displaying these tags inside the guest as well. Am I reading it wrong? It seems like it *is* available inside the guest. At very least with config drive on i know it is. The main difference between user-data and metadata is that metadata is available to the api, whereas user-data is only available in the guest. So to avoid confusion, if the intent was tags, I think we should disable the 'meta.js' file injection, and get over the screams now. half and half is just confusing. Thoughts? ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Setting VM passwords when not running on Xen
On Tue, 3 Jul 2012, John Garbutt wrote: This seemed to crop up quite a lot in different sessions at the Design summit. I am certainly interested in a standard way to inject information into VMs. What I think we need is a cross hypervisor two-way guest communication channel that is fairly transparent to the user of that VM (i.e. ideally not a network connection). If I understand things correctly, we currently have these setup ideas: * Config Drive (not supported by XenAPI, but not a two way transport) * Cloud-Init / Metadata service (depends on DHCP(?), and not a two-way transport) cloud-init does not require dhcp. It explicitly supports the passing of network interface definitions into it in Ubuntu 12.04. Ie, config-drive with static networking passed in works as it should. But to set the password, we ideally want two way communication. We currently have these: * XenAPI guest plugin (XenServer specific, uses XenStore, but two way, no networking assumed ) * Serial port (used by http://wiki.libvirt.org/page/Qemu_guest_agent but not supported on XenServer) I like the idea of building a common interface (maybe write out to a known file system location) for the above two hypervisor specific mechanisms. The agent should be able to pick which mechanism works. Then on top of that, we could write a common agent that can be shared for all the different hypervisors. You could also fallback to the metadata service and config drive when no two way communication is available. I would love this Guest Agent to be an OpenStack project that can then be up streamed into many Linux distribution cloud images. The only thing I don't like about this is there is so very little need for long-lived communication between the hypervisor and the guest. I personally think that cloud-init gets this right. Its goal is to do what it needs to do and get out of the way. It should provide enough to bootstrap you to a more intelligent management framework, such as puppet, chef, juju there are literally dozens of these things that work perfectly well without a hypervisor specific transport. They're widely available, work right now, and don't care if they're running on bare metal, xen, kvm, microsoft virtual server... They just assume that you can connect to them via network, which is a really well defined and tested thing. Its 2012, do we really need to design for the case where IP is broken or not available? I'm not arguing that guest agents are not necessary, I'm arguing that there is extremely little need for them to be hypervisor aware. (yes, there is value in the host being able to request the guest to freeze its filesystem after receiving a API freeze request. However, even *that* could happen over IP). Sadly, I don't have any time to work on this right now, but hopefully that will change in the near future. Cheers, John From: openstack-bounces+john.garbutt=eu.citrix@lists.launchpad.net [mailto:openstack-bounces+john.garbutt=eu.citrix@lists.launchpad.net] On Behalf Of Day, Phil Sent: 03 July 2012 3:07 To: openstack@lists.launchpad.net (openstack@lists.launchpad.net) (openstack@lists.launchpad.net) Subject: [Openstack] Setting VM passwords when not running on Xen Hi Folks, Is anyone else looking at how to support images that need a password rather than an ssh key (windows) on hypervisors that don't support set_admin_password (e.g. libvirt) ? Thanks Phil ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Anyone using instance metadata?
On Tue, 3 Jul 2012, Vishvananda Ishaya wrote: The config drive was a later addition because we thought it might be useful. The plan was to add it to the metadata server once we had a /openstack available. The main difference between user-data and metadata is that metadata is available to the api, whereas user-data is only available in the guest. So to avoid confusion, if the intent was tags, I think we should disable the 'meta.js' file injection, and get over the screams now. half and half is just confusing. Thoughts? Seems much more useful in metadata server than in config drive, but we should probably keep the same semantics we have been discussing i.e. the same values are in both places. Config drive is information as it was during launch and metadata is current information. That does make sense. The only thing that is confusing is its similarity to user-data. putting it in the MD then leads us to the other thread/request ([Openstack] Setting VM passwords when not running on Xen) of having it be writable from the instance. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] ssh not authenticating
On Fri, 22 Jun 2012, Alexey Ababilov wrote: Hi, Scott! I knew that the password was already on the cirros image. However, we (Grid Dynamics engineers) faced a strange problem: 1) if an instance is launched without an ssh key, you can login using the password (generated by nova in our case); 2) if an instance is launched _with_ an ssh key, you cannot login using either password or the ssh key. I've not seen this myself. could you provide a console log (euca-get-console-output) of the image that is broken? And exactly what cirros image you're using? (ie, the disk.img or the uec tarbal...) I really can't imagine anything that would mess up password based auth with the cirros user. So, VM's filesystem synchronization can be a dark and tricky thing. That's why we are not guaranteed to login by password even if the password was stored in the image originally. Sure, I wouldn't recommend password based auth anyway. It is only enabled in the cirros image as its intent is test. please do not run cirros images with default password for anything important. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] ssh not authenticating
On Thu, 21 Jun 2012, udit agarwal wrote: Hi Mandar, Thanks for your reply. But I am still stuck in the problem. I tried with cirros but with no luck. And for the next one, the system still asks me for password, but it was supposed to be password-less. Can anybody help me with this?? As Alexey pointed out, the password for cirros is cubswin:). The *user* is 'cirros'. So, to login with password auth: ssh cirros@ip-address-here password-less ssh *should* work and should *not* depend on filesystem injection by nova. cirros will read the public keys specified from the metadata service and import them into the cirros user's account. You have to specify a key when you launch the instance either by: * nova boot --key_name KEYNAME or * euca-run-instances --key KEYNAME To debug why passwordless ssh is not working: * get the console output of the instance. It likely is complaining that it cannot find the metadata service. * ssh in with password auth and run 'ec2metadata --public-keys'. This will probably fail. Basically, I suspect that your metadata service is not available to the instance. Scott Thanks in advance. --Udit On Thu, Jun 21, 2012 at 6:53 PM, Vaze, Mandar mandar.v...@nttdata.comwrote: Default password for cirros is cubswin:) - including the smiley I hope you entered correct password (It is easy to ignore the last two characters thinking it is a smiley) Other options is to use password less ssh login using keypair. See http://docs.openstack.org/essex/openstack-compute/starter/content/Launch_and_manage_instances-d1e1885.html -Mandar From: openstack-bounces+mandar.vaze=nttdata@lists.launchpad.net[mailto: openstack-bounces+mandar.vaze=nttdata@lists.launchpad.net] On Behalf Of udit agarwal Sent: Thursday, June 21, 2012 11:13 AM To: openstack@lists.launchpad.net Subject: [Openstack] ssh not authenticating Hi, I have setup openstack on my system using the manual http://docs.openstack.org/essex/openstack-compute/install/apt/content/. I tried to run the test image cirros on my system. While I can easily ping it, I faced some problems in authenticating via ssh. First, it asks for password, even providing the correct password for the same, it re asked the password 2 more times and finally denied me the permission. I couldn't resolve this issue then. Can anybody help me with this??? thanks in advance. --Udit __ Disclaimer:This email and any attachments are sent in strictest confidence for the sole use of the addressee and may contain legally privileged, confidential, and proprietary data. If you are not the intended recipient, please advise the sender by replying promptly to this email and then delete and destroy this email and any attachments without any further use, copying or forwarding ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] File injection support
On Mon, 11 Jun 2012, Fredric Morenius wrote: Hello again, I did some further tests on this issue with the qcow2 image, the qcow2 image converted to raw and the _blank (raw) image from the UEC tar archive. I loaded nbd with $ sudo modprobe nbd max_part=8 *** qcow2 image *** $ k$ md5sum cirros-0.3.0-x86_64-disk.img 50bdc35edb03a38d91b1b071afb20a3c cirros-0.3.0-x86_64-disk.img this image is a qcow compressed disk image. The disk contains a partition table, grub (0.97) installed on the MBR, and a root filesystem on the first partition. That creates the device /dev/nbd15p1 kpartx on top of that: $ sudo kpartx -a /dev/nbd15 device-mapper: resume ioctl failed: Invalid argument create/reload failed on nbd15p1 $ sudo qemu-nbd -d /dev/nbd15 /dev/nbd15 disconnected That seems like working as designed. at least on ubuntu 12.04, the kpartx is no longer necessary. I'd have to dig more to figure out exactly why, but some combination of kernel partition scanner and udev do the right thing when this new network block device is set up. *** UEC image *** $ qemu-img info cirros-0.3.0-x86_64-blank.img image: cirros-0.3.0-x86_64-blank.img file format: raw virtual size: 24M (25165824 bytes) disk size: 24M $ sudo qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-blank.img This image is essentially the same as you'd get if you did: truncate --size 24M disk.img mkfs.ext3 -F disk.img -L cirros-rootfs Ie, it is a unpartitioned disk with an (empty) filesystem on it. There are no partitions, or data in the filesystem. On boot, cirros initramfs copies itself to that disk, and then pivot_roots to it. Cirros will run perfectly fine as an initramfs with no root filesystem too, fwiw. That DOES NOT create the device /dev/nbd15p1 kpartx on top of that: $ sudo kpartx -a /dev/nbd15 kpartx executes without error $ sudo qemu-nbd -d /dev/nbd15 /dev/nbd15 disconnected This also seems like working as designed. *** raw image *** $ qemu-img convert -p -O raw cirros-0.3.0-x86_64-disk.img cirros-0.3.0-x86_64-disk_raw.img $ qemu-img info cirros-0.3.0-x86_64-disk_raw.img image: cirros-0.3.0-x86_64-disk_raw.img file format: raw virtual size: 39M (41126400 bytes) disk size: 12M $ sudo qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-disk_raw.img That creates the device /dev/nbd15p1 kpartx on top of that: $ sudo kpartx -a /dev/nbd15 device-mapper: resume ioctl failed: Invalid argument create/reload failed on nbd15p1 $ sudo qemu-nbd -d /dev/nbd15 /dev/nbd15 disconnected So, the issue is not with the qcow2 vs raw image formats. Instead kpartx only executes without error provided that the device /dev/nbd15p1 is not created. When p1 is created by qemu-nbd, kpartx is not needed to mount the image device, but the problem is qemu-nbd does not create p1 in all cases. I have not checked the details of the OpenStack code, but an (ugly) work around could be to only execute kpartx when it is needed. Without digging around on older versions of OS's and their included kernel/udev/kpartx i'd have to say that from your message above, and my testing that running 'kpartx' is no longer necessary. That can reasonably be expected to be created by the kernel or some other plumbing bits automatically. If it just happens magically,though, I'm not sure how you would know if it hasn't happened yet. That would imply that this might be racey. I ran into similar bugs http://pad.lv/937352 and http://pad.lv/942788 that affected the growing of the root partition from an initramfs. The thing we ended up settling on was that after re-writing the partition table, we had to run 'udevadm --settle'. its quite possible that 'qemu-nbd --connect' sets off a similar flurry of events that would need to be waited on before a /dev/nbdXpY would exist. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] File injection support
On Fri, 8 Jun 2012, Pádraig Brady wrote: On 06/08/2012 09:20 AM, Fredric Morenius wrote: Hello All, An update on the use of the qemu-nbd/kpartx based solution to inject files into VM images: After some more testing it has turned out that injection into the UEC version of CirrOS (this: https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-uec.tar.gz) works fine, but injection into the qcow2 version of the image produces the error given in the previous mail, so there seems to be robustness problems with these tools. Yes it's seems that qemu-nbd has issues with this image? # rpm -qf $(which qemu-nbd) qemu-common-1.0-17.fc17.x86_64 # qemu-img info cirros-0.3.0-x86_64-disk.img image: cirros-0.3.0-x86_64-disk.img file format: qcow2 virtual size: 39M (41126400 bytes) disk size: 9.3M cluster_size: 65536 # qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-disk.img # ls -la /sys/block/nbd15/pid -r--r--r--. 1 root root 4096 Jun 8 10:19 /sys/block/nbd15/pid # kpartx -a /dev/nbd15 device-mapper: resume ioctl on nbd15p1 failed: Invalid argument create/reload failed on nbd15p1 If I convert to raw with qemu-img, I can mount loopback fine. Well, interesting. It does seem to work here: $ dpkg-query --show qemu-utils qemu-utils 1.0+noroms-0ubuntu13 $ cd /tmp $ wget -q https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-disk.img $ md5sum cirros-0.3.0-x86_64-disk.img 6654705afc4b74fda3e1f4330559d066 cirros-0.3.0-x86_64-disk.img $ qemu-img info cirros-0.3.0-x86_64-disk.img image: cirros-0.3.0-x86_64-disk.img file format: qcow2 virtual size: 39M (41126400 bytes) disk size: 11M cluster_size: 65536 $ sudo qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-disk.img $ ls -l /dev/nbd15* brw-rw 1 root disk 43, 240 Jun 8 11:32 /dev/nbd15 brw-rw 1 root disk 43, 241 Jun 8 11:32 /dev/nbd15p1 $ sudo mount /dev/nbd15p1 /mnt $ ls /mnt bin etc initrd.img lost+found opt run tmp vmlinuz boot home lib mnt proc sbin usr dev init linuxrc old-rootroot sys var $ sudo umount /mnt $ sudo qemu-nbd -d /dev/nbd15 /dev/nbd15 disconnected___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] File injection support
On Fri, 8 Jun 2012, Scott Moser wrote: On Fri, 8 Jun 2012, Pádraig Brady wrote: On 06/08/2012 09:20 AM, Fredric Morenius wrote: Hello All, An update on the use of the qemu-nbd/kpartx based solution to inject files into VM images: After some more testing it has turned out that injection into the UEC version of CirrOS (this: https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-uec.tar.gz) works fine, but injection into the qcow2 version of the image produces the error given in the previous mail, so there seems to be robustness problems with these tools. Yes it's seems that qemu-nbd has issues with this image? # rpm -qf $(which qemu-nbd) qemu-common-1.0-17.fc17.x86_64 # qemu-img info cirros-0.3.0-x86_64-disk.img image: cirros-0.3.0-x86_64-disk.img file format: qcow2 virtual size: 39M (41126400 bytes) disk size: 9.3M cluster_size: 65536 # qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-disk.img # ls -la /sys/block/nbd15/pid -r--r--r--. 1 root root 4096 Jun 8 10:19 /sys/block/nbd15/pid # kpartx -a /dev/nbd15 device-mapper: resume ioctl on nbd15p1 failed: Invalid argument create/reload failed on nbd15p1 If I convert to raw with qemu-img, I can mount loopback fine. Well, interesting. It does seem to work here: $ dpkg-query --show qemu-utils qemu-utils 1.0+noroms-0ubuntu13 $ cd /tmp $ wget -q https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-disk.img $ md5sum cirros-0.3.0-x86_64-disk.img 6654705afc4b74fda3e1f4330559d066 cirros-0.3.0-x86_64-disk.img Whoops. I had a local image that was different (must have been laying around from long ago, the wget saved to .1). It does still work though. $ md5sum cirros-0.3.0-x86_64-disk.img 50bdc35edb03a38d91b1b071afb20a3c cirros-0.3.0-x86_64-disk.img $ qemu-img info cirros-0.3.0-x86_64-disk.img image: cirros-0.3.0-x86_64-disk.img file format: qcow2 virtual size: 39M (41126400 bytes) disk size: 9.3M cluster_size: 65536 The same nbd -c... mount work as shown.___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] File injection support
On Wed, 30 May 2012, Fredric Morenius wrote: Hello Pádraig, I am also trying to get file injection to work in Essex, but have run into some issues, as stated here: https://answers.launchpad.net/nova/+question/198878 The image I am launching is a simple bare container qcow2 image (CirrOS, this: https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-disk.img ) I generally think that file injection is significantly less preferable to use of user-data or config-drive (an api/convention for providing information to an instance). cirros supports executing user-data starting with '#!', so you can probably accomplish what you're after via passing data like this: #!/bin/sh cat /path/to/file EOF file contents here EOF wget http://some.url/my.tar | ( cd / tar xvzf - ) base64 -d /usr/bin/my-exe EOF IyEvYmluL3NoCmVjaG8gImhpIHdvcmxkIgo= EOF chmod 755 /usr/bin/my-exe end user data ### You could also use 'shar' to do this.___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Can't ssh into instance
On Fri, 25 May 2012, Anton Haldin wrote:
Thank you very much guys Sébastien and Leander
It was very interesting case.
I thought if I see this lines in compute.log then sshkey injection was
successfully
Given functional metadata service,
ssh key injection from the host is not necessary for ubuntu images.
And I was sure I have few vms with metadata server access issue and
without ssh connection issue .
Ubuntu cloud images do not have any ssh HostKey generated inside them
(/etc/ssh/ssh_host_{ecdsa,dsa,rsa}_key). The keys are generated by
cloud-init after it finds a metadata service. Without a metadata service,
they do not get generated. ssh will drop your connections immediately
without HostKeys.
This is arguably a bug, and there is an open bug to have the keys
generated independent of ability to find a metadata service
(http://pad.lv/906669).
It looks like I need some additional tests and need more time for code
reading : - )
Fix the metadata issue, you're going to want it.
Alternatively, use configdrive
(http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc/configdrive/README)
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Can't ssh into instance
On Thu, 24 May 2012, Leander Bessa Beernaert wrote: Compute log: Log: http://paste.openstack.org/show/18149/ I've tried bot root and ubuntu as account names (ssh -i key.pem root@10.1.2.3 and ssh -i key.pem ubuntu@10.1.2.3) and the result is still Read from socket failed: Connection reset by peer On Thu, May 24, 2012 at 1:57 PM, Leander Bessa Beernaert leande...@gmail.com wrote: Complete log: http://paste.openstack.org/show/18144/ the issue is that the instance cannot access the metadata service. The node is not routing traffic destined to 169.254.169.254 to an API server. This will not work until that is fixed. Cirros instances will boot, and let you log in, and verify/troubleshoot that. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] centos 6 images
On Tue, 22 May 2012, Jason Ford wrote: Scott, If you need someone to test your changes, I would be happy to do it. Please just give me some basic instructions on how to put it in place and I will get it working. As for your request for comments/features, personally I would like to see the following parts done initially: I'm not sure of whether or not these things work in the status of these things in fedora/RHEL. I suspect that all below is true other than disk space resize. However, the following is true for ubuntu: - hostname set to instance name - hostname is set to whatever is in local-hostname of the metadata service. I think that actually does correlate to 'name' in the nova launch api. - disk space resize to flavor size - cloud-init runs resize2fs on the root partition on start up. However, if you have a partition table on the disk, right now, that relies upon initramfs code delivered in cloud-initramfs-growroot [1] to grow the partition , and tell the kernel to re-read the partition while the filesystem is not mounted (otherwise you have to reboot). [1] https://launchpad.net/ubuntu/+source/cloud-initramfs-tools - ssh-key pull - cloud-init does that by default to the default user and adds it to the root user's .ssh/authorized_keys, with a message that says you need to login as the-configured-user - report a random password for root user (or default user) if this is possible - by default, cloud-init does not set a password for a user. However, it can be told to via configuration (cloud-config in the image or passed in via userdata). That looks like: password: RANDOM or chpasswd: list: | user1:password1 user2:RANDOM expire: True There is more info at [2]. [2] http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc/examples/cloud-config.txt Thanks for looking at this. jason - Original Message - From: Scott Moser smo...@ubuntu.com To: Pádraig Brady p...@draigbrady.com Cc: Fedora Cloud SIG cl...@lists.fedoraproject.org, Andy Grimm agr...@gmail.com, openstack openstack@lists.launchpad.net Sent: Tuesday, May 22, 2012 12:51:42 PM Subject: Re: [Openstack] centos 6 images On Tue, 22 May 2012, Pádraig Brady wrote: On 05/22/2012 03:39 PM, Andy Grimm wrote: On Tue, May 22, 2012 at 9:38 AM, Pádraig Brady p...@draigbrady.com wrote: On 05/22/2012 04:07 AM, Jason Ford wrote: I am trying to put together an image for centos 6 that works like cloud-init on ubuntu does. Currently I have ssh keys getting imported but having some problems getting the disk to dynamically resize to the flavor template as well as the hostname set in horizon to be pushed into the image. Does anyone have any howtos or suggestions on how to get this done? Is there cloud-init for centos just like ubuntu? I would also be interested in how to do this with debian as well. Well I notice there is no cloud-init package for EPEL. I took a quick stab at it here: http://pbrady.fedorapeople.org/cloud-init-el6/ I've already responded in IRC, but it wouldn't hurt to have a response in the mail archive. In short, the reason there isn't already a cloud-init for EL6 (or EL5, for that matter) is that upstream has been using python 2.7-only calls for a while now. In particular, a couple of calls to subprocess.check_output need to be replaced, and I think there are a few other issues as well. I don't think it's a huge It would help if you'd bring that up with upstream :) I'm interested in cloud-init working in the most places it can. I'll try to pull in the sysvinit scripts that Pádraig added and grab other changes that are there. amount of work to make it functional, but it hasn't been high on anyone's list. It would be cool if you have time to fix / test it, though. Ok I've fixed the check_output calls at the above URL. If anyone has features / issues they'd like addressed in cloud-init, please feel free to ping me (smoser). I'll most likely ask you to open a bug at http://bugs.launchpad.net/cloud-init , and may even invite you to submit a patch. One way or another, though, I'm interested in making cloud-init better, so comments/concerns/participation is welcome and encouraged. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] centos 6 images
On Tue, 22 May 2012, Pádraig Brady wrote: On 05/22/2012 03:39 PM, Andy Grimm wrote: On Tue, May 22, 2012 at 9:38 AM, Pádraig Brady p...@draigbrady.com wrote: On 05/22/2012 04:07 AM, Jason Ford wrote: I am trying to put together an image for centos 6 that works like cloud-init on ubuntu does. Currently I have ssh keys getting imported but having some problems getting the disk to dynamically resize to the flavor template as well as the hostname set in horizon to be pushed into the image. Does anyone have any howtos or suggestions on how to get this done? Is there cloud-init for centos just like ubuntu? I would also be interested in how to do this with debian as well. Well I notice there is no cloud-init package for EPEL. I took a quick stab at it here: http://pbrady.fedorapeople.org/cloud-init-el6/ I've already responded in IRC, but it wouldn't hurt to have a response in the mail archive. In short, the reason there isn't already a cloud-init for EL6 (or EL5, for that matter) is that upstream has been using python 2.7-only calls for a while now. In particular, a couple of calls to subprocess.check_output need to be replaced, and I think there are a few other issues as well. I don't think it's a huge It would help if you'd bring that up with upstream :) I'm interested in cloud-init working in the most places it can. I'll try to pull in the sysvinit scripts that Pádraig added and grab other changes that are there. amount of work to make it functional, but it hasn't been high on anyone's list. It would be cool if you have time to fix / test it, though. Ok I've fixed the check_output calls at the above URL. If anyone has features / issues they'd like addressed in cloud-init, please feel free to ping me (smoser). I'll most likely ask you to open a bug at http://bugs.launchpad.net/cloud-init , and may even invite you to submit a patch. One way or another, though, I'm interested in making cloud-init better, so comments/concerns/participation is welcome and encouraged.___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Public image repository via a synchronization script
On Wed, 25 Apr 2012, Justin Santa Barbara wrote: I think it would be very convenient to have public image repositories and an easy way to use them, so someone installing OpenStack can easily get images into Glance with minimal work. At the design summit, there were a number of talks which all seemed to hinge on how to get an image into glance from a public store. I'd like to make a proposal to get the ball rolling, which we can collectively pick apart to figure out the best way. 1. A public image repository lives on an HTTP / FTP / ... site and can be dumb. (Glance could one day be the server here as well, but this should also work on a CDN or simple webserver etc). 2. Each image is stored as a (possibly compressed) file. fwiw, qcow images support compression inside: qemu-img convert -c disk.img -O qcow2 my.qcow2 compression is potentially useful elsewhere. It might be good thoguh, in your properties to explicitly list this rather than implying by the name. 3. Each image has a .properties file alongside it. 4. There is a 'directory' file which lists the property files. An md5sum of the directory is easy, and the hashes be useful generally. 5. If there are different sets of images, we have multiple directory files. They can point to the same images. 6. We don't try to solve security initially. But this could be as simple as signing the directory files, if they had hashes of all the other files (e.g. md5sum). Overall, I think this looks reasonable. A couple nits: * I don't like using md5sum as a directory file. Better to explicitly name something for that purpose. There could be other files in that directory that are not images (and would be desireable to appear in the md5sum file) * your example .properties file contains things that aren't really properties (where properties == things that would be added to glance as tags) disk_format=qcow2 container_format=bare is_public=True * Disk format, container_format are useful, but its possibly confusing as its an attribute of the image, while other things are tags. * is_public=True Just because you thought this image should be public does not mean I do when I sync it. I might suggest, something like this (hand-yaml formated): tags: org.openstack__1__os_distro=debian.org org.openstack__1__os_version=6.0.4 org.openstack__1__architecture=x64 org.openstack__1__updated_at=20120425 data: file: debian_squeeze_20120425.qcow2.gz disk_format: qcow2 container_format: bare md5sum: 224237ee1b9a341ac7d79bcebda0580d size: 627965952 compression: gzip Alternatively, 'data' above could be named 'org.openstack.sync__1__image' or something. Ben Howard has been doing something similar to this with https://code.launchpad.net/~utlemming/+junk/sync-images which reads data at https://cloud-images.ubuntu.com/query2/ I'd like to work together to get something that addresses both sets of needs. Also, if we do this well, I suppose that the glance aggregator described at https://blueprints.launchpad.net/glance/+spec/public-glance http://etherpad.openstack.org/FolsomGlanceReadOnlyGlance could potentially support this source something it presented in its aggregated list. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Agreeing a common set of Image Properties
On Sat, 7 Apr 2012, Justin Santa Barbara wrote: Is there a (de-facto) standard for image metadata/properties? I'd like to be able to able to launch e.g. the Debian Squeeze image provided by the cloud. This is particularly important for clouds that don't allow image upload, but likely this will remain useful because different clouds will have different tweaks needed (e.g installing the right drivers based on the hypervisor). I could try smart-parsing the names, but it seems like metadata is the right way to do this, and I see no reason why any cloud would gain any advantage from not adopting a common convention. I know some clouds have started implementing their own approaches, but I don't believe anyone is locked into anything. In the interest of efficiency, I'm going to make a proposal for people to attack: 3 main pieces of metadata: os:distro, os:version_major, os:version_minor I generally disagree that these are the 3 main pieces of meta-data. It might seems strange that someone with a @distro.com address would say this, but I believe that distro is largely irrelevant. If I'm looking for a image to run, the thing that is most important to me is the provider of the image. In some clouds, with no public images, the owner of the image is quite likely the cloud provider, and I have little option other than to trust them on the contents of an image (I already trust them almost completely by launching an instance there). However, in a large public cloud (amazon for example) there are literally thousands of providers, and thousands that claim Ubuntu, and 10.04, and would (justifiably) also upload package lists that contained a list like you suggested later in this thread. The problem is that I don't trust those image providers, I know nothing about them, and any metadata provided by them other than GPG signed data or well-known IDS is not going to change my lack of trust (ie, Canonical images on EC2 are owned by id 099720109477, redhat publishes under 432018295444 and 309956199498). The data you're after might be useful to you, and might scratch an itch. I will not discount that, but I would much prefer a bit of metadata associated with an image that was signed by an entity I trusted that identified the image as good. The ubuntu name strings look like this: ubuntu-oneiric-11.10-i386-server-20120401 If I see that string, and content signed by a key that I trust, then I can be assured that it is what I think it is. I may well trust 'cloud-provider-foowhiz's signature on that string, and not require it to be signed by an Ubuntu key. I may not. I may accept centos-6.04-i386-server-20120401 signed by centos.org key to be a replacement for rhel-6.04-i386-server-20120401 What i'm getting at, is that I think vendor is more important than os in the long run (and even the short run). OS distro, version_major, version_minor are even less important where you don't care (or know) that your OS came from Canonical or RedHat, what you were really interested in is running WhizBang! Fooberator version 2.0. One feature thing I think glance could (and probably does provide me with) is md5/sha1 sums of the images that were uploaded. If I know that an image I'm about to run has md5sum that matches that of a published image on http://cloud-images.ubuntu.com then I'm more willing to trust it (given that I already trust the cloud provider). I guess if Canonical provided a signed sum of: ubuntu-oneiric-11.10-i386-server-20120401:md5sum Then the cloud uploader could just upload the existing Canonical sum as a tag on that image, and then my signature checking would work also. I trust the cloud provider out of necessity, and I trust Canonical, so the 2 things work out. So, I guess in summary, * I don't think os:version and the like are encompassing or particularly useful by themselves. * I can't trust tagged data by itself * I wouldn't trust any scraping of package contents by libguestfs, as it can most certainly be fooled or broken. I can see that some tagged info on the contents of the image would be useful for certain things, but specifically OS specific information is just not that important. The vast, *vast* majority of us don't care that e2fsprogs is in the Ubuntu image, or that it is version 1.4.1-0ubuntu1. I personally hope that I've done a good job of making them more interested in the fact that it is an Ubuntu cloud image. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Metadata and File Injection (code summit session?)
On Tue, 10 Apr 2012, Andrew Bogott wrote: I'm reviving this ancient thread to ask: Will there be a code summit session about this? And/or are there plans to start developing a standard set of guest agents for Folsom? http://summit.openstack.org/sessions/view/100 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Agreeing a common set of Image Properties
On Tue, 10 Apr 2012, Justin Santa Barbara wrote: Signing would definitely be a great v2 feature. For v1, we just need some way to know that an image is provided by the cloud provider, and some idea of what that image is. I believe every cloud is stuck respinning their own images, because we haven't been able to agree a golden image standard. So signing etc by the distros is pointless until we get that figured out. Really? Because the ubuntu cloud images work in EC2 or Openstack or Eucalyptus or insert-other-lesser-known-cloud-here. No modification is necessary. I trust the cloud providers today - I have no choice but to do so. I think you're trying to solve a much harder problem - how do I cope in a world where I trust Canonical but not my cloud? Once we have hardware trust of clouds, then we'll have to up our game substantially on every front here. No. You trust the cloud provider, thats one thing. The cloud provider is not necessarily the image provider. I trust Canonical, I trust Amazon. I do not trust image id 938181839409 who produced 'hackerz/Ubuntu-11.10-great-package-list-for-smoser'. 3 scenarios: a.) canonical publishes an image to a public cloud (like they do to EC2). I trust Amazon, I trust canonical, I know that canonical publishes images under id X. b.) public cloud startupcloud.com publishes Ubuntu images. If i'm running images on startupcloud.com, I obviously have to trust them. Canonical couldn't care less about them, so startupcloud.com, uses the data at https://cloud-images.ubuntu.com/query2/ to mirror ubuntu images. They use the naming convention suggested by the Ubuntu to identify those images. Because of my trust in startupcloud.com I can reasonably assume that these images are what they expect. If glance provides me the MD5SUM of the image that i'm going to run, I can actually, then verify that against the data in /query2 and have even higher confidence that this is what I think it is. Here, startupcloud.com could have glance lying to me, thats true, but I'm already stuck trusting them. So this is just a more specific bit of information available. c.) joeuser starts using startupcloud.com and likes them, but thinks they can improve on their Ubuntu images, so he starts making public images named joeuser/ubuntu-images/ubuntu-naming-convention I try one of these, and start to develop a trust in joeuser as a provider. Now, I can use his public gpg keys and trust him. On Tue, Apr 10, 2012 at 8:04 AM, Scott Moser smo...@ubuntu.com wrote: The data you're after might be useful to you, and might scratch an itch. I will not discount that, but I would much prefer a bit of metadata associated with an image that was signed by an entity I trusted that identified the image as good. I have to trust my cloud provider. A single protected flag in metadata saying official cloud image is no less secure than anything more complicated at the moment (sadly) Agreed. OS distro, version_major, version_minor are even less important where you don't care (or know) that your OS came from Canonical or RedHat, what you were really interested in is running WhizBang! Fooberator version 2.0. Unless the distros stop changing config directory locations, or agree a common init.d approach, then this simply isn't true. Well, in the case that you care about that, yes. In that case, you need to know things like that. But largely that can be determined from inside the instance, and is usually handled by tools accounting for such differences. Maybe you're talking about running pre-built appliances? I'm talking about not treating the machines as infallible black boxes (I think mine is the more common use case, but irrespective, mine is definitely a valid use case) The Ubuntu images are pre-built appliances. All images are in some sense. They come from a vendor, which you hopefully have some level of trust in. You interact with the appliance via some interface. Some might be a web interface, some might be cloud-init, some might be puppet or chef. Its all just use of an interface. The OS is just a means to an end. Your case is valid, yes. but if you have public images, then you cannot trust the content of the image with out trusting the source. So user provided tags (or even tags pulled by libguestfs) are useless. It's very important to me as a consumer of images. How are you coding image selection for launching instances on the public OpenStack clouds? I'm interested in any alternative. I guess largely my issue is that your wanting openstack to define a namespace for users of it to adhere to. You're suggesting that in that namespace shoudl be things like os_version, which I do believe is important, but not all important. Having recently played with iscsi, I read the naming convention bits at http://tools.ietf.org/html/rfc3721#section-1.1 . I feel that it would
Re: [Openstack] Metadata and File Injection (code summit session?)
On Tue, 10 Apr 2012, Justin Santa Barbara wrote: Config drive can support all EC2 functionality, I believe. Images would need to be respun for OpenStack with config-drive, unless we populated the config drive in a way that worked with cloud-init. (Scott?) cloud-init supports config-drive now. https://lists.launchpad.net/openstack/msg07721.html Personally, I'd rather our effort went into producing great images for OpenStack, than into compatibility with last-generation clouds. Any idea what the important EC2 images are? Are there a handful of images that we could duplicate and then just forget about EC2? Stock OS images are boring, but i've ranted at you about that on another thread :) My $0.02 on the config drive stuff, has been said at https://lists.launchpad.net/openstack/msg07585.html and https://lists.launchpad.net/openstack/msg07633.html I will try to update the data for the session at http://summit.openstack.org/sessions/view/100 and have a solid proposal on a simplish config-drive suggestion. bring tomatoes. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] python-glanceclient
On Thu, 5 Apr 2012, Dan Wendlandt wrote: Hi Joe, I've often had a similar thought myself of whether a single 'osclient' would be better. I see consistency across all clients as the bare minimum, and a single client as an interesting option to explore. Thanks for driving this Dean, I look forward to talking about it at the summit. I don't know if there is another session scheduled already or not, but I had added this one a week or so ago. http://summit.openstack.org/sessions/view/99 Either we can use that one, or ditch replace it with any other session that had been opened. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] OVF vs. bare container formats for qcow2 images
On Wed, 28 Mar 2012, Lorin Hochstein wrote:
All:
Given that I have a qcow2 image from somewhere (e.g., downloaded it from a
uec-images.ubuntu.com, created one from a raw image using qemu-img) that i
want to add to glance:
1. How can I tell whether it's an ovf or bare container format?
2. Why does it matter?
I dont know either, but I do know what you want, taken from stack.sh in
devstack, which should probably be updated to not use '-A'
glance add -A $TOKEN \
name=${IMAGE_NAME%.img} is_public=true
container_format=ami disk_format=ami ${IMAGE}
You can also add ramdisk_id= and kernel_id= tags in the upload
if you want that.
Note, you can do this with the Ubuntu cloud-images, and that is the best
way to put images for openstack into glance.
$
url=https://cloud-images.ubuntu.com/precise/current/precise-server-cloudimg-amd64-disk1.img
$ wget $url -O ${url##*/}
$ glance add name=${url##*/} is_public=true container_format=ami \
disk_format=ami ${url##*/}
Whenever I add a qcow2 image to glance, I always choose ovf, even
though it's probably bare, because I saw an example somewhere, and it
just works, so I keep doing it. But I don't know how to inspect a binary
file to determine what its container is (if file image.qcow2 says it's
a QEMU QCOW2 Image (v2), does that mean it's bare?). In particular,
why does the user need to specify this information?
Also, are there any Linux command-line tools for inspecting/manipulating OVF
containers?
Theres open-ovf, which has my name on it, but is really abandoned.
from my perspective, the OVF support in glance/openstack is really to be
ignored.
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] OVF vs. bare container formats for qcow2 images
On Thu, 29 Mar 2012, Daniel P. Berrange wrote: On Wed, Mar 28, 2012 at 04:41:28PM -0400, Lorin Hochstein wrote: All: Given that I have a qcow2 image from somewhere (e.g., downloaded it from a uec-images.ubuntu.com, created one from a raw image using qemu-img) that i want to add to glance: 1. How can I tell whether it's an ovf or bare container format? You are mixing up terminology here. Disk image formats are things like raw, qcow2, vmdk, etc. OVF refers to the format of a metadata file provided alongside the disk image, which describes various requirements for running the image. The two are not tied together at all, merely complementary to each other. Well, the implementation in glance is not really aligned with reality. There was discussion on the list a while ago. http://www.mail-archive.com/openstack@lists.launchpad.net/msg05803.html that has more info. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] problem ssh-ing into vms
On Wed, 28 Mar 2012, Pierre Amadio wrote: Hi there ! I'm puzzled with a ssh issue. Things used to work before, and now, i cannot access my vms anymore. Thank you for attaching euca-get-console-output. That provides the real hint here. | cloud-init start-local running: Wed, 28 Mar 2012 13:09:35 +. up 43.07 seconds | no instance data found in start-local | ci-info: lo: 1 127.0.0.1 255.0.0.0 | ci-info: eth0 : 1 10.0.0.3255.255.255.0 fa:16:3e:59:63:1b | ci-info: route-0: 0.0.0.0 10.0.0.10.0.0.0 eth0 UG | ci-info: route-1: 10.0.0.00.0.0.0 255.255.255.0 eth0 U | cloud-init start running: Wed, 28 Mar 2012 13:09:48 +. up 56.41 seconds | 2012-03-28 13:09:49,331 - DataSourceEc2.py[WARNING]: 'http://169.254.169.254' failed: url error [[Errno 111] Connection refused] | 2012-03-28 13:09:50,349 - DataSourceEc2.py[WARNING]: 'http://169.254.169.254' failed: url error [[Errno 111] Connection refused] | ... | 2012-03-28 13:11:29,162 - DataSourceEc2.py[WARNING]: 'http://169.254.169.254' failed: url error [[Errno 111] Connection refused] | 2012-03-28 13:11:35,174 - DataSourceEc2.py[CRITICAL]: giving up on md after 105 seconds The instance was unable to connect to the metadata service to get your public keys and put them into the instance. The Ubuntu images have an annoyance, in that if it can't get to the MD it doesn't even generate ssh server keys, so the ssh server just drops you immediately. But your real problem is that the guests can't see the host. You could potentially debug some more by using a cirros image, which will let you do password auth in after it fails to reach the data source. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] KVM disk performance
Can you provide information about what version of kvm you're using ? I suspect this is really unrelated to openstack specifically. On Mon, 26 Mar 2012, Martin van Wilderen - JDN BV wrote: Hi List, I have a question about KVM disk performance. We are using Openstack Nova on three machines. These machines have a SSD drive with a dd write performance of about 130mb/s Within the instance the write performance is down to about 5 mb/s. When using the allocate trick (dd zero to disk before newfs) we get a performance of 20 mb/s. Things a have tried but don't give any extra results are: - Settings the disklayout from qcow2 to raw - Settings the cache type in libvirt.xml (writeback, writethrough, none) - Switching KSM on and off. - Tested with different guests OS, Linux, FreeBSD, Windows. Is there someone who had some extra info i can check? Or are there more people with this issue? Snippet from libvirt.xml driver type='qcow2'/ cache='writeback'/ source file='/var/lib/nova/instances/instance-0113/disk'/ target dev='vda' bus='virtio'/ Kind regards, Martin ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] KVM disk performance
On Mon, 26 Mar 2012, Martin van Wilderen - JDN BV wrote: Hi Scott, kvm --version QEMU emulator version 0.13.91 (qemu-kvm-0.14.0-rc1), Copyright (c) 2003-2008 Fabrice Bellard virsh --version 0.8.8 uname -r 2.6.35-22-server Need any more info? Well, unfortunately there is lots more info needed, and I'm not really able to dedicate resources to debugging this at this point. I was curious because I had circumstantial evidence pointing to qemu-kvm 1.0 (in ubuntu 12.04) being slower than 11.10. However, your data above is similar to what I've seen in 12.04. Most definitely, if you want write performance, you will need fully allocated disks. The qcow sparse is good for quick deployment, but not good for write speed. You may also want to try using qed format disk as that is supposed to be a bit better. I had one kvm person suggest looking at barriers of the host's ext4 filesystem, saying: if you have ext4 filesystem underneath with barriers enabled, then you'll find that the guest will submit barriers to the host filesystem which will end up flushing the device cache A quick google showed http://blog.smartlogicsolutions.com/2009/06/04/mount-options-to-improve-ext4-file-system-performance on generally how to get some additional performance out of ext4. I'm interested in hearing what you find out, but unfortunately at the moment, I dont' have a magic bullet for you. Met vriendelijke groet, Martin van Wilderen Technical Manager On Mar 26, 2012, at 5:12 PM, Scott Moser wrote: Can you provide information about what version of kvm you're using ? I suspect this is really unrelated to openstack specifically. On Mon, 26 Mar 2012, Martin van Wilderen - JDN BV wrote: Hi List, I have a question about KVM disk performance. We are using Openstack Nova on three machines. These machines have a SSD drive with a dd write performance of about 130mb/s Within the instance the write performance is down to about 5 mb/s. When using the allocate trick (dd zero to disk before newfs) we get a performance of 20 mb/s. Things a have tried but don't give any extra results are: - Settings the disklayout from qcow2 to raw - Settings the cache type in libvirt.xml (writeback, writethrough, none) - Switching KSM on and off. - Tested with different guests OS, Linux, FreeBSD, Windows. Is there someone who had some extra info i can check? Or are there more people with this issue? Snippet from libvirt.xml driver type='qcow2'/ cache='writeback'/ source file='/var/lib/nova/instances/instance-0113/disk'/ target dev='vda' bus='virtio'/ Kind regards, Martin ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] cloud-init support for config-drive
Hey all,
I recently (yesterday) added 'configdrive' support to cloud-init.
The most recent daily builds of ubuntu 12.04 (precise) [1] should
be able to take advantage of configdrive.
I tried to document what I have at
http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc/configdrive/README
I'd really like people to take it for a spin. From the code I've read,
it looks like I need to setup multinic in order to get 'interfaces' file
inserted, so I've not actually tested that path. I'm not sure how to
configure multinic (and get nova command line to use '-nic' correctly).
Anyway, the most recent cloud images at
https://cloud-images.ubuntu.com/server/precise/current/ (after 20111216)
have support baked in.
Example of how to use it below. The goal is that you can now use
cloud-init with configdrive and no EC2 metadata service.
Please give it a try and let me know what you find.
## Now, 'ud_value' has same content of my-user-data file, but
## with the string USER_DATA FROM META KEY
## launch an instance with dsmode=pass
## This will really not use the configdrive for anything as the mode
## for the datasource is 'pass', meaning it will still expect some
## other data source (DataSourceEc2).
$ nova boot --image=$img_id --config-drive=1 --flavor=$flav_id \
--key_name=$keyname \
--user_data=my-user-data \
--meta=instance-id=iid-001 \
--meta=user-data=${ud_keyval} \
--meta=dsmode=pass cfgdrive-dsmode-pass
$ euca-get-console-output i-001 | grep USER_DATA
echo USER_DATA FROM EC2 MD | tee /ud.log
## Now, launch an instance with dsmode=local
## This time, the only metadata and userdata available to cloud-init
## are on the config-drive
$ nova boot --image=$img_id --config-drive=1 --flavor=$flav_id \
--key_name=$keyname \
--user_data=my-user-data \
--meta=instance-id=iid-001 \
--meta=user-data=${ud_keyval} \
--meta=dsmode=local cfgdrive-dsmode-local
$ euca-get-console-output i-002 | grep USER_DATA
echo USER_DATA FROM META KEY | tee /ud.log
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] RHEL / CentOS - interfaces.template
On Wed, 15 Feb 2012, Richard W.M. Jones wrote:
On Tue, Feb 14, 2012 at 10:56:18PM +, Pádraig Brady wrote:
The netcf lib looks interesting. Perhaps it could leverage
libguestfs (already integrated) to maximise the types and
configurations of guests it could target?
It's an interesting thought, not one that I'd really thought about
before. It may be possible to do something like using guestmount[1]
to mount the guest's disk(s) and then run netcf in a chroot on those
disks. That would seem to be the minimal disruption to netcf, but I
will have to discuss this with Laine Stump and look at the netcf code ...
Its possible I just wasn't clear in my original description, but I really
prefer *no* mounting of guests or expectations of their filesystems or
knowledge of their internals.
I whole-heartedly agree that libguestfs provides an awesome feature set,
and if injection needs to be done, its probably the right way to do it.
However, even with libguestfs, a 'chroot' makes all sorts of assumptions
about the guest that you don't *have* to make if you just attach a
config_drive with a description of the network interfaces to the OS and
let it deal with that.
Personally, the ideal solution to me is that every guest has a management
network (which likely comes up as eth0 -- i realize that is non-trivial).
On that network, dhcp is available, and after successful acquisition of
address via that interface, a network service is available. That network
service then provides network configuration for other devices that are
attached. Example hack-like script that I'd add to cirrOS.
dhclient -i eth0
wget http://169.254.169.254/netcfg/1.0 -O out
cirrosNetFromNetCfg out
Where 'cirrosNetFromNetCfg' is some utility that I write for cirros that
knows how to turn netcfg into cirros specific network configuration.
Apparently, having dhcp even on a management only network is
non-desireable in some cases. In *that* case, my preferred mechanism is
changed a bit, and looks like this to support both cases:
cfg_drive_candidate=$(tail -n 1 /proc/partitions)
netcfg=/tmp/netcfg
if [ -b $cfg_drive_candidate ]
mount -t vfat $cfg_drive_candidate /mnt; then
if [ -e /mnt/netcfg/1.0 ]; then
cp /mnt/netcfg/1.0 $netcfg
netcfg=/tmp/netcfg
fi
umount /mnt
fi
if ! [ -e $netcfg ]; then
dhclient -i eth0
wget http://169.254.169.254/netcfg/1.0 -O $netcfg
fi
[ -e $netcfg ] cirrosNetFromNetCfg $netcfg ||
{ echo FAIL! no netcfg!; exit 1; }
That way, as the author of cirros, if I want it to run in openstack, I can
do everything myself. I don't have to first get 'netcfg' to support
cirros configuration or worry about getting a tiny bug fixed in it and
propogated back to versions that would be running on old hosts. I don't
have to have pick a filesystem that guestfs understands, or make sure that
some random linux version is going to be able to 'chroot' into a
filesystem after it mounted it.
All I had to do was:
a.) support vfat (and virtio)
b.) write 'cirrosNetFromNetCfg'
c.) support dhcp
The same general solution can also apply to ssh keys, although I
personally think that network configuration is the only thing that I
*need* to bootstrap, and then its best to have network metadata services
rather as opposed to putting more stuff into the config_drive.
IMO its best to put burdon on image creators. It puts power in their
hands and makes them able to make images that say
Runs on Openstack Essex without saying assuming you have libguestfs
support for btrfs and netcfg version 3.2.___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] RHEL / CentOS - interfaces.template
On Wed, 15 Feb 2012, Joshua Harlow wrote: Sure that makes sense (less dependency on guest file-systems). Although one of my concerns was that I thought this config drive stuff was only in the openstack api and not in the EC2 one. So that limits the market there (especially as openstack really needs some love given to the EC2 stuff)? You could most certainly just attach network config-drive by default if nova is not expecting dhcp to work, then that covers the ec2 api case. It seems like cloud-init could work with this netcfg service and setup the network, that might be the best approach. Its just cloud-init needs to be made less distro-centric. Is there any plans for that? It seems like it could do this (or interact with python-netcf). -Josh I have no objections to cloud-init being less distro-centric. Credit to Garrett Holmstrom , cloud-init is now in Fedora. Its not finished [1], but, the work is at least known. [1] http://pkgs.fedoraproject.org/gitweb/?p=cloud-init.git;a=blob;f=cloud-init-README.fedora;h=99bf7ab9095d09a6c5435d11138d81e5574a45f7;hb=HEAD I personally would much prefer less interaction with the guest as opposed to more. I realize people will argue for this having something just *try* to configure the guest's networking via mount and /etc/ insertion of some mechanism or another. But the issue with that is it can go wrong, breaking the image, or losing data (ie, by incompletely updating/overwriting /etc/network/interfaces). I'm willing to add a blueprint for openstack F and implement network config-drive basically as I've described in the other post. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] RHEL / CentOS - interfaces.template
On Tue, 14 Feb 2012, Leandro Reox wrote: Hi guys, Anyone already implemented networking injection to RHEL systems acting as a guest ? If no any plans to make it to Essex final ? Before we go down the road of trying to write system network configuration scripts for each potential guest OS, I'd suggest that its would be better to either: a.) just accept that 'interfaces' is the openstack format and guests should need to read that. b.) create a OS agnostic interface configuration format. While you may be looking at my email address and assuming that I think 'a' is the right answer only to make it harder for anyone else. However, the reason I dislike the current solution (or going down a path of implementing population mechanisms for other operating systems) is 1.) you cannot possibly support all possible operating systems 2.) injecting files assumes host OS knowledge (or guestfs knowledge) of the guest filesystem 3.) specific files indicates that the host can somehow determine which format the guest is expecting (or also unacceptable, only allowing this configuration for one OS per cloud). 4.) injecting files via overwriting them is lossy and possibly destructive to a guest (imagine other vpn routes inserted there or something else more advanced). I would *much* rather there be a openstack networking configuration file format that was put into config_drive if dhcp was not sufficient. Then, the guests are just made to read that information that is in a standard, easily documetable format and respond accordingly. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Metadata and File Injection
On Thu, 15 Dec 2011, Vishvananda Ishaya wrote: On Dec 15, 2011, at 10:37 AM, Scott Moser wrote: What does a Server Personality give me that cannot be accomplished accomplished by guest and lauching-entity agreeing on a key-value pair with keyname server-personality. I think you are stating the same thing here. Basically repurposing the existing personality code to be key, value instead of filename, data Then I violently agree. Sorry for additional spam. Jesse, Thank you for leading this. I think its very valuable. I've just now subscribed to http://wiki.openstack.org/guest-configuration, but is there some other way I can make sure I follow this? I'd like to have cloud-init and 12.04 images using the preferred mechanism. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Metadata and File Injection
On Fri, 16 Dec 2011, John Garbutt wrote: One use case is providing metadata to an instance before that instance starts their network stack. A bit of an edge case, but I guess one instance of this use case is the network injection logic used for Flat networking. Would injected files cover the above case? Would it still work after the proposed changes? I guess it should work, assuming nova injects the files in the same way it does for injecting network configuration, rather than relying on the agent to write the files once the system has booted by getting data from the metadata service. Maybe this is what the configuration drive could be used for (assuming either the agent isn't responsible for correctly mounting the configuration drive, or it can mount the disk before it has access to any network)? I think it is reasonable to inject static networking configuration via configuration drive. It could even be re-named network-configuration-drive if there is seen to be no other use for it. I'd add code to cloud-init for ubuntu configuration based on that. I've been meaning to get around to it anyway, I just haven't done it. I think it would be quite clean to do logic on boot like: * if a scan of block devices shows one labeled netconfig * mount that (iso9660 or vfat) filesystem, read /config file in its root filesystem. * modify /etc/network/interfaces accordingly * continue on with boot. The only issue above that there is nothing that guarantees that the config drive volume is going to be present at the instant in boot when I start looking (it may just not have shown up yet on the scsi bus or whatever). If its not there, and cloud-init goes on with life it may expose a race. I'd have to think about that more. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Metadata and File Injection
On Thu, 15 Dec 2011, McNally, Dave wrote: Thanks for the responses guys. Jesse I can see how there might be concerns around using file injection but it seems as if that was chosen as a method of inserting user-specified metadata into an instance which is why I was wondering why it wasn't expanded to use the same process to insert files? The biggest flaw in this is that it expects that the host can read a filesystem of the guest. That is simply *guaranteed* to result in a case where this is broken for future guests (which use btrfs, btrfs2.0 or smoserFS-1.2). I'm happy that a network based metadata service is generally seen as the smartest solution. This way its up to the guest to fit into the cloud, no changes to the hypervisors or openstack have to be made to support specific guests. Dave -Original Message- From: Jesse Andrews [mailto:anotherje...@gmail.com] Sent: 15 December 2011 08:34 To: Jay Pipes Cc: McNally, Dave; openstack@lists.launchpad.net Subject: Re: [Openstack] Metadata and File Injection Great question. Right now there are 3 approaches to metadata/runtime config: * ec2 metadata service - http://169.254.169.254/ (used by ubuntu's cloud-init for example) * config drive - added in diablo * xenstore via openstack agent - https://launchpad.net/openstack-guest-agents * injecting files into the filesystem prior to launch - on some hypervisors / filesystems combinations There are issues with each approach currently: * metadata service - requires that cloud uses DHCP to configure network * config drive - only sets values at runtime and no support from any guests * xenstore approach - hypervisor specific. * injecting into filesystem - brittle as it requires the host to support arbitrary filesystems If we assume that instance networking can be configured by DHCP, then a metadata service is probably our best choice for simplicity of implementation and user experience. Given that Rackspace Public Cloud is the only openstack developer/user (I know of) that doesn't use DHCP for network configuration of instances, I reached out to the team to see if DHCP would be an option. They are researching either a way use DHCP _or_ propose an approach to initial network configuration that can be executed by the openstack guest agent. Based on those conversations we've been writing a proposal that would recommend guests use a (currently non-existing) openstack metadata api - http://wiki.openstack.org/guest-configuration Hope this helps. Thoughts on the optimal experience for essex? On Wed, Dec 14, 2011 at 9:28 AM, Jay Pipes jaypi...@gmail.com wrote: On Wed, Dec 14, 2011 at 9:04 AM, McNally, Dave dave.mcna...@hp.com wrote: Hi, I've recently been looking at file and metadata injection in Nova and I have a question relating to it. (BTW this is based off what I have seen in nova/virt/disk.py) I notice that for key/value pairs specified as metadata during boot of an instance these values are injected into a file /meta.js in the instance. However if a file (and corresponding injection location) are specified when booting the instance the file does not get injected. Not on libvirt/KVM. It works on Xen, though: https://bugs.launchpad.net/nova/+bug/755168 I was wondering if there was an intentional decision not to use a similar method to that used when injecting meta.js to inject other files? Because it seems to me the addition of such functionality would be fairly straightforward. I'm interested in the answer to this question as well... Also on a vaguely related note why is the metadata injected into a file rather than stored in a location where it can be retrieved from the metadata service? No idea. :( Cheers, -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Metadata and File Injection
On Thu, 15 Dec 2011, Jesse Andrews wrote: Great question. Right now there are 3 approaches to metadata/runtime config: * ec2 metadata service - http://169.254.169.254/ (used by ubuntu's cloud-init for example) * config drive - added in diablo * xenstore via openstack agent - https://launchpad.net/openstack-guest-agents * injecting files into the filesystem prior to launch - on some hypervisors / filesystems combinations There are issues with each approach currently: * metadata service - requires that cloud uses DHCP to configure network * config drive - only sets values at runtime and no support from any guests * xenstore approach - hypervisor specific. * injecting into filesystem - brittle as it requires the host to support arbitrary filesystems If we assume that instance networking can be configured by DHCP, then a metadata service is probably our best choice for simplicity of implementation and user experience. Given that Rackspace Public Cloud is the only openstack developer/user (I know of) that doesn't use DHCP for network configuration of instances, I reached out to the team to see if DHCP would be an option. They are researching either a way use DHCP _or_ propose an approach to initial network configuration that can be executed by the openstack guest agent. Based on those conversations we've been writing a proposal that would recommend guests use a (currently non-existing) openstack metadata api - http://wiki.openstack.org/guest-configuration I'm just curious, what are the motivations behind inventing something other than the EC2 Metadata service? It is generally functional, and quite a lot can (and has) built atop this simple service. Secondly, There are 2 features that I feel are missing from the metadata service. And I'd hope that these could be accounted for if there is going to be invention done. a.) user-data is a single entity. There are potentially multiple sources that want to provide input to a guest (the end user might want to install some packages at boot, and the cloud infrastructure might want to tell the guest of a local mirror). cloud-init supports multipart-mime in userdata, so that there can be separate pieces inside that single source, but even then, all parties involved have to agree that they do not completely own that resource. b.) There is no way to disable it. cloud-init supports writing a null-route to the metadata service, which can make it inaccessible to any non-root entity on the system. However, it would be nicer if there was a way to disable it entirely. With that in place, passing credentials into the guest would be easier as once they're consumed they can be removed. c.) No way to modify contents of the service after instance launch. OK, I said 2 features, and really this one is wishlist. If we had an arbitrary key-value store that was available, the user could interact with the guest using this service. It'd have to be poll-based, but a in-guest hypervisor could watch for creation and deletion of keys. Potentially, the MD might be RW from inside guest, meaning it could even convey information back. I consider this wishlist because really, there are perfectly good solutions for communicating to in-guest agents, theres not a lot of reason to invent a new one. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Metadata and File Injection
On Thu, 15 Dec 2011, Jesse Andrews wrote: On Thu, Dec 15, 2011 at 8:43 AM, Scott Moser smo...@ubuntu.com wrote: I'm just curious, what are the motivations behind inventing something other than the EC2 Metadata service? It is generally functional, and quite a lot can (and has) built atop this simple service. I should clarify - the idea is that ec2 metadata service is a great starting point - but there are already a handful of additions that should be added. In the existing (ec2) metadata service the urls look like: http://169.254.169.254/(version)/(resource) My hope is that we can add features like: * injected files / personality (or you can think of them as multiple named userdata sections) See below, but I don't see a reason to call something injected files, as opposed to a key of: files/etc/passwd : contents here and letting the guest agents and consumers sort out naming conventions. * updatable metadata params (the key/value params in the openstack api) with CRUD (so you can remove/update from guest) Secondly, There are 2 features that I feel are missing from the metadata service. And I'd hope that these could be accounted for if there is going to be invention done. a.) user-data is a single entity. There are potentially multiple sources that want to provide input to a guest (the end user might want to install some packages at boot, and the cloud infrastructure might want to tell the guest of a local mirror). cloud-init supports multipart-mime in userdata, so that there can be separate pieces inside that single source, but even then, all parties involved have to agree that they do not completely own that resource. The multipart-mime stuff is a great hack but isn't very user friendly as you state. In the openstack api we have server personalities that are similar to userdata except you can have multiple of them and they are named. http://docs.openstack.org/api/openstack-compute/1.1/content/Server_Personality-d1e2543.html If the guest (client) is in charge of pulling the personality data instead of the host injecting it, would this fulfill the usecase? I find specific designed use like this to be unnecessary. EC2 basically says heres a place to store a blob of data, do with it what you want. From that, best practices, good ideas, bad ideas, and even standards will evolve. What does a Server Personality give me that cannot be accomplished accomplished by guest and lauching-entity agreeing on a key-value pair with keyname server-personality. Basically, I'm saying dont bother with special keys/locations in a spec if there is no reason to. b.) There is no way to disable it. cloud-init supports writing a null-route to the metadata service, which can make it inaccessible to any non-root entity on the system. However, it would be nicer if there was a way to disable it entirely. With that in place, passing credentials into the guest would be easier as once they're consumed they can be removed. The usecase of passing credentials is one reason we want to have the metadata passed via the openstack API mutable via the metadata service http://docs.openstack.org/api/openstack-compute/1.1/content/Server_Metadata-d1e2529.html A hashed password could be provided to the guest via the metadata service in a key bcrypt_password. The guest agent could retrieve the password and then DELETE it: curl -X DELETE http://169.254.169.254/os/meta/bcrypt_password Well, theres no reason it can't be plaintext either. but, yes, thats basically what i'm hoping for. c.) No way to modify contents of the service after instance launch. OK, I said 2 features, and really this one is wishlist. If we had an arbitrary key-value store that was available, the user could interact with the guest using this service. It'd have to be poll-based, but a in-guest hypervisor could watch for creation and deletion of keys. Potentially, the MD might be RW from inside guest, meaning it could even convey information back. I consider this wishlist because really, there are perfectly good solutions for communicating to in-guest agents, theres not a lot of reason to invent a new one. The openstack API already has CRUD of the key/value metadata in the non-metadata (regular) API. http://docs.openstack.org/api/openstack-compute/1.1/content/MetadataSection.html By exposing this through the metadata service I think we get what you are asking for. My wishlist is the able to do long-polling against the metadata service to be alerted of changes from the guest :) I dont know that its necessary is the only thing. What is the scenario where polling this metadata service is better than the instance polling another network resource? Or listening to a SQS or zookeeper, or anything. I agree that it seems useful (I brought it up), but I honestly
Re: [Openstack] [GLANCE] Proposal: Combine the container_format and disk_format fields in 2.0 Images API
On Fri, 2 Dec 2011, Soren Hansen wrote: 2011/12/1 Jay Pipes jaypi...@gmail.com: structure tar'd up. However, I think this can be more easily accomplished by consolidating the disk and container formats in the 2.0 API to just a single format field with the possible values: ova - This indicates the data stored in Glance is an OVF container that may actually contain multiple virtual appliances that has been tar'd into the single-file OVA format raw - This is an unstructured disk image format vhd - This is the VHD disk format, a common disk format used by virtual machine monitors from VMWare, Xen, Microsoft, VirtualBox, and others vmdk - Another common disk format supported by many common virtual machine monitors vdi - A disk format supported by VirtualBox virtual machine monitor and the QEMU emulator iso - An archive format for the data contents of an optical disc (e.g. CDROM). qcow2 - A disk format supported by the QEMU emulator that can expand dynamically and supports Copy on Write aki - This indicates what is stored in Glance is an Amazon kernel image ari - This indicates what is stored in Glance is an Amazon ramdisk image ami - This indicates what is stored in Glance is an Amazon machine image What do people think of this proposal to combine the two into a single format field? I agree the current disk_format/container_format tuple isn't ideal. There's overlap between the two and at the same time, there are things that can't be expressed with the current selection of valid settings. I do think having two separate fields defining the contents, though. There are basically two things that are relevant: The image type and the container format. The image type can be either of kernel, ramdisk, filesystem, iso9660, disk, or other. The container type can be: raw, cow, qcow, qcow2, vhd, vmdk, vdi or qed (and probably others I've forgotten). I generally agree with Soren here, but Jay originally asked: | The problem I see is that really OVF is the only real container | format and I'm just not sure it's useful to have users set a | container format. The goal was to allow Glance to report that the | image file stored in Glance is an OVA file and not an image file | itself. An OVA file is a single file that contains the OVF directory | structure tar'd up. However, I think this can be more easily | accomplished by consolidating the disk and container formats in the | 2.0 API to just a single format field with the possible values: OVF is neither container format, nor image [content] type. It is a thing that deals with a higher level object. These things are VirtualSystems and VirtualSystemCollections. A VirtualSystem has Disk elements, which have an ovf:format attribute which would contain a value describing the same thing as container type above. A VirtualSystemCollection defines a group of VirtualSystems and relationships between them. One thing Jay did not mention (which seems relevant to me) is that he really described OVA. OVF is a superset of OVA which additionally describes a set of files description format, where you have the same ovf manifest file, but 'File' elements have 'ovf:href=' attributes which can be arbitrary uri. The use of that could potentiall be a .ovf file stored somewhere [possibly glance] with 'File' elements that reference glance:// files. Almost certainly, though, if you *were* using OVF like that, you'd need to have urls that referenced something other than disks (such as manifest file and language files. Any way you look at it, OVF is definitely the oddball here, and I do not think it really fits where it is put. Is anything actually using OVA from glance?___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Cloud-init server?
On Thu, 3 Nov 2011, Joshua Harlow wrote: Hi I am wondering if there is any documentation on the server that is created for cloud-init to connect to? I'm guessing you're referring to the meta-data service. This is implemented in nova/api/ec2/cloud.py . It basically mimics the ec2 metadata service http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/index.html?AESDG-chapter-instancedata.html I have read online about how a iptables route is setup, but does that just forward back to a openstack api (the metadata api?) I am trying to track down how userdata gets eventually fetched by cloud-init (or other service) since in diablo it doesn't seem like its placed in the image anywhere. It generally *does* work in in diablo, so there must be some issue (as you suspect) on your local usage. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Cannot boot from volume with 2 devices
On Wed, 2 Nov 2011, Gaurav Gupta wrote: On Tue, Nov 1, 2011 at 3:51 PM, Scott Moser smo...@ubuntu.com wrote: On Tue, 1 Nov 2011, Vishvananda Ishaya wrote: Sounds like we can work around this pretty easily by sorting the disks before we pass them into the xml template. The long term solution here is not to load the kernel and the ramdisk outside the image, but rather let grub load it with root=LABEL= or root=UUID= . If you boot one of the full disk Ubuntu image (-disk1.img) files at https://cloud-images.ubuntu.com/releases/oneiric/release/ or https://cloud-images.ubuntu.com/server/natty/current/ , then you wont have the problem. You'll also be able to 'apt-get update apt-get dist-upgrade reboot' and get a new kernel. That is not possible with the hypervisor doing the kernel and ramdisk loading. Actually you can create a bootable image where the kernel and ramdisk are picked from the root file system. AFAIU you can't do this with volumes thought, you have to create an image and boot from it. I'll try to take a look at this next week, and if it isn't functioning correctly try to get it addressed in essex. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Cannot boot from volume with 2 devices
On Tue, 1 Nov 2011, Vishvananda Ishaya wrote: Sounds like we can work around this pretty easily by sorting the disks before we pass them into the xml template. The long term solution here is not to load the kernel and the ramdisk outside the image, but rather let grub load it with root=LABEL= or root=UUID= . If you boot one of the full disk Ubuntu image (-disk1.img) files at https://cloud-images.ubuntu.com/releases/oneiric/release/ or https://cloud-images.ubuntu.com/server/natty/current/ , then you wont have the problem. You'll also be able to 'apt-get update apt-get dist-upgrade reboot' and get a new kernel. That is not possible with the hypervisor doing the kernel and ramdisk loading. This is assuming that in the multiple-disks-attached scenario, the *real* root disk (the one with the bootloader on it) is found by bios. static device names were deprecated several years ago by all linux distributions. Lets move towards using the better solution. Vish On Nov 1, 2011, at 9:52 AM, Gaurav Gupta wrote: Hi all, I asked a question on Launchpad. but haven't heard back anything yet. Trying this forum to see if someone has any idea how to resolve this issue: https://answers.launchpad.net/nova/+question/176938 To summarize: -- Say I had 2 disks, disk1 and disk2 (represented by 2 volumes). disk1 has the root-file-system and disk2 has some data. I boot an instances using the boot-from-volumes extension, and specify the 2 disks such as disk1 should be attached to /dev/vda and disk2 to /dev/vdb. When the instance is launched it fails to boot, because it tries to find the root-filesystem on disk2 instead. The underlying problem is with virsh/libvirt. Boot fails because in the libvirt.xml file created by Openstack, disk2 (/dev/vdb) is listed before disk1 (/dev/vda). So, what happens is that the hypervisor attaches disk2 first (since its listed first in the XML). Therefore when these disks are attached on the guest, disk2 appears as /dev/vda and disk1 as /dev/vdb, which causes the boot failure. Later the kernel tries to find the root filesystem on '/dev/vda' (because thats' what is selected as the root) and it fails for obvious reason. I think it's a virsh bug. It should be smart about it and attach the devices in the right order. ___ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] ANN: (Unofficial) OpenStack Diablo packages for Ubuntu Oneiric
On Tue, 25 Oct 2011, Kiall Mac Innes wrote: On Oct 25, 2011 8:04 p.m., Dave Walker davewal...@ubuntu.com wrote: Can you point me to this discussion? It sound awfully like there has been some confusion.. Who was it that said this? It was on one of the OpenStack or Ubuntu IRC channels a week or so ago, I'd have to go digging through logs (if there are any public logs) to remember the specifics! It was probably me, Dave, and I pinged you and zul also to verify or confirm, and said that I wasn't sure. 10/20/11 17:06:41 Kiall smoser, BTW, any idea if openstack keystone in oneiric will be fixed relatively soon? (as in .. next month or so..) 10/20/11 17:06:55 Kiall (basically .. will I get it going from git, or wait) 10/20/11 17:08:03 smoser ask zul or Daviey but i would really not expect keystone fix in official ubuntu updates 10/20/11 17:08:09 smoser i could be wrong though. $ bzr bd -S ; cd ../ ; dput *.change (or pbuilder/sbuild) -S means source package (more details below) This part I get, I'm on my phone so I'll have to go into more detail another time.. Thanks, Kiall ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] can't ssh,but ping is OK
On Wed, 19 Oct 2011, aliex_liu wrote: /NOVA-INIT/instances/instacnes-002a/console.log when i run a instance,the default route has been deleted, and why? can i add the default route? only one network interface eth0 A more complete console log would be useful. The instance can't seem to get to the metadata service, so its failing. You can probably get better debug information out of an 11.10 instance, or try the ttylinux images as they'll still let you in so you can potentially ssh with password auth and look around from inside the instance. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] using uec qcow2 images in openstack?
On Mon, 26 Sep 2011, Jesse Andrews wrote: I can add ami style images to diablo nova/glance via: wget http://uec-images.ubuntu.com/natty/current/natty-server-cloudimg-amd64.tar.gz tar -zxvf natty-server-cloudimg-amd64.tar.gz glance add -A KEYSTONE_TOKEN name=uec-natty-kernel is_public=true container_format=aki \ disk_format=aki natty-server-cloudimg-amd64-vmlinuz-virtual # use the returned id as the kernel for the image glance add -A KEYSTONE_TOKEN name=uec-natty is_public=true container_format=ami kernel_id=4 \ disk_format=ami natty-server-cloudimg-amd64.img Just in case you were unaware, Ubuntu ships 'cloud-utils' that makes the above easier. cloud-publish-tarball natty-server-cloudimg-amd64.tar.gz my-bucket x86_64 BUT I don't know how to use the newer oneiric qcow2 image: wget http://uec-images.ubuntu.com/oneiric/current/oneiric-server-cloudimg-armel-disk1.img glance add -A KEYSTONE_TOKEN name=uec-oneiric-qcow2 is_public=true container_format=ovf \ disk_format=qcow2 --host=dev2.rcb.me oneiric-server-cloudimg-armel-disk1.img Well, specifically in this case, that is an ARM arch disk image. So unless you've got a nova-compute node running on arm I would not expect it to work ;). I've not used the 'glance add' path, but you can publish those disk images with cloud-utils via: $ wget http://uec-images.ubuntu.com/oneiric/current/oneiric-server-cloudimg-amd64-disk1.img $ cloud-publish-image --type image \ x86_64 oneiric-server-cloudimg-amd64-disk1.img my-bucket # note that releases prior to oneiric this command was 'uec-publish-image' Using the disk images is the preferred method for loading images to nova, as the disk image path (as opposed to the tarball path above) will boot using the grub2 loader embedded in the image, and subsequent 'apt-get dist-upgrade reboots' will get new kernels. If you *are* interested in using the partition image path (for older Ubuntu releases), then to get the kernel upgrade function, you'll have to use a loader kernel, which isn't well documented, but can be found at http://people.canonical.com/~smoser/lucid-loaders/ Scott ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] ImageTooLarge: Image is larger than instance type allows
On Sat, 24 Sep 2011, darkfower wrote: Hi, (nova.rpc): TRACE: raise exception.ImageTooLarge() (nova.rpc): TRACE: ImageTooLarge: Image is larger than instance type allows (nova.rpc): TRACE: image size biggest 20GB ? why ? You need to change the 'local_gb' for the instance_type if you want to accommodate a larger root disk. http://padl.lv/851301 has some more information. Use 'nova-manage' to modify the instance type with the 'instance_type' sub-command. I'm pretty sure you can't modify a type other than by deleting and re-adding. nova-manage instance_type create --flavor=0 --memory=512 --local_gb=2 --swap=512 --cpu=1 --name m1.xsmall ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Ubuntu run instance error + xen
On Fri, 26 Aug 2011, Leo van den Bulck wrote: Hi, Talking about cloud-init and Ubuntu instances... I have a problem related to that part which I can't really reproduce reliably. When spinning up an Ubuntu tty image, I'm often seeing this, during the cloud-init/cloud-setup phase: cloud-setup: cloudinit: getting ssh keys: [0=test] stty: /dev/console System is configured for NO sshd. Well, if you can help figure out what is going wrong, then I'll fix that in the images. Just looking through code, I can't see what would be doing it other than you passing a kernel command line option to your instances of 'nosshd' . But surely you'd know that. and then I'm unable to ssh to my instance: I'm getting connection refused, because there's no sshd runnng on port 22. The weird thing is, when I repeat the same Openstack/nova installation procedure a few days later, I would typically not get this issue. Then after another installation attempt yet a few days later the issue would crop up again. Does any recognize this error or know what it means? Maybe cloud-init/setup fails to fetch the keys and as a result won't start up sshd ? There is a bug that I should fix where the image tries to figure out if this is a slow system, and if so, it will not generate ssh keys. clearly you *always* want ssh keys generated if its your only access to the system even if it takes a long time. The only thing that MIGHT be a factor is that at one point I pulled sources from Launchpad/Bazaar and next time I used the Git mirror. I could try that again and see if that's the culprit. So then it would be down to the particular version of the source code or the build of the tty image, etc. I can't imagine that this would be an issue. Could you give a url to exactly what you're using as the ttylinux image? I've recently been working on this some, and I plan to start making those things with more sane version numbers than are present now. the recent snapshots i have at http://smoser.brickies.net/ubuntu/ttylinux-uec/dev/ support running under nova-lxc also. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Ubuntu run instance error + xen
On Fri, 19 Aug 2011, Joshua Harlow wrote: Begin: Running /scripts/init-bottom ... done. [1.348832] EXT4-fs (sda1): re-mounted. Opts: (null) cloud-init start-local running: Fri, 19 Aug 2011 21:34:59 +. up 4.01 seconds no instance data found in start-local init: cloud-init-local main process (186) terminated with status 1 What would be causing that? If anyone knows. Its hung waiting for networking to come up. cloud-init has two init scripts, 'cloud-init local' and 'cloud-init start' (bad name). The second runs only when the networking has come up. That is where you're blocked. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] nova on Debian
On Thu, 21 Jul 2011, Simon Guerrero wrote:
Hi folks
I've been trying to get an install of openStack working on Debian (sid, for
my sins). I figured
a good start point might be the Ubuntu Maverick guide.
I followed the guide, then the Installing Nova On A Single Host guide. I
had to make a few
changes, but it looked generally OK. However, I'm finding nova-compute won't
start because it
gets a Permission Denied error, namely:
(nova.exception): TRACE: if ret is None:raise
libvirtError('virConnectOpenAuth() failed')
(nova.exception): TRACE: libvirtError: unable to connect to
'/var/run/libvirt/libvirt-sock',
libvirtd may need to be started: Permission denied
(nova.exception): TRACE:
2011-07-21 15:00:48,808 CRITICAL nova [-] unable to connect to
'/var/run/libvirt/libvirt-sock',
libvirtd may need to be started: Permission denied
(nova): TRACE: Traceback (most recent call last):
(nova): TRACE: File /usr/bin/nova-compute, line 48, in module
(nova): TRACE: service.serve()
(nova): TRACE: File /usr/lib/pymodules/python2.6/nova/service.py, line
290, in serve
(nova): TRACE: x.start()
(nova): TRACE: File /usr/lib/pymodules/python2.6/nova/service.py, line
96, in start
(nova): TRACE: self.manager.update_available_resource(ctxt)
(nova): TRACE: File /usr/lib/pymodules/python2.6/nova/exception.py, line
126, in _wrap
(nova): TRACE: raise Error(str(e))
(nova): TRACE: Error: unable to connect to '/var/run/libvirt/libvirt-sock',
libvirtd may need to
be started: Permission denied
(nova): TRACE:
perms on libvirt-sock are 0700, owner root, group libvirt. nova is a member
of libvirt.
The command virsh -c qemu:///system version works just fine, as both root
and nova.
Any ideas what might be going wrong? The only similar log I can find on the
net is a chunk in
pastebin. :-)
Sorry I can't help here.
I also observed something interesting when creating a manifest for the kernel
using a sample
image:
euca-bundle-image -i images/aki-lucid/image -p kernel --kernel true
I received the error:
tar: s/aki-lucid: Cannot chdir: No such file or directory
Looks like it might be a problem with euca2ools. It seems to be taking the
path
images/aki-lucid/image and trying to remove the end of it, but doing a
rather greedy match on
image and ending up taking out the start too!
euca2ools is fairly old in debian (1.2.1). I would suggest grabbing the
ubuntu package. We fixed several issues in it (which are now all
upstream). Then, you might find that cloud-utils would also be
useful as you can do things like:
- euca-bundle-image -i images/aki-lucid/image -p kernel --kernel true
- euca-upload-bundle
- euca-register ...
+ uec-publish-image --type kernel amd64 images/aki-lucid/image my-bucket
(I'm aware that I should make 'uec-publish-image' not contain the string
'uec' and that i should work to cloud-utils into debian).
Finally, I always get the following warning with nova-manage:
/usr/lib/python2.6/dist-packages/paste/deploy/loadwsgi.py:8: UserWarning:
Module paste was
already imported from None, but /usr/lib/python2.6/dist-packages is being
added to sys.path
That is listed under ubuntu bug 810019.
https://bugs.launchpad.net/ubuntu/+source/paste/+bug/810019
and
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=473584___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Feedback on Portable Configuration Drive Blueprint
On Tue, 21 Jun 2011, Thorsten von Eicken wrote: Comments inline below. On 6/20/2011 1:40 PM, Scott Moser wrote: On Fri, 17 Jun 2011, Thorsten von Eicken wrote: We're very much looking forward to the new portable configuration drive functionality and would like to provide feedback. If this is not the best forum, please point me to it. The blueprint is: https://blueprints.launchpad.net/nova/+spec/configuration-drive We reviewed the initial work in: https://github.com/ChristopherMacGown/nova/commit/47c041a642ff32085b3140475d7a2a62dcb62c1a Feedback: This spec is very similar to implementing what is OVF Transport. So similar in fact that I think it should/could be made compatible. Read more at http://www.dmtf.org/standards/published_documents/DSP0243_1.1.0.pdf . Sounds like pretty much the same thing. I wouldn't make it identical in the naming of the file, though. Also, the ability to change the contents of the configuration drive needs to be defined carefully, for example, when stopping and restarting the VM after configuration changes. If you don't like the name, thats fine, thats why I suggested it be made configurable. Additionally, I want the user to bypass openstack completely, and give what looks like garbage to it, but the VM will understand. This is something Amazon did right in their user data. You can give whatever you want in it, and it can be obtained from inside the instance bit-for-bit. The only real restriction is a 16k limit (which I would hope is not duplicated here). I'm hoping that this config drive can be used in the same way. I'm not asking you to implement OVF ISO transport, I'm asking you to let the entity that launches the instance decide what exactly the contents are. 1. It is not always obvious how to enumerate attached devices and the document doesn't explicitly cover how we'd determine which attached device represents the configuration volume. Specifically consider Windows as well. I am not very knowledgeable about windows. However, I'd think as long as the volume doesn't end up being the boot volume (C:) or whatever, then you do not need to worry. The data on the disk should be formatted in such a way that it is identifiable. That can be done in a number of ways: * filesystem label (iso9660 and even i think FAT have a label) * contents of the filesystem The guest OS can be expected to go searching around for its configuration data. We do worry, because we do have to implement it, and we've been burned in the past with this stuff! I appreciate your number of ways but we'd much rather have one well=documented way that we can rely on. Our point here is that this should be thought out and specified in a non-ambiguous way. That's all. My suggestion really is to keep openstack providing dynamically provisioned hardware (and disk contents). As you've been burned in the past, so have others. Very simply, you can *not* come up with a solution that will will be easy for all current and future operating systems. By letting the entity launching the instance indicate what the contents of the disk are, your only restriction on future guests is that they can read the disk. 2. Will the configuration drive be available at boot time on a new instance? Or does it appear later (that would be bad)? It needs to be available immediately on first boot. It is very difficult to figure out that the guest is done with the volume, so it should remain attached until the user did some explicit action to detach it. I really like the suggestion in the etherpad that it is a normal volume, and then it can be detached via api call. I'll look into the etherpad, but detaching via an API call rom the instance is not something generally attractive because it requires having the account's credentials on the instance, which most security conscious users should avoid as much as possible. It does not need to be made from inside the instance. The entity that launched the instance can make a subsequent API call to detach the volume after determining that it has been consumed. It *can* be be done from inside the instance, and the need for temporary credentials that can be utilized from inside the instance (and even possibly passed in via configuration data) is something that openstack is going to have to address at some point in the future. Amazon does this fairly well with Identity Access Management. I'm fairly sure there was a spec on this discussed at the summit, but some quick googling did not find it. 3. How does one send the configuration drive content? What is the API call where we provide the configuration information and what is the expected format? In another message on this thread, Christopher said : a small ~64MB ISO formatted volume that would be attached and mounted read-only by the guest instance on boot I think we can greatly increase the functionality of this feature by removing
Re: [Openstack] Feedback on Portable Configuration Drive Blueprint
On Tue, 21 Jun 2011, Vishvananda Ishaya wrote: If you don't like the name, thats fine, thats why I suggested it be made configurable. Additionally, I want the user to bypass openstack completely, and give what looks like garbage to it, but the VM will understand. This is something Amazon did right in their user data. You can give whatever you want in it, and it can be obtained from inside the instance bit-for-bit. The only real restriction is a 16k limit (which I would hope is not duplicated here). I'm hoping that this config drive can be used in the same way. I'm not asking you to implement OVF ISO transport, I'm asking you to let the entity that launches the instance decide what exactly the contents are. The problem is that there are two types of data as exemplified by user data and metadata in the amazon world. Some data needs to be configuratble by the user, but some should be set by the cloud. Yes, and I admit to not having a real good idea on how to allow for 2 types of metadata (1 user, 1 system) in that image. The only thing I thought was possibly just attaching 2 volumes, its not a modern operating system has issues handling multiple volumes. You'd just label one meta-data and one would have whatever the user put on it. My suggestion really is to keep openstack providing dynamically provisioned hardware (and disk contents). As you've been burned in the past, so have others. Very simply, you can *not* come up with a solution that will will be easy for all current and future operating systems. By letting the entity launching the instance indicate what the contents of the disk are, your only restriction on future guests is that they can read the disk. I understand the push to go towards a really general solution, but if we go too general we sacrifice ease of use. I think we need a middle ground here. Allow for some data to be set by the user in whatever format, and allow some to be set by the cloud in a standard format. Personally, I think forcing a specific disk format like fat32 is not too extreme. If we provide r/w access and a binary blob for whatever user data I think we have found a good middle ground between flexibility and ease of use. Well, it still is limiting. Much in the same way that real users have have found AWS user-data to be limiting. There are possibly multiple entities involved in the launching of an instance. Ie, rightscale and end user, or landscape and end user or turnkey and end user. With the current AWS solution, neither one fully controls the content of http://instance-data/latest/user-data . They have to share it, meaning you're forced to do something like cloud-init does with multi-part mime to allow multiple parts into that single name space. If you you switch from rightscale to landscape, it is possible that the agreement on how to share that space (if any) changes, and so your code has to change. This could be addressed by the API allowing multiple key,value pairs that would end up appearing in the filesystem as filename and filecontent. In that way, then rightscale or landscape or turnkey just need to allow the end user to specify some keys, and they specify some keys, and as long as the 2 don't collide, the namespace can easily be shared. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Feedback on Portable Configuration Drive Blueprint
On Tue, 21 Jun 2011, Thorsten von Eicken wrote: I agree that there is a difference in difficulty of implementation between a quick fix and a more generic fix. I believe what I'm suggesting is a generic fix that will address longer term needs of openstack, and do so in a fairly clean way. I don't think a simple local loopback volume is a quick fix. There I really do think its a quick fix. You have these *really* nice building blocks that are being added to openstack, but you don't want to use them, because it will make things more difficult right now. Instead we'd be adding a 3rd type of disk (instance-store, ebs, config-drive -- sorry for using AWS terms I'm just more familiar with them, and I think others generally understand them). As the etherpad/thread originally raised, there are decisions that have to be made specifically for this new type of disk, like: * can it be detached? * what size can it be? * is it read/write ? Those questions are already (or will have to be solved) for the other types. If you use those other types, you inherit their behavior. Ignoring the need for meta-data (provided by the cloud itself), we can almost implement this as a normal user of the API with *no* further changes to openstack. (See my original message regarding doing this with a utility instance.) That almost says to me that openstack should not bother with this, but let it be sorted out at a layer above. Thats what brought me to suggest basically a mechanism for dropping the need for a utility instance. just is a huge difference between passing a few KB of config data in a secure way to an instance and adding a network disk volume at boot. I I would not insist that at all that this needs to be a volume is, in fact a network block storage. It might suffice to be an 'instance-store' type. I only suggested at one point that if it *were* a network block storage, then the semantics would be already defined, and existing behavior (with existing API entry points) could be used to modify the volume content on shutdown and startup. sure want the latter, but not confuse it with the former. When passing some keys/creds into an instance, I rather not have to deal with the security implications of a network volume (for example, can someone else snapshot the contents and use that to impersonate the instance or get at the creds?) I think that network block storage is next to useless if openstack cannot correctly manage that. I'd even go so far as to say openstack's momentum and reputation would be severely tarnished if a bug leaked data from one account to another. Surely Rackspace or any other openstack-user/cloud-provider would be more than a little interested in making sure that one user does not get access to another user's volumes. Simply, I really don't think you have to deal with that. It had better already work. Also, I don't know whether network attached volumes are a feature that all OpenStack implementations must implement. Given how many clouds don't have that, and the extra HW cost required to make it work in a reasonable manner, I wonder whether it's smart to require it, and if you don't you loose the configuration data feature. network block storage does not imply/require that it be high end iscsi devices. We have nova-volume that works right now with commodity grade hardware/software. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] euca-bundle-instance error
On Fri, 10 Jun 2011, Diego Parrilla Santamaría wrote:
Hi all,
we are testing the Cactus release of Openstack Nova and we tried to bundle a
Windows instance with 'euca-bundle-instance' as follows:
Client OS: Ubuntu Natty 64 bits
euca2ools and dependencies: what comes in Natty by default, main-31337
2009-04-04 (1.3.1, because 1.2 did not have euca-bundle-instance)
Windows image: 2008SR2 Datacenter Version 64 bit
The Nova deployment is based on 10.04 and Cactus (Stackops distro based).
When I try to execute:
root@i-113c:/home/ubuntu# euca-bundle-instance i-1136 -b bucket -p
acens -o ${EC2_ACCESS_KEY} -w ${EC2_SECRET_KEY}
I get this error:
EC2Connection instance has no attribute 'bundle_instance'
So I don't know if I'm doing something wrong or euca2ools does not support
instance bundling yet... Anybody can give me a hint? Is there another way to
bundle an image?
I believe the issue is that openstack does not support the
'BundleInstance' [1] api entry point [2].
--
[1]
http://docs.amazonwebservices.com/AWSEC2/latest/APIReference/index.html?ApiReference-query-BundleInstance.html
[2]
http://bazaar.launchpad.net/~hudson-openstack/nova/trunk/view/head:/nova/api/ec2/__init__.py___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] OS API server password generation
On Thu, 3 Mar 2011, George Reese wrote: Of all the boostrapping mechanisms I have encountered, the AWS model still remains the best. Specifically, with the guest OS pulling the keys from a trusted platform source. Any mechanism that requires an agent or requires any ability of the hypervisor or cloud platform to inject a password creates trust issues. In particular, the hypervisor and platform should avoid operations that reach into the guest. The guest should have the option of complete control over its data. I completely agree with this. Generally, I can think of only 2 real needs of openstack in this regard. a.) needs to be able to accept some blob of data in the launch-instance API call, and make that data available to the instance in a private channel (such as the xen store). b.) needs to be able to read data that the instance writes to that channel and pass that data back through to a user who invokes some API call. Amazon gets along fairly well with just 'a' (the metadata service and 'user-data'). Additionally, theres is 1 time write only (you can only specify user-data on instance start, and cannot change it... other than with an instance stop and start). You can get by by simple api that allows the entity that launches the instance to make data available to the instance, and stop there. If you take a list of key and values that can be put in, it would even be better. For example, ec2 looks like this: $ ec2-run-instance ami-abcdeg --user-data=my. data here The open stack could look like this: $ openstack-run-instance \ --data-user-data=my. data here --data-admin-data=some other data here The end result of which would be the metadata service having source/user-data : my. data here source/admin-data : some other data here In EC2 terms, what i'm really hoping for would be: http://169.254.169.254/latest/user-data http://169.254.169.254/latest/admin-data I think this is very powerful, and allows Rackspace to implement their initial password setting very easily via an installed guest agent. Ie, the guest comes up, the agent grabs http://169.254.169.254/latest/guest-agent-data that contains root_password : foo other-data here and more here it doesn't materr And then sets the root password. The above is very generic, easily re-purposed, and carries along the legacy Amazon user-data very easily. If you also make that data able to be updated via an API call, then you're even better off. That would allow the launcher to set the root password, verify it can get in to the instance, and then clear that data from the store so that it couldn't be seen again. Scott == small note == My primary interest in the key/value rather than just value in a single user-data (as Amazon has it) is that then only 1 party really has complete control of that user-data. For example, again, in the EC2 world, Rightscale wants to inject some data into the instance, and the user wants to inject some data into the instance. But both of these have to be shoved into that one 'user-data' field. So, you have to end up putting a multipart blob of data there and making both consumers aware that http://169.254.169.254/latest/user-data will not have exactly what they launched the instance with. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] OS API server password generation
On Thu, 3 Mar 2011, Ed Leafe wrote: On Mar 3, 2011, at 10:36 AM, George Reese wrote: It's a mistake to send OpenStack pursuing a flaw in Cloud Servers. Boy... Isn't this a rat hole that I've helped us go down. I am completely pro-agent (well, more agnostic). I would certainly hope that the api for openstack allows for communication to an in-instance agent, but that in-instance agent could be provided by the cloud-provider or the instance launcher. The reason I piped up, and the reason I think there is some misunderstanding is (from Ed): Again, you seem to have missed my point. OpenStack is *not* working on this. This is an OpenStack mailing list. The original mail says: | Subject: [Openstack] OS API server password generation | | We created a blueprint on adding support for password generation when | creating servers. This is needed | for Openstack API/Cloud Servers API v1.0 parity. | .. | https://blueprints.launchpad.net/nova/+spec/openstack-api-server-passwords | http://etherpad.openstack.org/openstack-api-server-passwords So, I'm sure you can understand how people thought this was about an openstack api specifically for setting server passwords. I am against that. I think it is important to have a mechanism in the openstack api to pass information to the guest out of band. that information may be read by an agent, it may be read by the android emulator running inside running angry birds. My point is that it should not be a.) agent specific b.) password specific The original mail surely seemed to indicate that what was being designed, on the openstack mailing list, was both a and b. If you read further on and go to the etherpad, it appears that the idea is to use the xen api to push data to the guest, and have an in-instance agent read it from there. This is all very fine to me, and actually mostly in line with what I suggested in my other mail. Scott ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] xen server agent code in nova?
On Thu, 10 Feb 2011, Thierry Carrez wrote: Paul Voccio wrote: So the question I want to pose to the community is if they are interested in the code, where should it go and how should we move forward on extending it? I think it's always interesting to publish the code. I'm not convinced of the value of shipping it inside Nova though. I agree with this. To me, the agent is a completely separate piece of code from nova. The agent would be able to interact via some communication channel with *something*. That something should not need to be openstack. I'm not aware of all that your guest agent does, later in this thread, configuring networking and setting password was mentioned. If you ignore configuring of initial networking, then its really not tied at all to the hypervisor. I could communicate with an agent that allows setting of root password via tcp/ip with that agent running in a guest kvm on my local machine. What I'm getting at, is that I think you should position this as a separate project. Define that openstack supports communication with a guest agent via a known protocol that can happen over a list of transports (xen store, vmware host-guest link...). Then, multiple guest agents that speak that protocol can arise. Overall, I think it both reduces the complexity of openstack and increases the ability for guest OS innovation by clearly defining that boundary. I'm not in the POC so it's not my job to define what should be considered Openstack core and what should not, but compatible guest agents IMO typically warrant their own project... ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
