Re: [Openstack] netns kernel of RedHat

[Pádraig Brady]

On 05/29/2013 11:02 AM, Pádraig Brady wrote:
> On 05/29/2013 09:59 AM, 彭勇 wrote:
>> there is no netns support in RedHat kernel:
>>
>> https://bugzilla.redhat.com/show_bug.cgi?id=869004
>>
>> is there any progress of this feature?
> 
> We expect to release an netns enabled kernel to
> RDO (http://openstack.redhat.com/) within the next couple of days

An updated kernel is now in place at the RDO repositories
referenced from the above.

thanks,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] netns kernel of RedHat

[Pádraig Brady]

On 05/29/2013 09:59 AM, 彭勇 wrote:
> there is no netns support in RedHat kernel:
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=869004
> 
> is there any progress of this feature?

We expect to release an netns enabled kernel to
RDO (http://openstack.redhat.com/) within the next couple of days

thanks,
Pádraig.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Installing OpenvSwitch on CentOS 6.4

[Pádraig Brady]

On 05/07/2013 05:53 AM, Ashutosh Narayan wrote:
> Hi Folks,
> 
> I have installed Grizzly on CentOS 6.4 and facing some issues while
> installing OpenvSwitch in order to make Quantum service work properly.
> Can anybody point me to some link which has useful information for the same ?
> I was following this link to set up Quantum -
> https://fedoraproject.org/wiki/Packstack_to_Quantum
> 
> And the below mentioned link to install OpenvSwitch -
> http://networkstatic.net/open-vswitch-red-hat-installation/
> I am not able to compile the source. It fails while running make.

That may be old now redundant info.
kernel >= 2.6.32-343 should facilitate OpenvSwitch.
Note also that user space rpms are available from the RDO repositories.
Please see http://openstack.redhat.com/Quickstart for setup details.

thanks,
Pádraig.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] ceilometer and heat tutorial

[Pádraig Brady]

On 04/28/2013 07:35 PM, Michaël Van de Borne wrote:
> Le 26/04/2013 20:28, Steven Hardy a écrit :
>> On Fri, Apr 26, 2013 at 05:02:54PM +0200, Michaël Van de Borne wrote:
>>> Hi all,
>>>
>>> I'm looking for install and usage tutorials for ceilometer and heat.
>>> The best I could find so far are these:
>>> - ceilometer: http://docs.openstack.org/developer/ceilometer/install.html
>>> - heat: 
>>> https://wiki.openstack.org/wiki/Heat/GettingStartedUsingMasterOnUbuntu
>>>
>>> Unfortunately, they seem to be intended for developers. Still have
>>> to git clone the softwares.
>>>
>>> I'm looking for tutorials like this one:
>>> http://docs.openstack.org/trunk/basic-install/content/
>>> (using apt-get, telling how to configure everything on every node,
>>> and finishing with a simple use case to validate the installation).
>>>
>>> Does anyone have something like this?
>> For Heat, the we don't yet have this sort of package-orientated
>> documentation, for Ubuntu this is because heat is not yet packaged for
>> Ubuntu, it is now in Debian Experimental though:
>>
>> https://launchpad.net/debian/experimental/+source/heat/2013.1-1
>>
>> Heat is packaged for Fedora, there are some instructions here:
>>
>> http://fedoraproject.org/wiki/Test_Day:2012-09-18_OpenStack

> thank you steve. Would you please keep posted as soon as a tutorial is 
> available. I'm looking forward to test Heat.
>
> Anybody has a good tutorial about ceilometer?

Note there was a newer Grizzly test day for Fedora which included
Ceilometer notes as well as the previously mentioned heat notes:

https://fedoraproject.org/wiki/Test_Day:2013-04-02_OpenStack#Heat_basic_functionality
https://fedoraproject.org/wiki/Test_Day:2013-04-02_OpenStack#Ceilometer_functionality

Note also that both heat and ceilometer are packaged for RHEL and derivatives,
which you can try quickly in a VM or on baremetal by following the simple 
process at
http://openstack.redhat.com/Quickstart and then following the Ceilometer 
instructions
at the URL above.

thanks,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Gerrit Review + SSH

[Pádraig Brady]

On 04/04/2013 06:51 PM, Ronak Shah wrote:
> Hi,
> 
> As OS dev cycle involves Gerrit review tool which requires ssh into the 
> gerrit server, I was wondering if any of you guys face problems where your 
> company/org does not allow ssh to external hosts.
> 
> In general, what is the best practice in terms of environment for generating 
> code review?
> 
> I appreciate the response in advance.

I'd just fire up a server in the cloud for a couple of minutes,
that has sshd listening on port 443, that could be used to
tunnel to gerrit port 22. Cost would be negligible.

Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [OpenStack] How to speed up the instance launch time?

[Pádraig Brady]

On 03/06/2013 04:12 PM, Mark Lehrer wrote:
> 
>>> The change I made that helped the most was to disable NBD.  You have
>>
>> Was that to disable injection at startup?
> 
> No, NBD is just slow and unreliable for some reason.  Disabling it made
> my users happy.  I haven't had time to find out why NBD is so horrible.
> 
> Eventually I would like to see /var/lib/glance/images and
> /var/lib/nova/instances/_base be on the same BTRFS file system and use
> reflink copies.

Good call on that.
`cp --reflink=auto` will do a reflink if possible or a normal copy otherwise.
There are two related changes for grizzly.

The first was with use_cow_images=True and force_raw_images=False,
there is actually no transformation done to the image in _base
and so one might be able to even `cp -l` in this case.
That's a bit of a layering violation though, but it might
be possible to handle within nova.

The second is support for direct copy from glance to nova:
https://github.com/openstack/nova/commit/20fca1a2
That isn't ideal at present as it uses shutil.copy()
but it's probably easy to update to calling `cp --reflink=auto`

thanks,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [OpenStack] How to speed up the instance launch time?

[Pádraig Brady]

On 03/06/2013 02:46 PM, Mark Lehrer wrote:
> 
>> are make in the compute node. So all this takes long an dis proportional to
>> the image size. Is there any way to speed up this process? Is a SAN based
>> backend the only way to go?
> 
> No, but if you have multiple compute nodes you will want to have them
> all share /var/lib/nova/instances on a fast NFS server.  Make sure
> that your glance machine is as fast as possible too.  I also mount
> /tmp from /dev/shm which makes the snapshot process more tolerable,
> but obviously this will require machines with lots of RAM to spare.
> 
> The change I made that helped the most was to disable NBD.  You have
> to do this in /etc/init/nova-compute.conf and just comment out the
> modprobe nbd line.

Was that to disable injection at startup?
That can be done through config in grizzly
by setting libvirt_injection_partition=-2

thanks,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Poor guest disk IO after disk usage.

[Pádraig Brady]

On 03/06/2013 02:56 PM, Samuel Winchenbach wrote:
> On Tue, Mar 5, 2013 at 12:11 PM, Pádraig Brady  <mailto:p...@draigbrady.com>> wrote:
> Just did some testing here, writing in a VM backed by a local file 
> system, using:
>   dd if=/dev/zero of=file bs=1M count=1k conv=notrunc,fdatasync 
> oflag=append
> 
> I didn't see a degredation after a while, but did see
> quite different performance depending on the formats used:
> 
> disk performance outside VM = 120MB/s
> raw in $instance_dir/ = 105MB/s
> qcow copy with preallocation=metadata in $instance_dir/ = 100MB/s
> qcow CoW with fallocate full size in $instance_dir/ = 55MB/s
>   Note perf a bit more stable than without fallocate
>   I didn't test with full host disk where improvements would be more 
> noticeable
> qcow CoW in $instance_dir/ = 52MB/s
> qcow CoW in $instance_dir/ backed by qcow with preallocation=metadata in 
> base = 52MB/s
> 
> Wow thanks for testing that.   I don't think I can preallocate on an NFS 
> mount.  I keep getting an "Operation not supported" error.

Right, fallocate isn't supported on NFS currently and won't make much perf 
difference anyway as noted above.
Using raw is probably your best bet currently.

thanks,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Poor guest disk IO after disk usage.

[Pádraig Brady]

On 03/06/2013 02:55 PM, Samuel Winchenbach wrote:
> N
> o harm will come to existing VMs if I set "use_cow_images=False" and restart 
> nova-api correct?  Just all new VMs will be created with a raw disk backend?

right.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] How to create a Image which can extend the partition automatically.

[Pádraig Brady]

On 03/05/2013 02:04 PM, Scott Moser wrote:
> On Tue, 5 Mar 2013, Sylvain Bauza wrote:
> 
>> If you look at
>> http://www.pixelbeat.org/docs/openstack_libvirt_images/#1361764412, you'll 
>> see
>> that resize2fs is performed. But there is a caveat with RHEL6 which is Linux
>> 2.6 (contrary to Ubuntu 12.04 which is Linux 3.0).
>> If you look at "man resize2fs" :
> 
> Please don't do this, or rely on this.  Having the hypervisor do this for
> your guest is simply wrong.  Hypervisors should know little to nothing
> about the internals of the instances they're launching.

Just to point out the alternative for when the VM doesn't
have the smarts to resize itself, is to use something like virt-resize:
https://blueprints.launchpad.net/nova/+spec/resize-no-raw

> Please see this thread:
> http://www.gossamer-threads.com/lists/openstack/dev/23572?do=post_view_threaded#23572
> 
> That describes how to do this the right way, and wont have the host OS
> accidentally break your filesystem or lose your data because it didn't
> realize you were running some filesystem it didn't have perfect support
> for.
> 
> Other points:
>  * I've just added the ability for growpart to run outside the
>initramfs and still function.  This same functionality is available in
>patched versions of parted also (resizepart), but not yet in upstream.
>https://code.launchpad.net/~psusi/ubuntu/raring/parted/resize/+merge/151401
>  * Juerg added support for cloud-initramfs-utils to run in fedora/rhel and
>drakut

Juerg has cloud-utils for RHEL available here:
http://kojipkgs.fedoraproject.org/packages/cloud-utils/0.27/0.2.bzr216.el6/noarch/
And cloud-initramfs-utils under dev here:
https://bugzilla.redhat.com/916087

>  * I'm adding support for cloud-init to do this rather than have it from
>cloud-initramfs-utils.  I suspect that in some point in the future, the
>requirements for "just works" will be:
> * cloud-init 0.7.2
> * (cloud-utils 0.27 | parted 3.2): after Philip Susi's work for
>   'resizepart' lands.
> * kernel > 3.8
>  * for what currently works:
> * cloud-initramfs-utils (0.20 or from trunk)
> * cloud-init > 0.7 (cloud-init invokes resize2fs)
> 
>You'll need cloud-utils 0.27 and sgdisk to get GPT resizing support.
>Dos partition format requires sfdisk (from util-linux).

thanks for the info,
Pádraig.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Poor guest disk IO after disk usage.

[Pádraig Brady]

On 03/04/2013 07:54 PM, Pádraig Brady wrote:
> On 03/04/2013 07:19 PM, Samuel Winchenbach wrote:
>> Hi All,
>>
>> I have a cluster of three nodes set up.  Live migration is accomplished via 
>> an NFS mount over 10GigE shared by all three nodes.
>>
>> I have noticed that when a VM is brand new I am getting close to 60MB/s disk 
>> write when using "dd if=/dev/zero of=file bs=1M count=1k conv=fdatasync". 
>> after doing this 10s of times the perfomance of the disk seems to drop  to 
>> around 4 - 12 MB/s.
>>
>> I have also noticed that doing a live-migration causes a similar effect 
>> immediately.
>>
>> Here is the virsh xml output of one of my VMs:  
>> https://gist.github.com/swinchen/397fbe3bb74305064944
>>
>> I have read several "tuning" guides and most of the suggestions seems to be 
>> configured already (cache='none', virtio for network, disk and memballoon).  
>>  
>>
>> Do you think qcow2 is causing my issue and if so is there a way to boot an 
>> instance and override the disk format?
> 
> qcow may possibly be the issue.
> 
> You could use raw disk images by setting the
> use_cow_images=False nova config option.
> The tradeoff there is slower instance start and
> increased (but more guaranteed) disk usage.
> 
> Alternatively you could try to fallocate the disk
> image before running the perf test like:
>   # get virt_size
>   qemu-img info /var/lib/nova/instances/instance-002f/disk
>   fallocate -n -l $virt_size /var/lib/nova/instances/instance-002f/disk
> 
> There is also the possibility of adjusting nova
> to use preallocation=metadata on the qcow images
> (forfeiting CoW in the process), as discussed in "future work" at:
> https://blueprints.launchpad.net/nova/+spec/preallocated-images

Just did some testing here, writing in a VM backed by a local file system, 
using:
  dd if=/dev/zero of=file bs=1M count=1k conv=notrunc,fdatasync oflag=append

I didn't see a degredation after a while, but did see
quite different performance depending on the formats used:

disk performance outside VM = 120MB/s
raw in $instance_dir/ = 105MB/s
qcow copy with preallocation=metadata in $instance_dir/ = 100MB/s
qcow CoW with fallocate full size in $instance_dir/ = 55MB/s
  Note perf a bit more stable than without fallocate
  I didn't test with full host disk where improvements would be more noticeable
qcow CoW in $instance_dir/ = 52MB/s
qcow CoW in $instance_dir/ backed by qcow with preallocation=metadata in base = 
52MB/s

thanks,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [OpenStack] How to speed up the instance launch time?

[Pádraig Brady]

On 03/05/2013 02:07 PM, Balamurugan V G wrote:
> Thanks for the responses and the link to pixelbeat. Its very helpful.
> 
> Start with QCOW, even before image is uploaded to Glance, dit help since the 
> image size camedown from 20Gb to 8+Gb. I even set the the 
> force_raw_images=False, and use_cow_images=True in nova.conf and restarted 
> nova services. I still see that the image is copied to _base and then copied 
> again to instances folder. This is probably expected and is the caching 
> feature. But under _base I see two files one with the underscore suffix and 
> one without. Does it mean that each time I launch an instance with a diferent 
> flavour(say disk set to 30Gb), another copy will be created in _base?
> 
> root@openstack-kvm1:/var/lib/nova/instances# ls -l _base/ | grep 
> 2b4a8e97317619077645de369a8f9fd63acd1f05
> -rw-rw-r-- 1 nova nova  8691056640 Mar  5 19:23 
> 2b4a8e97317619077645de369a8f9fd63acd1f05
> -rw-rw-r-- 1 libvirt-qemu kvm   8691056640 Mar  5 19:23 
> 2b4a8e97317619077645de369a8f9fd63acd1f05_20

These files in base are only update once per source image / size combination.
I.E. if you have 100 instances booted from a 20G flavor,
you'll only have those 2 files cached on the first boot.
Note there is a blueprint in place for precaching those too:
https://blueprints.launchpad.net/nova/+spec/nova-image-cache-management-2

thanks,
Pádraig.

p.s. those _20 files will not be there for grizzly
as the caching was adjusted there.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [OpenStack] How to speed up the instance launch time?

[Pádraig Brady]

On 03/05/2013 11:03 AM, Balamurugan V G wrote:
> Hi,
> 
> I have a Folsom 2012.2 3 nodes setup like the one specified at 
> https://github.com/mseknibilel/OpenStack-Folsom-Install-guide/blob/master/OpenStack_Folsom_Install_Guide_WebVersion.rst.
>  Each node has a 1Tb hard disk with no raid.
> 
> Be it the image creation or the Instance creation, takes a long time(factor 
> of the disk size of the image), about 15mins for a 20gb image. I can 
> understand the slow image creation due to the http transfer of the file from 
> a seperate web server hosting my images to the controller node. But even the 
> instance creation takes that much or even a bit longer. I see that the image 
> is copied from the Controller to Compute node and multiple copies are make in 
> the compute node. So all this takes long an dis proportional to the image 
> size. Is there any way to speed up this process? Is a SAN based backend the 
> only way to go?

It sounds like you're using raw images throughout?
You might consider using qcow2 images in glance.
Then you can avoid the conversion to raw in the libvirt base
directory by setting force_raw_images=False in nova.conf
That will avoid some of the initial caching penalty.

Ensuring that you have use_cow_images=True set,
with use CoW images for the instances and improve
instance startup latency.

Details of the operations and tradeoffs involved are at:
http://www.pixelbeat.org/docs/openstack_libvirt_images/

thanks,
Pádraig.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Poor guest disk IO after disk usage.

[Pádraig Brady]

On 03/04/2013 07:19 PM, Samuel Winchenbach wrote:
> Hi All,
> 
> I have a cluster of three nodes set up.  Live migration is accomplished via 
> an NFS mount over 10GigE shared by all three nodes.
> 
> I have noticed that when a VM is brand new I am getting close to 60MB/s disk 
> write when using "dd if=/dev/zero of=file bs=1M count=1k conv=fdatasync". 
> after doing this 10s of times the perfomance of the disk seems to drop  to 
> around 4 - 12 MB/s.
> 
> I have also noticed that doing a live-migration causes a similar effect 
> immediately.
> 
> Here is the virsh xml output of one of my VMs:  
> https://gist.github.com/swinchen/397fbe3bb74305064944
> 
> I have read several "tuning" guides and most of the suggestions seems to be 
> configured already (cache='none', virtio for network, disk and memballoon).   
> 
> Do you think qcow2 is causing my issue and if so is there a way to boot an 
> instance and override the disk format?

qcow may possibly be the issue.

You could use raw disk images by setting the
use_cow_images=False nova config option.
The tradeoff there is slower instance start and
increased (but more guaranteed) disk usage.

Alternatively you could try to fallocate the disk
image before running the perf test like:
  # get virt_size
  qemu-img info /var/lib/nova/instances/instance-002f/disk
  fallocate -n -l $virt_size /var/lib/nova/instances/instance-002f/disk

There is also the possibility of adjusting nova
to use preallocation=metadata on the qcow images
(forfeiting CoW in the process), as discussed in "future work" at:
https://blueprints.launchpad.net/nova/+spec/preallocated-images

thanks,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] openstack brctl addbr not supported on RHEL 6.3

[Pádraig Brady]

A quick search suggests that kernel has CONFIG_BRIDGE=n set.
So I presume there another method is possible through config.
Gary any ideas?

On 02/11/2013 04:33 PM, Greg Chavez wrote:


Running latest EPEL Folsom packages on RHEL 6.3.  Three nodes right now, one 
controller, one network node, one compute node.  The network node has three 
NICs, one for external net, one for management net, one for VM network traffic. 
 It has been a miserable journey so far.

The lastest calamity began with a failed spawn of the Cirros test image.  I 
booted it like this:

# nova --os-username demo --os-password demo --os-tenant-name demoProject boot 
--image aefa581f-47b0-4d46-8dbc-1a1f7f02dfa0 --flavor 2  --nic 
net-id=3de1e780-07d1-42af-89cc-0feaf1ece6e9 server-01

This succeeded but went directly into an ERROR state.  The compute node's 
/var/log/nova/compute.log showed this:

ProcessExecutionError: Unexpected error while running command.
Command: sudo nova-rootwrap /etc/nova/rootwrap.conf brctl addbr qbr2218b8c4-7d
Exit code: 1
Stdout: ''
Stderr: 'add bridge failed: Package not installed\n'

Hrm.  So then I ran this:

# brctl show
bridge namebridge idSTP enabledinterfaces
br-eth1/sys/class/net/br-eth1/bridge: No such file or directory
/sys/class/net/br-eth1/bridge: No such file or directory
/sys/class/net/br-eth1/bridge: No such file or directory
/sys/class/net/br-eth1/bridge: No such file or directory
/sys/class/net/br-eth1/bridge: No such file or directory
/sys/class/net/br-eth1/bridge: No such file or directory
/sys/class/net/br-eth1/bridge: No such file or directory
/sys/class/net/br-eth1/bridge: No such file or directory
/sys/class/net/br-eth1/bridge: No such file or directory
/sys/class/net/br-eth1/bridge: No such file or directory
/sys/class/net/br-eth1/bridge: No such file or directory
/sys/class/net/br-eth1/bridge: No such file or directory
.bc305befedd1no
br-int/sys/class/net/br-int/bridge: No such file or directory
/sys/class/net/br-int/bridge: No such file or directory
/sys/class/net/br-int/bridge: No such file or directory
/sys/class/net/br-int/bridge: No such file or directory
/sys/class/net/br-int/bridge: No such file or directory
/sys/class/net/br-int/bridge: No such file or directory
/sys/class/net/br-int/bridge: No such file or directory
/sys/class/net/br-int/bridge: No such file or directory
/sys/class/net/br-int/bridge: No such file or directory
/sys/class/net/br-int/bridge: No such file or directory
/sys/class/net/br-int/bridge: No such file or directory
/sys/class/net/br-int/bridge: No such file or directory
.7e1636f42c4bno

GAH!   What!!! First of all, bridge capability is set by default in the RHEL 
6.3 kernel.  Secondly, nova knows that it's supposed to be using openvswitch.  
The ProcessExecutionError's trace showed that the offending code came from 
/usr/lib/python2.6/site-packages/nova/virt/libvirt/vif.py line 216 which has 
this comment:

 def plug(self, instance, vif):
 """Plug using hybrid strategy

 Create a per-VIF linux bridge, then link that bridge to the OVS
 integration bridge via a veth device, setting up the other end
 of the veth device just like a normal OVS port.  Then boot the
 VIF on the linux bridge using standard libvirt mechanisms
 """

Thirdly, ovs-vsctrl is happy:

# ovs-vsctl show
44435595-8cc8-469c-ace4-ded76a7b864d
 Bridge "br-eth1"
 Port "br-eth1"
 Interface "br-eth1"
 type: internal
 Port "phy-br-eth1"
 Interface "phy-br-eth1"
 Port "eth1"
 Interface "eth1"
 Bridge br-int
 Port "int-br-eth1"
 Interface "int-br-eth1"
 Port br-int
 Interface br-int
 type: internal
 ovs_version: "1.7.3"

Final note, my network node fails the same way, but the controller does not.

I hope so much that somebody knows what is going on here.  This is very 
terrible for me as I am struggling to achieve minimal functionality.  Thanks.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] initramfs-growroot or LVM

[Pádraig Brady]

On 02/11/2013 10:46 AM, Davide Guerri wrote:

On 11/feb/2013, at 11:23, Pádraig Brady  wrote:


On 02/10/2013 03:54 AM, Owen Synge wrote:
This is where a system like libguestfs is very useful as it
separates image details from the host completely.
OpenStack file injection supports libguestfs automatically if available,
and this in turn supports cloud images with LVM.



Cool. Is this supported in Folsom?


It has bee supported since Essex.

thanks,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] initramfs-growroot or LVM

[Pádraig Brady]

On 02/10/2013 03:54 AM, Owen Synge wrote:

Dear Davide,

Please dont use LVM in cloud images unless you want to encrypt the
content and then please use a very unique volume group name. Reason follows.

If you want to allow the mangement domain to mount your partitons and
make edits then the management domain must first use something like
kpartx which allows you to present virtual disk partitons. These virtual
disk partitions can then be mounted if its a normal file system, but if
you used LVM, the partitions must be scanned by your system and added to
your systems volume group space, if these volume groups names clash with
volume groups being used on the management domain their can be problems
for the management domain to release the resources.


This is where a system like libguestfs is very useful as it
separates image details from the host completely.
OpenStack file injection supports libguestfs automatically if available,
and this in turn supports cloud images with LVM.

thanks,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] initramfs-growroot or LVM

[Pádraig Brady]

On 02/10/2013 02:38 AM, Robert Collins wrote:

On 10 February 2013 03:27, Davide Guerri  wrote:

Brady, thanks for the infos and for the bugzilla link.

I made some tests and some some other researches about potential performance 
penalties of LVM. These seem not to be noticeable especially with recent linux 
versions. (please see for instance [1]).

Now I'm even more curious about the choice of canonical: there must be a good reason for 
using such "rough" (pass me the term) filesystem resize technique over a more 
clean and practical lvm resize.


LVM is strictly more complexity: you still have to resize the block
device, and resize the filesystem metadata on top of it.


More complex but more flexible in when/how the resize of the device is done.
Whether that flexibility is needed is another question,
but I suppose supporting LVM would support more standard
cloud images where LVM is enabled by default.

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] initramfs-growroot or LVM

[Pádraig Brady]

On 02/08/2013 08:55 AM, Davide Guerri wrote:

Hi all,
I'm preparing some cloud images for the major Linux distributions and I'd like 
they to grow their root fs on boot (to use all the available space).

Ubuntu cloud images (http://cloud-images.ubuntu.com) use initramfs-growroot but 
installing it (and maintaining it across kernel upgrade) could be tricky -at 
least for me- on redhat derived like centos or fedora.


Note cloud-utils (including growroot) is currently in review for Red Hat 
flavored distros:
https://bugzilla.redhat.com/show_bug.cgi?id=907756


So my question is: what are pros and cons of using an ext3/4 root-fs and 
initramfs-growroot, or LVM (with a custom script that runs on first boot)?


LVM might be a bit heavy weight for this?

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Configuring instances during first boot

[Pádraig Brady]

On 01/29/2013 01:08 PM, JuanFra Rodriguez Cardoso wrote:

Thanks for your replies!!

Has anyone tried to resize root disk of Centos/Fedora instances with cloud-init?


Juerg Haefliger is working on cloud-utils and cloud-initramfs-tools
packages for Fedora/Red Hat, which support the growroot feature.
He might have some packages for you to test.

thanks,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Openstack - cinder - volume driver NFS

[Pádraig Brady]

On 01/15/2013 10:41 AM, Benoit ML wrote:

Hello,

(plz ignore my previous mail)

I'm sorry because I probabably misunderstand something.
I have :
- configured cinder like the file you show (thank you again)  (apport
the nfs_mount_point_base = /etc/cinder/volumes and volumes_dir =
/etc/cinder/volumes )
- mounted the nfs share in /etc/cinder/volumes
- echo '/etc/cinder/volumes' > /var/lib/cinder/nfsshare && chown
cinder /var/lib/cinder/nfsshare

And when I try to create a volume, it doesn't work. The volume is in
"error" state and in the log  (with debug/verbose activated) :
- cinder try to create a lv  : why create a lv ? not create a file on the nfs ?
- cinder try to "stat" a directory inside the nfs_share but failed
because it doesn't existe (of course does not create it before)




/usr/lib/python2.6/site-packages/cinder/utils.py:180
2013-01-15 11:32:08 WARNING cinder.volume.driver
[req-0ed3bc1e-d5a1-43c6-8b88-3401734695ca
108305fe03e24aa98626344b0f47e3a3 295b7cf015664e02ab54eb56eb95ee0c]
Exception during mounting Unexpected error while running command.
Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf stat
/etc/cinder/volumes/6614325979630346338
Exit code: 1
Stdout: ''
Stderr: "/usr/bin/stat: impossible d'\xc3\xa9valuer
\xc2\xab\xc2\xa0/etc/cinder/volumes/6614325979630346338\xc2\xa0\xc2\xbb:
Aucun fichier ou dossier de ce type\n"


I guessed that the following might fix your issue on the Fedora cloud list:

https://review.openstack.org/#/c/17761/
https://review.openstack.org/#/c/17762/

And with the extra info provided above, gives greater possibility
they will fix your issue. These should be backported to stable/folsom.

thanks,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Instances RedHat / Centos / Fedora vs /dev/vdx

[Pádraig Brady]

On 01/10/2013 12:18 PM, Alex Vitola wrote:

I'm having the following problem

Release Folsom

When I install any version of "Like" Redhat (Fedora all release,
CentOS 6 and 6.3), during the installation the anaconda not find hard
disk.

But, if i switch to the console and give it an "fdisk -l" the Disk
/dev/vda is there.

Inclusive i can create partitions and format in ext3.


This is some of the Anaconda Bug?

Installing Ubuntu in the same environment that does not happen.


Can you git more details of the steps/openstack commands you're using,
as I'm not sure what you're using exactly. What virt driver are
you using for example?

Note for preparing images for use in OpenStack, based on install media,
you can use a tool like Oz:
http://docs.openstack.org/trunk/openstack-compute/admin/content/creating-new-images.html

thanks,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] cloud-init and resizing centos image

[Pádraig Brady]

On 12/28/2012 10:05 AM, YIP Wai Peng wrote:

Dear all,

I am trying to build a CentOS image. I followed the instructions here:
at 
http://docs.openstack.org/essex/openstack-compute/starter/content/CentOS-de1592.html

I have also added EPEL cloud-init package to the image that I build.

I can boot the image fine, however, the root partition does not resize
upon boot.

I am reading around but the documentation is quite sketchy. All I know
that I might need cloud-initramfs-growroot or something, but there's
no docs on how to achieve that.

Can I know if there's anyone on the list that has successfully create
CentOS images and using cloud-init to expand the root partition?

- WP


Yes, nova currently can only grow raw, unpartitioned, ext[234] images.
As for cloud-initramfs-growroot for EPEL, it currently doesn't exist, as per:
http://lists.fedoraproject.org/pipermail/cloud/2012-December/002110.html
The workaround suggested in that thread was to
"run fdisk from initrd and expand the partition and fs."

thanks,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Compute not restarting, qemu-img error?

[Pádraig Brady]

On 12/18/2012 09:43 PM, Greg C wrote:

I suspended instances and rebooted one of my compute nodes, but not compute 
wont start up.



 From nova-compute.log:



2012-12-18 12:38:16 TRACE nova   File 
"/usr/lib/python2.7/dist-packages/nova/virt/images.py", line 56, in 
qemu_img_info
2012-12-18 12:38:16 TRACE nova if val[0] == " ":
2012-12-18 12:38:16 TRACE nova IndexError: string index out of range
2012-12-18 12:38:16 TRACE nova


Possibly https://review.openstack.org/#/c/17988/

thanks,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] python-swiftclient is missing from epel repo

[Pádraig Brady]

On 11/18/2012 09:01 PM, George Lekatsas wrote:

Hello,

after a yum update in centos 6.3 i have the following error

¨--> Processing Dependency: python-swiftclient for package: 
python-glance-2012.2-3.el6.noarch
--> Finished Dependency Resolution
Error: Package: python-glance-2012.2-3.el6.noarch (epel)
Requires: python-swiftclient
  You could try using --skip-broken to work around the problem
  You could try running: rpm -Va --nofiles --nodigest

It seems that python-swiftclient is missing from epel repo.


Note that's an update from Essex to Folsom.
If you want to stay on Essex please use:

http://repos.fedorapeople.org/repos/openstack/openstack-essex/epel-6/
http://repos.fedorapeople.org/repos/openstack/openstack-essex/README

If you do want to upgrade then please consider:

https://fedoraproject.org/wiki/Talk:Getting_started_with_OpenStack_EPEL

thanks,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] nova-volumes problem after host reboot

[Pádraig Brady]

On 11/10/2012 03:17 PM, Ronivon Costa wrote:

Hi there,

I am dealing with this issue for a while, but could not figure out what is 
going on.

After a reboot in the openstack server, I am not able to restart ANY instance 
that had a nova-volume attached.
I tried the DR procedure here without any improvement:
http://docs.openstack.org/trunk/openstack-compute/admin/content/nova-disaster-recovery-process.html

The error in compute.log is:
ERROR nova.compute.manager [req-adacca25-ede8-4c6d-be92-9e8bd8578469 
cb302c58bb4245cebc61e132c79c 768bd68a0ac149eb8e300665eb3d3950] [instance: 
3cd109e4-addf-4aa8-bf66-b69df6573cea] Cannot reboot instance: iSCSI device not 
found at 
/dev/disk/by-path/ip-10.100.200.120:3260-iscsi-iqn.2010-10.org.openstack:volume-20db45cc-c97f-4589-9c9f-ed283b0bc16e-lun-1

This is a very restrictive issue, because I can not simply attach volumes to 
instances knowing that in a power failure or reboot for maintenance I will have 
my instances unavailable.

Below there is some info about my setup.
Any idea? Anything!
:)




Linux nova-controller 2.6.32-279.11.1.el6.x86_64 #1 SMP Tue Oct 16 15:57:10 UTC 
2012 x86_64 x86_64 x86_64 GNU/Linux

rpm -qa |grep openstack
openstack-nova-api-2012.2-2.el6.noarch


If you follow:

https://fedoraproject.org/wiki/Getting_started_with_OpenStack_EPEL
https://fedoraproject.org/wiki/Test_Day:2012-09-18_OpenStack#Setup_OpenStack_volumes
https://fedoraproject.org/wiki/QA:Testcase_Create_Cinder_Volumes

You get the note:

On RHEL based systems the config files in /etc/tgt/conf.d/ don't currently 
honor globbing.
Only the main /etc/tgt/targets.conf seems does. So to avoid that issue causing 
tgtd to not start:

  sed -i '1iinclude /etc/nova/volumes/*' /etc/tgt/targets.conf
  sed -i '1iinclude /etc/cinder/volumes/*' /etc/tgt/targets.conf

then restart tgtd:

  service tgtd restart

Hopefully that will setup everything on boot.

thanks,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] glance add : 'utf8' codec can't decode byte 0x94

[Pádraig Brady]

On 11/15/2012 08:49 AM, G. Lekatsas wrote:

hello,

sorry for the delay but after a restart i have to deal with another problem and without 
being able to check the previous  "'utf8' codec" error.



   File "/usr/lib/python2.6/site-packages/keystone/catalog/backends/sql.py", 
line 169, in get_catalog
 catalog[region][srv_type]['internalURL'] = internal_url % d
TypeError: %d format: a number is required, not dict


You seem to have an invalid internalurl. I.E. one with %d... in it?
You can see what's configured by running the following as the admin user:

  keystone endpoint-list

Any % there should be just used to reference dict items, for example:

  %(tenant_id)s

thanks,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] glance add : 'utf8' codec can't decode byte 0x94

[Pádraig Brady]

On 11/13/2012 11:25 AM, G. Lekatsas wrote:

Hello,

i have installed openstack glance on a centos 6.3 server.

After running:
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=ovf 
copy_from=http://berrange.fedorapeople.org/images/2012-02-29/f16-x86_64-openstack-sda.qcow2

or
  glance  add name="tty-linux-kernel" disk_format=aki container_format=aki < 
ttylinux-uec-amd64-12.1_2.6.35-22_1-vmlinuz

i always have the following error:
Failed to add image. Got error:
'utf8' codec can't decode byte 0x94 in position 0: invalid start byte
Note: Your image metadata may still be in the registry, but the image's status 
will likely be 'killed'.

Thanks in advance.

George


I recognise 0x94 as ” in cp1252.
Perhaps you're copy and pasting the command from
some where with incorrect quoting?

Could you manually type the double quote chars in your command.

thanks,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] n-api installation problem with devstack (on centos)

[Pádraig Brady]

On 11/08/2012 04:12 PM, Mauch, Viktor (SCC) wrote:

Hi Guys,

Now I found the solution for my problem:

n-api installation/start (via devstack) failed on centos 6.x with the error
message below



The problem is the old python-paste version of centos.

So if you get this error, just remove python-paste with:

yum remove python-paste python-paste-deploy python-paste-script

and install the new one via pip:

pip install --upgrade paste PasteDeploy PasteScript

in the last step you have to remove the package names (paste, paste-deploy,
paste-script) in devstack (./files/rpms/*) , because stack.sh will install
this package with the next run.


I've not tried devstack on centos, but I do know of
specific updates to packages in EPEL to support OpenStack.

 python-paste-deploy1.5
 python-routes1.12
 python-sqlalchemy0.7
 python-webob1.0

Note the appended version number which differentiates
those parallel installable versions from the "official"
non backwards compatible version in Centos 6.
Anyway you may be able to install the above and
remove the "official" versions.

Or you can also proceed with your pip install method,
while also considering the other 3 packages I noted.

thanks,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Handling of adminPass is arguably broken (essex)

[Pádraig Brady]

On 11/02/2012 07:03 PM, Lars Kellogg-Stedman wrote:

On Thu, Nov 01, 2012 at 11:03:14AM -0700, Vishvananda Ishaya wrote:

The new config drive code defaults to iso-9660, so that should work. The
vfat version should probably create a partition table.


Is that what Folsom is using?  Or is it new-er than that?


That's in Folsom



___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] How to install Cloud Init on the Image

[Pádraig Brady]

On 10/30/2012 06:30 PM, AK Sathiya wrote:

All, I would like to install the cloud init on the custom image that i have 
created. I could not quite get the steps to install the cloud init. Is there 
any documentation anyone can point me to?


cloud-init is in EPEL, so you could enable the EPEL repo
and yum install cloud-init, or manually install the current RPM from:
http://dl.fedoraproject.org/pub/epel/6/x86_64/cloud-init-0.6.3-0.7.bzr532.el6.noarch.rpm

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Handling of adminPass is arguably broken (essex)

[Pádraig Brady]

On 11/01/2012 05:14 PM, Lars Kellogg-Stedman wrote:

On Wed, Oct 31, 2012 at 09:09:14PM -0400, Lars Kellogg-Stedman wrote:

TL;DR: The way OpenStack handles the adminPass attribute during
metadata injection is not useful on operating systems without an
/etc/passwd and /etc/shadow.  I would like to make the adminPass value
available on a Windows instance, and this is my proposal for how to do
it.


Oh geez, it gets worse.  The configuration disk created by OpenStack
is a whole-disk filesystem (no partition map), so Windows thinks it's
all unallocated space...so even with my patches in place I still can't
get at the data.

I can see I'm traveling through largely unexplored territory here.


It's somewhere on my TODO list to refactor the new
config disk stuff to reuse the mounting logic from
virt/disk/ and thus could support partitions if required.

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Error while launching instance RHEL (cannot run lease-init script nova-dhcpbridge )

[Pádraig Brady]

On 10/28/2012 06:00 PM, Daniel Vázquez wrote:


2012/10/27 Pádraig Brady :

On 10/26/2012 06:38 PM, Daniel Vázquez wrote:



2012/10/26 Pádraig Brady :


On 10/25/2012 06:30 PM, Pavan Kulkarni wrote:



Hi all,

 I am facing errors while launching instances on RHEL.
The network.log says *cannot run lease-init script
/usr/bin/nova-dhcpbridge*
I did a liitle search and found out this link
<https://lists.launchpad.net/openstack/msg08790.html>, followed the
instructions.
I have the flag in nova.conf set i.e *dhcpbridge =
/usr/bin/nova-dhcpbridge ( This file does exist)*

But I still get the same error while launching instances.
Any help is highly appreciated .Thanks




What version of RHEL?
What version of OpenStack?
What version of selinux-policy?

The error message (which you seem to have truncated) comes from
dnsmasq itself, and is mentioned here in relation to SELinux:
https://bugzilla.redhat.com/show_bug.cgi?id=734346
You might want to update your SELinux policies.




I'm experiencing same problems on Centos 6.3 (it's RHEL based). All
network system seem to stay ok, Openstack produces private IPs and nat
   public iPs to instances.
But instances don't discover DHCP.

Here:
Centos 6.3
Essex version
nova-networking flatDCHP
selinux permisive


I can't reproduce here.
Are you using the EPEL OpenStack packages?
On 6.3 based systems please ensure you install the dnsmasq-utils,
which installs utils that may be related to this.

If using the EPEL packages, I'd advise logging this
at http://bugzilla.redhat.com/ against the EPEL version
of openstack-nova, so that we don't miss anything.

>
> Using epel and dnsmasq-utils, here.

OK please see if there are corresponding messages in /var/log/audit/audit.log
as shown in https://bugzilla.redhat.com/show_bug.cgi?id=734346

If you turn on debugging you should see the `sudo ... dnsmasq`
command being run, and you can try that manually with various
settings to pinpoint your issue.
I know you said you've SELinux=permissive.
Maybe start with SElinux=disabled, and if that works,
you know it's a policy issue on your system.
If it doesn't work then see can you run the script
manually without dnsmasq and build up from there.

thanks,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Error while launching instance RHEL (cannot run lease-init script nova-dhcpbridge )

[Pádraig Brady]

On 10/26/2012 06:38 PM, Daniel Vázquez wrote:


2012/10/26 Pádraig Brady :

On 10/25/2012 06:30 PM, Pavan Kulkarni wrote:


Hi all,

I am facing errors while launching instances on RHEL.
The network.log says *cannot run lease-init script
/usr/bin/nova-dhcpbridge*
I did a liitle search and found out this link
<https://lists.launchpad.net/openstack/msg08790.html>, followed the
instructions.
I have the flag in nova.conf set i.e *dhcpbridge =
/usr/bin/nova-dhcpbridge ( This file does exist)*

But I still get the same error while launching instances.
Any help is highly appreciated .Thanks



What version of RHEL?
What version of OpenStack?
What version of selinux-policy?

The error message (which you seem to have truncated) comes from
dnsmasq itself, and is mentioned here in relation to SELinux:
https://bugzilla.redhat.com/show_bug.cgi?id=734346
You might want to update your SELinux policies.


> I'm experiencing same problems on Centos 6.3 (it's RHEL based). All
> network system seem to stay ok, Openstack produces private IPs and nat
>   public iPs to instances.
> But instances don't discover DHCP.
>
> Here:
> Centos 6.3
> Essex version
> nova-networking flatDCHP
> selinux permisive

I can't reproduce here.
Are you using the EPEL OpenStack packages?
On 6.3 based systems please ensure you install the dnsmasq-utils,
which installs utils that may be related to this.

If using the EPEL packages, I'd advise logging this
at http://bugzilla.redhat.com/ against the EPEL version
of openstack-nova, so that we don't miss anything.

thanks,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Error while launching instance RHEL (cannot run lease-init script nova-dhcpbridge )

[Pádraig Brady]

On 10/25/2012 06:30 PM, Pavan Kulkarni wrote:

Hi all,

   I am facing errors while launching instances on RHEL.
The network.log says *cannot run lease-init script /usr/bin/nova-dhcpbridge*
I did a liitle search and found out this link 
, followed the 
instructions.
I have the flag in nova.conf set i.e *dhcpbridge = /usr/bin/nova-dhcpbridge ( 
This file does exist)*
But I still get the same error while launching instances.
Any help is highly appreciated .Thanks


What version of RHEL?
What version of OpenStack?
What version of selinux-policy?

The error message (which you seem to have truncated) comes from
dnsmasq itself, and is mentioned here in relation to SELinux:
https://bugzilla.redhat.com/show_bug.cgi?id=734346
You might want to update your SELinux policies.

thanks,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Finding/Making Windows 7 Images for OpenStack

[Pádraig Brady]

On 10/21/2012 12:56 AM, Curtis C. wrote:

Hi,

Has anyone seen any recent documentation on creating Windows 7 images
for OpenStack? I know it's supposed to be as easy as using kvm to
install it initially (as per the OpenStack docs) then importing that
image into glance, but there are some subtle things that I might be
missing because I haven't really used Windows in a decade. I've
certainly done a lot of searching so if it's a link on the first ten
pages page of google I've probably seen it already. :)

Perhaps the best thing would be if anyone knew of a virtualbox/vagrant
or similar methodology for automatically creating Windows 7 images
that I could start from. Someone must be generating new Win 7 images
daily for their private cloud somewhere/somehow... :)

Any pointers much appreciated.


You could generate images for glance using OZ:
https://github.com/clalancette/oz/wiki

(I've not tried windows myself, but it's listed as being supported)

thanks,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Can't boot lxc instance

[Pádraig Brady]

On 10/22/2012 10:33 AM, hzguanqiang wrote:

Hi Pádraig
I think you are right. I met the same probelm just as 念远 did when I tried lxc 
with openstack folsom.
To solve the problem, I tried to modify the function setup_container, adding a 
partition argument to _DiskImage(...) just as follow:

  #img = _DiskImage(image=image, use_cow=use_cow, mount_dir=container_dir)
  img = _DiskImage(image=image, partition=FLAGS.libvirt_inject_partition, 
use_cow=use_cow, mount_dir=container_dir)

And it works!


Right. I'm not sure why LXC is treated differently in this regard,
but I'm looking at improving config in this area.
Note a global libvirt_inject_partition config option isn't
appropriate for disparate images, so it seems like we could benefit
from some image introspection, and/or image tagging in glance,
to indicate which partition to use.
Note if using libguestfs with libvirt_inject_partition=-1
we already support inspection of the image to determine
the best way to mount.


But I still have a little doubt about the lxc image.
It seems to me that many kvm images can not be used to create lxc while another 
little few can.
Why does this happen?


I can only guess that a single partition within the image
is not enough to use it. I'd need info about the images though
and specific errors to deduce further.

thanks,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Can't boot lxc instance

[Pádraig Brady]

On 10/19/2012 09:15 PM, 念远 wrote:

Hi,all!

In my ubuntu server (12.04.1), install openstack folsom, but i can't boot 
instance,nova-compute.log display blow log !

who can help me,thanks!



2012-10-20 08:58:45 TRACE nova.compute.manager [instance: 
d0a1c023-800c-4f1f-bb27-fb8279c6712e]   File "/usr/lib/python2.7/di
st-packages/nova/virt/disk/api.py", line 326, in setup_container
2012-10-20 08:58:45 TRACE nova.compute.manager [instance: 
d0a1c023-800c-4f1f-bb27-fb8279c6712e] raise exception.NovaExcep
tion(img.errors)
2012-10-20 08:58:45 TRACE nova.compute.manager [instance: 
d0a1c023-800c-4f1f-bb27-fb8279c6712e] NovaException:
2012-10-20 08:58:45 TRACE nova.compute.manager [instance: 
d0a1c023-800c-4f1f-bb27-fb8279c6712e] --
2012-10-20 08:58:45 TRACE nova.compute.manager [instance: 
d0a1c023-800c-4f1f-bb27-fb8279c6712e] Failed to mount filesystem: U
nexpected error while running command.


Are you using a partitioned image?
LXC doesn't currently support that.
You should be able copy out the partition of interest
if this is that case, and use that directly image.

thanks,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] CLA Contributor list is missing from the wiki?

[Pádraig Brady]

I tried to approve a few new CLA signers and noticed
that the http://wiki.openstack.org/Contributors page is missing.
Trying to restore a previous version resulted in an error.
Could someone with appropriate admin rights look into this?

thanks,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Terminating instance stuck at "deleting" - Openstack on Fedora Core 17

[Pádraig Brady]

On 09/26/2012 11:35 PM, Vishvananda Ishaya wrote:

FYI, I just tested on devstack and it looks like it works fine. If you are on 
older code you may be running into this bug:

https://bugs.launchpad.net/nova/+bug/1038266


To confirm.
The last Fedora 17 essex update just missed the above fix.
The next Fedora 17 update is planned for this week.

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] /var/lib/nova/instances over nfs/glusterfs issues

[Pádraig Brady]

On 09/08/2012 11:29 AM, Rich Wohlstadter wrote:

Hi everyone,

I'm running into an issue where I'm using nfs on my multi-node setup for
/var/lib/nova/instances so that I can use live migration. Things work
great unless I try to spawn multiple vm instances simultaneously and the
image does not yet exist in the _base directory. I will get multiple
spawning errors on a few of the instances that are trying to spawn,
usually with the following error: OSError: [Errno 2] No such file or
directory:
'/var/lib/nova/instances/_base/1cab2bf14012771017e66501a98bf594877c9d2d.part'.
It looks like multiple nodes are trying to do things on the same file and
1 node is deleting the file while the others are trying to access it thus
producing this error. Once the image and the resized image based on flavor
is out there and present in the _base directory then I no longer get
errors when spawning multiple instances at the same time. So basically it
seems to be an issue only the first time an image has to be copied over
from glance and resized. Anyone seen this or is this a known issue when
running multi-node instance directory over nfs and/or glusterfs?


There are two ways to support this.

https://review.openstack.org/#/q/Icff10ed75ba83f7256731614dc9e01e578b347a4,n,z
This older way was to have a separate base directory per compute node,
which you enable by doing this in nova.conf:
  base_dir_name=_base_$my_ip
It was suggested in that review (on master) that a shared lock dir could work
also, along with some potential issues with doing that.

https://review.openstack.org/#/q/Ic2ac87840904fa199c17774dae9556ad6c7a3eaf,n,z
only just implemented on master, implements the shared lock dir for a shared
instance directory implicitly. You can still use the config var in the
first method with this in place, (which might be required if you were running
into serialization issues between compute nodes).

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Nova] Getting error when injecting data into an image

[Pádraig Brady]

On 09/06/2012 01:45 PM, Nagaraju Bingi wrote:

Hi All,

I’m facing an issue with guest mount command and error as shown below. It would 
be appreciable if you could provide pointers.


Note guestmount is a method tried after nbd and loop.
I.E. only if they fail will it be attempted.

Looking at the error from "nbd" you can see:


2012-09-06 22:08:00 DEBUG nova.virt.disk.api [-] qemu-nbd error: sudo: unable 
to resolve host vmessex92


So it seems like you have an issue specific to sudo on your system.

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] openstack libvirt lxc

[Pádraig Brady]

On 08/21/2012 03:19 AM, 廖南海 wrote:
> 
> Who use the lxc virtual machine？
> Please give me some advices?
> I encountered some problems.
> Thank you!

Essex had some fundamental issues with LXC,
that are addressed with:
https://review.openstack.org/#/c/10962/

thanks,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Live Migration with libvirtError

[Pádraig Brady]

On 08/09/2012 07:55 AM, 王鹏 wrote:
> HI everyone,
> when I test live migration use NFS ,that's my sitting according to
> http://docs.openstack.org/essex/openstack-compute/admin/content/configuring-live-migrations.html
> 
> 1.add this line /var/lib/nova/instances *(rw,sync,fsid=0,no_root_squash) in 
> /etc/exports
> 2.mount -t nfs 172.18.32.7:/var/lib/nova/instances /var/lib/nova/instances at 
> compute
> and change some setting in libvirt.conf and libvird-bin.conf add a option -l
> 3.restart service
> 
> then I create instance in compute node ,find error
> that's my nova-compute.log as follow:

> 2012-08-09 13:13:03 TRACE nova.manager libvirtError: Failed to connect socket 
> to '/var/run/libvirt/libvirt-sock': No such file or directory

> what's /var/run/libvirt/libvirt-sock'?
> why this error?

That usually means libvirtd isn't running.
I'd double check the libvirt option settings you changed,
and then check in /var/log/libvirt/

For reference, here are the Fedora notes for configuring the feature:
https://fedoraproject.org/wiki/Getting_started_with_OpenStack_EPEL#Live_Migration_of_VM_instances

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [OSSA 2012-011] Compute node filesystem injection/corruption (CVE-2012-3447)

[Pádraig Brady]

On 08/08/2012 05:37 AM, Eric Windisch wrote:
> 
> Also notice that libguestfs is supported as an injection mechanism
> which mounts images in a separate VM, with one of the big advantages
> of that being better security.
> 
> 
> Are you sure about this? Reading the driver source, it appears to be using 
> 'guestmount' as glue between libguestfs and FUSE. Worse, this is done as 
> root.  This mounts the filesystem in userspace on the host, but the userspace 
> process runs as root.  Because the filesystem is mounted, all reads and 
> writes must also happen as root, leading to potential escalation scenarios.
> 
> It does seem that libguestfs could be used securely, but it isn't.

The image is handled in a separate VM.
guestmount sets up communication with this VM.

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [OSSA 2012-011] Compute node filesystem injection/corruption (CVE-2012-3447)

[Pádraig Brady]

On 08/08/2012 02:35 AM, Michael Still wrote:
> On 08/08/12 11:08, Pádraig Brady wrote:
> 
>> If supporting either of the above cases, it would be great to
>> reuse the existing image loopback mounting code:
>>
>> virt.disk.setup_container(image_file)
>> virt.disk.inject_file()
>> other tweaks
>> virt.disk.destroy_container(image_file)
> 
> This code doesn't seem to support _reading_ from the container though.
> The current process (if you specify a glance image is):
> 
> - fetch image from glance
> - mount it
> - inject the data into it
> - _copy_ the entire directory structure from the mounted image into the
> config disk image
> 
> Its that final step that I think is hard with the containers code,
> unless I am missing something.


> What's the security vulnerability here? Its writing to something which
> might be a symlink to somewhere special, right?

That's one vector.
Even mounting the image is a potential vector.
Anyway these issues should be kept within virt.disk.api
(which can use libguestfs as it is).

> Would it be better for example to mount the image from glance, copy its
> contents to the config disk image (skipping symlinks), and then umount
> it? The data could then be written to the config disk instead of to the
> image from glance. That would mean if there was a symlink pointing
> somewhere special in the glance image it couldn't be exploited.

That would help, but as mentioned above, the loop mount itself
can be dangerous. So just using the disk.setup_container()
as mentioned above, will help, and at least avoid reimplementing
loop back mounting code.

Keeping symlinks could be a useful feature BTW.
Perhaps {cp,tar,rsync} --one-file-system could be
leveraged to merge trees in a more secure way.

cheers,
Pǽdraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [OSSA 2012-011] Compute node filesystem injection/corruption (CVE-2012-3447)

[Pádraig Brady]

On 08/08/2012 01:00 AM, Michael Still wrote:
> On 08/08/12 07:38, Eric Windisch wrote:
>>
>>> Pádraig Brady from Red Hat discovered that the fix implemented for 
>>> CVE-2012-3361 (OSSA-2012-008) was not covering all attack
>>> scenarios. By crafting a malicious image with root-readable-only
>>> symlinks and requesting a server based on it, an authenticated user
>>> could still corrupt arbitrary files (all setups affected) or inject
>>> arbitrary files (Essex and later setups with OpenStack API enabled
>>> and a libvirt-based hypervisor) on the host filesystem, potentially
>>> resulting in full compromise of that compute node.
>>>
>>
>> Unfortunately, this won't be the end of vulnerabilities coming from
>> this "feature".
>>
>> Even if all the edge-cases around safely writing files are handled
>> (and I'm not sure they are), simply mounting a filesystem is a very
>> dangerous operation for the host.
>>
>> The idea had been suggested early-on to supporting ISO9660
>> filesystems created with mkisofs, which can be created in userspace,
>> are read-only, and fairly safe to produce, even as root on compute
>> host.
> 
> I am in the process of re-writing the config drive code as we speak. The
> re-write supports (and defaults to) providing the config drive as an
> iso9660 image.
> 
> There are two places that mounting occurs with the new code:
> 
>  - if the user wants a vfat config drive, as I couldn't find a way to
> create a vfat filesystem from a directory using userspace code. This
> should be "relatively safe" though because the filesystem which is
> mounted is created by the code just before the mount. [1]

That should be very safe.
Note you can change vfat file systems from user space like:

$ truncate -s10M fat.img
$ mkfs.vfat fat.img
$ mmd -i fat.img /etc
$ mtools # Gives a command list

$ mdir -i fat.img
 Volume in drive : has no label
 Volume Serial Number is 8994-9F2E
Directory for ::/

etc   2012-08-08   1:59  etc
1 file0 bytes
 10 444 800 bytes free

I wouldn't jump through those hoops though,
if we're creating the fat image ourselves.

> 
>  - if the user specifies an image from glance for the injection to occur
> to. This is almost certainly functionality that you're not going to like
> for the reasons stated above. Its there because v1 did it, and I'm
> willing to remove it if there is a consensus that's the right thing to
> do. However, file IO on this image mount is done as the nova user, not
> root, so that's a tiny bit safer (I hope).

If supporting either of the above cases, it would be great to
reuse the existing image loopback mounting code:

virt.disk.setup_container(image_file)
virt.disk.inject_file()
other tweaks
virt.disk.destroy_container(image_file)

> https://review.openstack.org/#/c/10934/

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [OSSA 2012-011] Compute node filesystem injection/corruption (CVE-2012-3447)

[Pádraig Brady]

On 08/07/2012 10:38 PM, Eric Windisch wrote:
> 
>> Pádraig Brady from Red Hat discovered that the fix implemented for
>> CVE-2012-3361 (OSSA-2012-008) was not covering all attack scenarios. By
>> crafting a malicious image with root-readable-only symlinks and
>> requesting a server based on it, an authenticated user could still
>> corrupt arbitrary files (all setups affected) or inject arbitrary files
>> (Essex and later setups with OpenStack API enabled and a libvirt-based
>> hypervisor) on the host filesystem, potentially resulting in full
>> compromise of that compute node.
>>  
> 
> Unfortunately, this won't be the end of vulnerabilities coming from this 
> "feature".
> 
> Even if all the edge-cases around safely writing files are handled (and I'm 
> not sure they are), simply mounting a filesystem is a very dangerous 
> operation for the host.
> 
> The idea had been suggested early-on to supporting ISO9660 filesystems 
> created with mkisofs, which can be created in userspace, are read-only, and 
> fairly safe to produce, even as root on compute host.
> 
> That idea was apparently shot-down because, "the people who 
> documented/requested the blueprint requested a read-write filesystem that you 
> cannot obtain with ISO9660".  Now, everyone has to live with a serious 
> technical blunder.
> 
> Per the summit discussion Etherpad:
>  "injecting files into a guest is a very popular desire."
> 
> Popular desires not necessary smart desires. We should remove all file 
> injection post-haste.

You can configure injection out depending on your requirements.
Also notice that libguestfs is supported as an injection mechanism
which mounts images in a separate VM, with one of the big advantages
of that being better security.

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] Heads-up: Minimum version dependency added on python-evenlet (0.9.17)

[Pádraig Brady]

https://review.openstack.org/#/c/10823/

0.9.17 was released 2012-08-03 and
includes two particular fixes related to OpenStack

1. https://bitbucket.org/which_linden/eventlet/issue/123/
Fix an exception thrown by epoll in certain cases.
This can cause jenkins to deadlock, triggered for example by:
https://review.openstack.org/#/c/10767/

2. https://bitbucket.org/which_linden/eventlet/issue/115/
https://bugs.launchpad.net/nova/+bug/903199
Fix a significant memory leak of _DummyThread objects.

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] qpid_heartbeat...doesn't?

[Pádraig Brady]

On 08/02/2012 05:35 PM, Lars Kellogg-Stedman wrote:
> On Thu, Aug 02, 2012 at 12:33:13PM -0400, Lars Kellogg-Stedman wrote:
>>> Looks like a typo.
>>> Could you try this.
>>
>> FYI: The same typo appears to exist in notify_qpid.py.
> 
> Err, that is, glance/notifier/notify_qpid.py, in case it wasn't
> obvious...

Well spotted.
I've submitted a patch for:
https://bugs.launchpad.net/glance/+bug/1032314

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Node Disk Cleaning Script

[Pádraig Brady]

On 08/02/2012 12:12 PM, Алексей Кайтаз wrote:
> Hi!
> I hope this script will usefull for somebody.
> 
> #!/bin/bash
> cd /var/lib/nova/instances
> find -name "disk*" | xargs -n1 qemu-img info | grep backing | sed 
> -e's/.*file: //' -e 's/ .*//' | sort | uniq > /tmp/ignore
> while read i; do
> ARGS="$ARGS  \( ! -path $i  \) "
> done < /tmp/ignore
> find /var/lib/nova/instances/_base/ -type f  $ARGS  -delete

This is done automatically by nova when you enable this in /etc/nova/nova.conf

  remove_unused_base_images = True

That is done in Fedora/EPEL packages for the last while,
and will default on in the next folsom release.

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] qpid_heartbeat...doesn't?

[Pádraig Brady]

On 07/29/2012 07:14 PM, Lars Kellogg-Stedman wrote:
>> Looks like a typo.
>> Could you try this.
> 
> That seems better...although while the documentation says that
> qpid_heartbeat is "Seconds between heartbeat messages" [1], observed
> behavior suggests that it is actually *minutes* between messages.
> 
> [1]: 
> http://docs.openstack.org/essex/openstack-compute/admin/content/configuration-qpid.html

That's surprising as the qpid code does:

if self.connection.heartbeat:
  times.append(time.time() + self.connection.heartbeat)

Notice how time.time() is seconds, and so
heartbeat must be given in seconds.
Perhaps there is another issue with the scheduling of this?
How are you monitoring the connection?

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] qpid_heartbeat...doesn't?

[Pádraig Brady]

On 07/29/2012 03:24 AM, Lars Kellogg-Stedman wrote:
> Our environment has connection-tracking firewalls that drop idle
> connections after an hour.  There is a connection between nova-compute
> and our qpidd server that appears to be idle for long periods of time.
> 
> When the firewall drops this connection, the participating hosts are
> unaware of that fact and ultimately stop communicating with each other
> until we restart nova-compute.
> 
> I was hoping that the qpid_heartbeat parameter would avoid this
> problem by keeping the connection active, but despite having
> qpid_heartbeat set explicitly in our configuration...
> 
>   # This is supposed to be the default
>   qpid_heartbeat = 5
> 
> ...there is no traffic across this connection
> 
> I can deal with this problem by forcing (via libkeepalive,
> http://libkeepalive.sourceforge.net) SO_KEEPALIVE on the AMQ sockets
> (and tuning the net.ipv4.tcp_keepalive_time sysctl to be < the
> firewall connection timeout), but that seems a bit of a hack.  It's
> also possible to work around this by disabling idle connection
> timeouts on the firewall, so we're not completely stymied...
> 
> ...but I would like to understand why setting qpid_heartbeat does not,
> in fact, result in the regular transmission of heartbeat packets
> across the connection.
> 
> We're running openstack-nova-2012.1.1-0.20120615.13614 from EPEL (and
> qpid 0.14).

Looks like a typo.
Could you try this.

cheers,
Pádraig.

diff --git a/nova/rpc/impl_qpid.py b/nova/rpc/impl_qpid.py
index 289f21b..e19079e 100644
--- a/nova/rpc/impl_qpid.py
+++ b/nova/rpc/impl_qpid.py
@@ -317,7 +317,7 @@ class Connection(object):
 FLAGS.qpid_reconnect_interval_min)
 if FLAGS.qpid_reconnect_interval:
 self.connection.reconnect_interval = FLAGS.qpid_reconnect_interval
-self.connection.hearbeat = FLAGS.qpid_heartbeat
+self.connection.heartbeat = FLAGS.qpid_heartbeat
 self.connection.protocol = FLAGS.qpid_protocol
 self.connection.tcp_nodelay = FLAGS.qpid_tcp_nodelay

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] 回复： [openstack] nova-compute always dead on RHEL6.1

[Pádraig Brady]

On 07/24/2012 02:05 AM, 延生 付 wrote:
> Dear Padraig,
>  
> Thanks for the help. I can see the log from console, really strange there is 
> no log generated under /var/log/nova, the permission is open to nova.

Ensure there are no root owned files under /var/log/nova

> Another question is Apache Qpid can not be connected, even from the same 
> server. I setup from default, no further config.
> I can see the port 5672 is listened, and I also turned iptables off. Is there 
> any Qpid log I can refer?

Try setting auth=no is set in /etc/qpidd.conf

Note these instructions have been used successfully by many:
https://fedoraproject.org/wiki/Getting_started_with_OpenStack_EPEL

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [openstack] nova-compute always dead on RHEL6.1

[Pádraig Brady]

On 07/23/2012 09:44 AM, 延生 付 wrote:
>  
> Dear all,
>  
> When I deply nova-compute based on epel repository, I found 
> openstack-nova-compute always dead but pid file exists.
> While there is no any log file generated in /var/log/nova.
> The OS is RHEL6.1. The nova.conf is copied from controller node.
>  
> [root@comp02-r11 nova]# service openstack-nova-compute status
> openstack-nova-compute dead but pid file exists
>  
> Does anybody have clues? Thanks in advance.

RHEL6.2 is the first version targeted by the EPEL packages,
though others have successfully used 6.1.
One thing to consider is upgrading libvirt.
Strange you don't get anything in the logs.
Perhaps you could run manually to debug:

/usr/bin/nova-compute --config-file /etc/nova/nova.conf

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] File injection support

[Pádraig Brady]

On 07/14/2012 12:09 AM, Mark Moseley wrote:
> On Tue, Jun 12, 2012 at 8:20 AM, Pádraig Brady  wrote:
>> On 06/12/2012 04:07 PM, Fredric Morenius wrote:
>>> From: Scott Moser [mailto:ssmos...@gmail.com] On Behalf Of Scott Moser
>>> Sent: den 11 juni 2012 23:16
>>>
>>>> ...Without digging around on older versions of OS's and their included 
>>>> kernel/udev/kpartx i'd have to say that from your message above, and 
>>>> my testing that running 'kpartx' is no longer necessary.  That can 
>>>> reasonably be expected to be created by >the kernel or some other plumbing 
>>>> bits automatically...
>>>
>>> I just now tested to remove the kpartx invocation from 
>>> /nova/virt/disk/mount.py, like this:
>>>
>>> diff --git a/nova/virt/disk/mount.py b/nova/virt/disk/mount.py
>>> index 11959b2..4d9527b 100644
>>> --- a/nova/virt/disk/mount.py
>>> +++ b/nova/virt/disk/mount.py
>>> @@ -61,25 +61,9 @@ class Mount(object):
>>>  if self.partition == -1:
>>>  self.error = _('partition search unsupported with %s') % 
>>> self.mode
>>>  elif self.partition:
>>> -map_path = '/dev/mapper/%sp%s' % 
>>> (os.path.basename(self.device),
>>> -  self.partition)
>>> -assert(not os.path.exists(map_path))
>>> -
>>> -# Note kpartx can output warnings to stderr and succeed
>>> -# Also it can output failures to stderr and "succeed"
>>> -# So we just go on the existence of the mapped device
>>> -_out, err = utils.trycmd('kpartx', '-a', self.device,
>>> - run_as_root=True, 
>>> discard_warnings=True)
>>> -
>>> -# Note kpartx does nothing when presented with a raw image,
>>> -# so given we only use it when we expect a partitioned image, 
>>> fail
>>> -if not os.path.exists(map_path):
>>> -if not err:
>>> -err = _('partition %s not found') % self.partition
>>> -self.error = _('Failed to map partitions: %s') % err
>>> -else:
>>> -self.mapped_device = map_path
>>> -self.mapped = True
>>> +#qemu-nbd already has mapped the partition for mounting
>>> +self.mapped_device = '%sp%s' % (self.device, self.partition)
>>> +self.mapped = True
>>
>> Looks good. That should also support raw through the /dev/loop%s devices.
>>
>> It might be safer to fallback to kpartx if not exists ...
>>   '%sp%s' % (self.device, self.partition)
>> That would support kernels before 3.2
>> I'd also add a comment in the code, that nbd must be mounted with param
>> max_part=16 or whatever.
> 
> 
> Might be totally unrelated to the above issues but I figured I'd throw
> it out there. I was messing with file injection today and was getting
> odd errors.
> 
> Some were due to hitting errors on a previous boot and partitions not
> getting cleaned up (e.g. kpartx -d never ran and the /dev/mapper/nbd*
> entries were still there, or qemu-nbd was still running). Some of
> those were likely a result of my own circumstances.
> 
> More interesting though, and what might be of use to other people, the
> "kpartx -a" calls get run and then the code in nova/virt/disk/mount.py
> immediately checks for whether or not the newly created
> /dev/mapper/nbdXXpX partitions exist. They'd actually get created but
> the os.exists call would fail. Apparently the os.exists call was
> getting run too soon. I added a time.sleep() after both the 'kpartx
> -a' and 'kpartx -d' calls, to give things time to catch up before the
> os.exists calls and things worked much better.

Sigh.

The amount of synchronization bugs I've noticed in lower
Linux layers lately is worrying.

Anyway I've created a bug for this:
https://bugs.launchpad.net/nova/+bug/1024586

cheers,
Pádraig

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] running HA cluster of guests within openstack

[Pádraig Brady]

On 04/13/2012 12:53 PM, Pádraig Brady wrote:
> On 04/13/2012 10:31 AM, ikke wrote:
>> I likely am not the first one to ask this, but since I didn't find a
>> thread about it I start one.
>>
>> Is there any shared experience available what are the capabilities of
>> OpenStack to run cluster of guests in the cloud? Do you have
>> experience of the following questions, or links to more info? The
>> questions relate to running a legacy HA cluster in virtual env, and
>> moving it into cloud...
> 
> I'll just point out two early stage projects
> that used in combination can provide a HA solution.
> 
> http://wiki.openstack.org/Heat
> http://wiki.openstack.org/ResourceMonitorAlertsandNotifications
> 
> These are similar to AWS CloudFormations and CloudWatch respectively.

I notice Heat V4 has just been released.

Here is some additional info on High Availability:
https://github.com/heat-api/heat/wiki/Roadmap-Feature:-High-Availability

and some notes on using it in its current form:
https://github.com/heat-api/heat/wiki/Using-HA

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Nova doesn't release ips when terminating instances

[Pádraig Brady]

On 06/26/2012 05:55 PM, Lars Kellogg-Stedman wrote:
>> force_dhcp_release=true should cause the ip to be released immediately,
>> assuming the relevant optional binary from dnsmasq is installed (it is in
>> the package dnsmasq-utils in ubuntu).
> 
> The dhcp_release command does not appear to be packaged with Fedora.

It's in the dnsmasq-utils package on Fedora 16 and 17
The Fedora openstack packages auto install this package
and enable the option to use it.

Oh I see you're using CentOS 6.2.
A rebuild of this would probably work:
http://ftp.redhat.com/pub/redhat/linux/enterprise/6Workstation/en/os/SRPMS/dnsmasq-2.48-6.el6.src.rpm

>> If it is set to false then the ips
>> should be reclaimed after a set timeout period (ten minutes by default) via
>> a periodic task in the network worker. If they are not being reclaimed
>> properly then there is definitely a bug somewhere...
> 
> It does not appear that ips are ever properly reclaimed.  They will
> hang around with allocated=0 and instance_id != NULL forever, until I
> manually correct the database.
> 


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [bug] cannot umount guestfs (Bug #1013689)

[Pádraig Brady]

On 06/21/2012 08:40 AM, Alexey Ababilov wrote:
> Hi!
> 
> On our machines ins Grid Dynamics (HP ProBooks and miscellaneous servers), 
> repeating unmount if it reported "device or resource is busy" while spawning 
> a VM is not acceptable. Very often, unmount is reported as _successful_ 
> during the _first_ attempt, but the filesystem remains _unsynchronized_ and 
> authorized_keys are not updated, so, the VM is not accessible by ssh. That 
> looks strange, but it really happens.
> 
> Could you share your experience in reproducing and fixing this bug, please?
> 
> The same problem will be with sleeping for 4 seconds - the filesystem remains 
> not synchronized.
> 
> Why are you against sync? It's innocent and it constantly fixes the bug.

Well we've just received the info above.
In any case syncing the whole system is not entirely innocent.
Though avoiding is awkward at present:
http://thread.gmane.org/gmane.linux.kernel.cross-arch/9383

I notice in the guestmount man page that it tries to sync before umount by 
default,
and that must be disabled with the guestmount --no-sync option.
I.E. it's doubly surprising that this is an issue.

Alexey, could you give the version of:
libguestfs-mount libguestfs fuse kernel

Richard what sync method is used.
I didn't see the definition of guestfs_internal_autosync()
in 10s grepping the source.

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] File injection support

[Pádraig Brady]

On 06/12/2012 04:07 PM, Fredric Morenius wrote:
> From: Scott Moser [mailto:ssmos...@gmail.com] On Behalf Of Scott Moser
> Sent: den 11 juni 2012 23:16
> 
>> ...Without digging around on older versions of OS's and their included 
>> kernel/udev/kpartx i'd have to say that from your message above, and my 
>> testing that running 'kpartx' is no longer necessary.  That can reasonably 
>> be expected to be created by >the kernel or some other plumbing bits 
>> automatically...
> 
> I just now tested to remove the kpartx invocation from 
> /nova/virt/disk/mount.py, like this:
> 
> diff --git a/nova/virt/disk/mount.py b/nova/virt/disk/mount.py
> index 11959b2..4d9527b 100644
> --- a/nova/virt/disk/mount.py
> +++ b/nova/virt/disk/mount.py
> @@ -61,25 +61,9 @@ class Mount(object):
>  if self.partition == -1:
>  self.error = _('partition search unsupported with %s') % 
> self.mode
>  elif self.partition:
> -map_path = '/dev/mapper/%sp%s' % (os.path.basename(self.device),
> -  self.partition)
> -assert(not os.path.exists(map_path))
> -
> -# Note kpartx can output warnings to stderr and succeed
> -# Also it can output failures to stderr and "succeed"
> -# So we just go on the existence of the mapped device
> -_out, err = utils.trycmd('kpartx', '-a', self.device,
> - run_as_root=True, discard_warnings=True)
> -
> -# Note kpartx does nothing when presented with a raw image,
> -# so given we only use it when we expect a partitioned image, 
> fail
> -if not os.path.exists(map_path):
> -if not err:
> -err = _('partition %s not found') % self.partition
> -self.error = _('Failed to map partitions: %s') % err
> -else:
> -self.mapped_device = map_path
> -self.mapped = True
> +#qemu-nbd already has mapped the partition for mounting
> +self.mapped_device = '%sp%s' % (self.device, self.partition)
> +self.mapped = True

Looks good. That should also support raw through the /dev/loop%s devices.

It might be safer to fallback to kpartx if not exists ...
  '%sp%s' % (self.device, self.partition)
That would support kernels before 3.2
I'd also add a comment in the code, that nbd must be mounted with param
max_part=16 or whatever.

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] File injection support

[Pádraig Brady]

On 06/11/2012 10:16 PM, Scott Moser wrote:
> Without digging around on older versions of OS's and their included
> kernel/udev/kpartx i'd have to say that from your message above, and
> my testing that running 'kpartx' is no longer necessary.  That can
> reasonably be expected to be created by the kernel or some other plumbing
> bits automatically.

It would be good to get away from kpartx. I've noticed these issues:

1. The git repo on kernel.org is no longer available.

2. kpartx -l had side effects:
  $ kpartx -l /bin/ls
  $ ls
  text file busy
To fix you need to run losetup -a to find the assigned loopback device
and then losetup -d /dev/loop...

3. On an unconnected loop device we get warnings, but an EXIT_SUCCESS ?
  # kpartx -a /dev/loop1 && echo EXIT_SUCCESS
  read error, sector 0
  llseek error
  llseek error
  llseek error
  EXIT_SUCCESS

4. Also for a loop device that is connected,
I get a "failed" warning, but the EXIT_SUCCESS
is appropriate in that case as the mapped device
is present and usable
  # kpartx -a /dev/loop0
 /dev/mapper/loop0p1: mknod for loop0p1 failed: File exists

That last item is related to the new code for auto parsing partitions.

That's only available since kernel 3.2 I think so we'll have to
be wary on relying on it.

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Welcome RHEL/CentOS/Fedora to docland!

[Pádraig Brady]

On 06/12/2012 10:59 AM, Rajesh Avula wrote:
> Hi Anne,
> 
> I am unable to open the link,  
> http://docs.openstack.org/essex/openstack-compute/install/apt/content/
> RHEL/CentOS/Fedora 
> 

Your mail client is messing up those I think.
Anyway the correct ones are:

¹ http://docs.openstack.org/essex/openstack-compute/install/apt/content/

² http://docs.openstack.org/essex/openstack-compute/install/yum/content/

¹ Ubuntu
² RHEL/Centos/Fedora

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] File injection support

[Pádraig Brady]

On 06/08/2012 04:35 PM, Scott Moser wrote:
> On Fri, 8 Jun 2012, Pádraig Brady wrote:
> 
>> On 06/08/2012 09:20 AM, Fredric Morenius wrote:
>>> Hello All,
>>>
>>> An update on the use of the qemu-nbd/kpartx based solution to inject files 
>>> into VM images:
>>>
>>> After some more testing it has turned out that injection into the UEC 
>>> version of CirrOS (this: 
>>> https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-uec.tar.gz)
>>>  works fine, but injection into the qcow2 version of the image produces the 
>>> error given in the previous mail, so there seems to be robustness problems 
>>> with these tools.
>>
>> Yes it's seems that qemu-nbd has issues with this image?
>>
>> # rpm -qf $(which qemu-nbd)
>> qemu-common-1.0-17.fc17.x86_64
>> # qemu-img info cirros-0.3.0-x86_64-disk.img
>> image: cirros-0.3.0-x86_64-disk.img
>> file format: qcow2
>> virtual size: 39M (41126400 bytes)
>> disk size: 9.3M
>> cluster_size: 65536
>> # qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-disk.img
>> # ls -la /sys/block/nbd15/pid
>> -r--r--r--. 1 root root 4096 Jun  8 10:19 /sys/block/nbd15/pid
>> # kpartx -a /dev/nbd15
>> device-mapper: resume ioctl on nbd15p1 failed: Invalid argument
>> create/reload failed on nbd15p1
>>
>> If I convert to raw with qemu-img, I can mount loopback fine.
> 
> Well, interesting. It does seem to work here:
> $ dpkg-query --show qemu-utils
> qemu-utils  1.0+noroms-0ubuntu13
> $ cd /tmp
> $ wget -q 
> https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-disk.img
> $ md5sum cirros-0.3.0-x86_64-disk.img
> 6654705afc4b74fda3e1f4330559d066  cirros-0.3.0-x86_64-disk.img

I downloaded again and get 50bdc35edb03a38d91b1b071afb20a3c
I presume the above md5sum is for a mounted/modified image?

> $ qemu-img info cirros-0.3.0-x86_64-disk.img
> image: cirros-0.3.0-x86_64-disk.img
> file format: qcow2
> virtual size: 39M (41126400 bytes)
> disk size: 11M
> cluster_size: 65536
> $ sudo qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-disk.img
> $ ls -l /dev/nbd15*
> brw-rw 1 root disk 43, 240 Jun  8 11:32 /dev/nbd15
> brw-rw 1 root disk 43, 241 Jun  8 11:32 /dev/nbd15p1

Very interesting. I don't get the ...p1 device here
on a 3.3.7.fc17.x86_64 kernel

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] File injection support

[Pádraig Brady]

On 06/08/2012 09:20 AM, Fredric Morenius wrote:
> Hello All,
> 
> An update on the use of the qemu-nbd/kpartx based solution to inject files 
> into VM images:
> 
> After some more testing it has turned out that injection into the UEC version 
> of CirrOS (this: 
> https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-uec.tar.gz)
>  works fine, but injection into the qcow2 version of the image produces the 
> error given in the previous mail, so there seems to be robustness problems 
> with these tools.

Yes it's seems that qemu-nbd has issues with this image?

# rpm -qf $(which qemu-nbd)
qemu-common-1.0-17.fc17.x86_64
# qemu-img info cirros-0.3.0-x86_64-disk.img
image: cirros-0.3.0-x86_64-disk.img
file format: qcow2
virtual size: 39M (41126400 bytes)
disk size: 9.3M
cluster_size: 65536
# qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-disk.img
# ls -la /sys/block/nbd15/pid
-r--r--r--. 1 root root 4096 Jun  8 10:19 /sys/block/nbd15/pid
# kpartx -a /dev/nbd15
device-mapper: resume ioctl on nbd15p1 failed: Invalid argument
create/reload failed on nbd15p1

If I convert to raw with qemu-img, I can mount loopback fine.

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Is openstack-glance package broken?

[Pádraig Brady]

On 06/05/2012 09:37 AM, Pádraig Brady wrote:
> On 06/05/2012 08:56 AM, Patrick Petit wrote:
>> Hi there,
>>
>> Trying to install OpenStack on Fedora 16 following instructions at 
>> http://fedoraproject.org/wiki/Getting_started_with_OpenStack_on_Fedora_17
>>
>> Very early in the installation process here is what I get. This is new to me 
>> as previous installs worked okay at this point. Does it ring a bell ?
>> Thanks
>> Patrick
>>
>> $ sudo openstack-db --service glance --init
>> Please enter the password for the 'root' MySQL user: 
>> Verified connectivity to MySQL.
>> Creating 'glance' database.
>> Asking openstack-glance to sync the database.
>> Traceback (most recent call last):
>>   File "/usr/bin/glance-manage", line 43, in 
>> from glance.common import cfg
>> ImportError: cannot import name cfg
>> ERROR 1146 (42S02) at line 1: Table 'glance.migrate_version' doesn't exist
>> Final sanity check failed.
>> Please file a bug report on bugzilla.redhat.com <http://bugzilla.redhat.com> 
>> against the openstack-glance package.
> 
> What package is this?
> $ rpm -q openstack-glance
> 
> Is this file present? It should be:
> /usr/lib/python2.7/site-packages/glance/common/cfg.py

Tying off loose ends...
Since glance.common.cfg was removed for folsom I suspected
that a folsom version of glance was installed and conflicting
with the packaged version.

This was confirmed with:

$ python -c 'from glance.common import config; print config.__file__'
/usr/lib/python2.7/site-packages/glance-2012.2-py2.7.egg/glance/common/config.pyc

So this issue is isolated to the OPs machine.

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Is openstack-glance package broken?

[Pádraig Brady]

On 06/05/2012 08:56 AM, Patrick Petit wrote:
> Hi there,
> 
> Trying to install OpenStack on Fedora 16 following instructions at 
> http://fedoraproject.org/wiki/Getting_started_with_OpenStack_on_Fedora_17
> 
> Very early in the installation process here is what I get. This is new to me 
> as previous installs worked okay at this point. Does it ring a bell ?
> Thanks
> Patrick
> 
> $ sudo openstack-db --service glance --init
> Please enter the password for the 'root' MySQL user: 
> Verified connectivity to MySQL.
> Creating 'glance' database.
> Asking openstack-glance to sync the database.
> Traceback (most recent call last):
>   File "/usr/bin/glance-manage", line 43, in 
> from glance.common import cfg
> ImportError: cannot import name cfg
> ERROR 1146 (42S02) at line 1: Table 'glance.migrate_version' doesn't exist
> Final sanity check failed.
> Please file a bug report on bugzilla.redhat.com  
> against the openstack-glance package.

What package is this?
$ rpm -q openstack-glance

Is this file present? It should be:
/usr/lib/python2.7/site-packages/glance/common/cfg.py

cheers,
Pádraig

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Single Server Test

[Pádraig Brady]

On 06/04/2012 01:00 AM, Chandana De Silva wrote:
> Hello All,
> 
> I have not worked with OpenStack before this, and want to set up a minimal 
> test instance of Open Stack on a single server.
> 
> Is this possible, and if so is there a document which goes through the steps ?
> 
> If there is not, could some one give me a (very brief ) outline, or at least 
> a series of links to existing docs in the correct order ?
> 
> I need to do this on a Centos 6.2 host, as that is the target deployment 
> platform.

This is possible. It's even possible to install within a VM. Details here:
https://fedoraproject.org/wiki/Getting_started_with_OpenStack_EPEL

Note the upstream OpenStack docs are also in the process of
being updated with Centos specific information.
It would be preferable to use those docs if possible,
and cross reference with the link above.
The upstream docs are available as a review commit at present:
https://review.openstack.org/#/c/7431/
Anne (CC'd) may have a preview pdf available for easier consumption.

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] metadata and file injection

[Pádraig Brady]

On 06/01/2012 10:55 AM, William Herry wrote:
> I have been spend all days to search metadata and file injection related info 
> with Google,
> there are few doc or blog about this topic (or I am not use the proper 
> keyword)
> I know about this is only pieces
> 
> what I know now is:
> cloud-init can inject files to instance and a lot other things, seems only on 
> ubuntu, 

cloud-init is available for Fedora now.
There is also a test package available for RHEL and derivatives:
http://pbrady.fedorapeople.org/cloud-init-el6/

> I can inject ssh key with qemu-nbd

If you can do that, you can also inject 'metadata',
'files' and root 'passwords'. Note only failure to
inject 'files' will result in a failure to boot the guest.

> my question is:
> what is in metadata, how can I use matadata
> there is no injection related sub command in nova
> 
> any related URL will be very appreciate

The `nova` command interface is defined in:
https://github.com/openstack/python-novaclient/blob/master/novaclient/v1_1/shell.py
It would be good to have a man page for the `nova`
command that you could reference.

Anyway here is what you get when executing the
help for the 'boot' command:


# nova help boot
usage: nova boot [--flavor ] [--image ] [--meta ]
 [--file ] [--key_name ]
 [--user_data ]
 [--availability_zone ]
 [--security_groups ]
 [--block_device_mapping ]
 [--hint ]
 [--nic ]
 [--config-drive ] [--poll]
 

Boot a new server.

Positional arguments:
  Name for the new server

Optional arguments:
  --flavor  Flavor ID (see 'nova flavor-list').
  --imageImage ID (see 'nova image-list').
  --meta Record arbitrary key/value metadata. May be give
multiple times.
  --file 
Store arbitrary files from  locally to  on the new server. You may store up to 5 files.
  --key_name 
Key name of keypair that should be created earlier
with the command keypair-add
  --user_data 
user data file to pass to be exposed by the metadata
server.
  --availability_zone 
The availability zone for instance placement.
  --security_groups 
comma separated list of security group names.
  --block_device_mapping 
Block device mapping in the format :::.
  --hint Send arbitrary key/value pairs to the scheduler for
custom use.
  --nic 
Create a NIC on the server. Specify option multiple
times to create multiple NICs. net-id: attach NIC to
network with this UUID (optional) v4-fixed-ip: IPv4
fixed address for NIC (optional).
  --config-drive 
Enable config drive
  --pollBlocks while instance builds so progress can be
reported.


Here are the related APIs that are used by the command above:
http://docs.openstack.org/api/openstack-compute/2/content/CreateServers.html

So what happens when you present --meta options above?
That will bubble through the API to:
https://github.com/openstack/nova/blob/master/nova/virt/disk/api.py
If you look at _inject_metadata_into_fs() there, you can see
that a 'meta.js' file is written to the / directory.
Logic within the guest can then inspect that as required.

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Openstack with CentOS 6.2

[Pádraig Brady]

On 05/31/2012 03:22 PM, Pádraig Brady wrote:
> On 05/31/2012 08:39 AM, Vogel Nicolas wrote:
>> Hi everybody,
>>
>>  
>>
>> I’m trying to install Openstack (Essex release) with CentOS 6.2 and I have a 
>> lot of problems because of dependencies and a deficiency of installation 
>> documents.
>>
>> I think I get all the actual packages and I try to make the install with the 
>> official doc, which is based on Ubuntu. So I have to search and modify at 
>> every step, and that’s very difficult for me.
>>
>> For example, beginning with keystone, the syntax for creating tenant, users 
>> and roles is different. The keystone.conf file must be changed too.
>>
>> And then I tried to continue with glance, but both .ini files are missing 
>> and I don’t know if I have to create them or if I have to adapt the .conf 
>> files.
>>
>>  
>>
>> If someone could give me documentation or a link for the installation of 
>> Essex release on CentOS/RHEL 6.2, I would be really gratefull.
> 
> The best approach at this stage is to use the packages from EPEL.
> https://fedoraproject.org/wiki/Getting_started_with_OpenStack_EPEL
> 
> The openstack upstream docs are currently in the process of
> being updated with much of this info.

Actually Anne's suggestion of following the official docs
(that she's sending you a preview of) is much better.
They're much more integrated and in-depth.
You can use the URL above as a cross reference for issues etc.

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Openstack with CentOS 6.2

[Pádraig Brady]

On 05/31/2012 08:39 AM, Vogel Nicolas wrote:
> Hi everybody,
> 
>  
> 
> I’m trying to install Openstack (Essex release) with CentOS 6.2 and I have a 
> lot of problems because of dependencies and a deficiency of installation 
> documents.
> 
> I think I get all the actual packages and I try to make the install with the 
> official doc, which is based on Ubuntu. So I have to search and modify at 
> every step, and that’s very difficult for me.
> 
> For example, beginning with keystone, the syntax for creating tenant, users 
> and roles is different. The keystone.conf file must be changed too.
> 
> And then I tried to continue with glance, but both .ini files are missing and 
> I don’t know if I have to create them or if I have to adapt the .conf files.
> 
>  
> 
> If someone could give me documentation or a link for the installation of 
> Essex release on CentOS/RHEL 6.2, I would be really gratefull.

The best approach at this stage is to use the packages from EPEL.
https://fedoraproject.org/wiki/Getting_started_with_OpenStack_EPEL

The openstack upstream docs are currently in the process of
being updated with much of this info.

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] File injection support

[Pádraig Brady]

On 05/30/2012 03:47 PM, Fredric Morenius wrote:
> Hello Pádraig,
> 
> I am also trying to get file injection to work in Essex, but have run into 
> some issues, as stated here: 
> https://answers.launchpad.net/nova/+question/198878

Igor Laskovy also had that "device-mapper: resume ioctl failed:" issue
with the "qemu-nbd" on a cirros image too,
though he didn't need the injection though and just avoided it.

> The image I am launching is a simple bare container qcow2 image (CirrOS, 
> this: 
> https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-disk.img
>  )

That image is a simple single partition image,
and so should not need the patch referenced below.
What system are you using? If you installed libguestfs then
that should be tried as a method without need for the patch below.

> Would it be possible to backport this: 
> https://github.com/openstack/nova/commit/2b3a1e7
> So that file injection as I am trying to do it will work? Or is there any 
> other way to make it work?

The backport is trivial and already done in the Fedora/EPEL Essex packages.

I was thinking though that this was extra functionality
and so not appropriate for the official stable branch?

> * NOT A CONTRIBUTION *
> The content of this email shall not be considered as a contribution to 
> OpenStack

:)

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] File injection support

[Pádraig Brady]

On 05/29/2012 07:35 PM, Nicolae Paladi wrote:
> Hello, 
> 
> I am looking for more information about file injection support in OpenStack
> and find it increasingly inconsistent and incomplete. I have several 
> questions:
> 
> * This wiki article (http://wiki.openstack.org/HypervisorSupportMatrix) 
> claims 
> that file injection is supported in XenServer, is NOT supported inn KVM
> it's not clear about the others. is that still the case? 

One of the key Essex Features was "Hypervisor Feature Parity", including:
Disk Config, resize and file injection.

I've updated the wiki accordingly.

So there is support in "kvm" for injecting ssh keys, net info, root password,
or arbitrary files.

By default there are 3 methods tried in turn to do this:
loopback mount, qemu-nbd, libguestfs.

In Essex injection is supported to the first partition of guest images.
Folsom enables further libguestfs functionality to inspect arbitrarily
complex guest images and find the root partition to inject to.

> At least there is 
> some mentioning of file injection in the openstack xenserver wiki:
> http://wiki.openstack.org/XenServer/Development
> 
> * Is there any guide or man page where file injection is explained? in this
> use case I would like to inject a file to e.g. /etc/security/.conf at 
> launch 
> time -- is that possible, and if yes how can it be done in the Essex release. 

The API docs explain the "personality" parameter to achieve this:
http://docs.openstack.org/api/openstack-compute/2/content/CreateServers.html
This is exposed on the nova command line tool though the --injected-files 
option.

> * Are there any discussions about "the future" of file injection?

There are other options, like cloud-init or using a config drive.
You enable a config drive with the nova --config-drive option,
with value as a boolean or a volume ID. See also:
https://blueprints.launchpad.net/nova/+spec/config-drive-v2

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Nova] Getting error when injecting data into an image

[Pádraig Brady]

On 05/25/2012 09:35 AM, Anton Haldin wrote:
> Thank you very much Pádraig
> 
> It's cool to have openstack with well configured selinux
> 
> Did you mean this page 
> http://fedoraproject.org/wiki/Getting_started_with_OpenStack_on_Fedora_17 ?

Right, and the EPEL variant

https://fedoraproject.org/wiki/Getting_started_with_OpenStack_EPEL

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Nova] Getting error when injecting data into an image

[Pádraig Brady]

On 05/24/2012 02:49 PM, Patrick Petit wrote:
> 
> 
> 2012/5/24 Pádraig Brady mailto:p...@draigbrady.com>>
> 
> On 05/24/2012 01:26 PM, Patrick Petit wrote:
> > Hi Pádraig,
> >
> > Thank you for your reply.
> >
> > I applied the suggested  libvirt_inject_partition = -1
> >
> > It does change things because it's not complaining about mount error 
> any more but generates further errors down the path in libvirt.py"
> 
> > 2012-05-24 14:11:54 TRACE nova.compute.manager [instance: 
> a5d5b5c7-093a-4eb4-9d70-d6cfa2b30a31]   File 
> "/usr/lib64/python2.7/site-packages/libvirt.py", line 540, in createWithFlags
> > 2012-05-24 14:11:54 TRACE nova.compute.manager [instance: 
> a5d5b5c7-093a-4eb4-9d70-d6cfa2b30a31] if ret == -1: raise libvirtError 
> ('virDomainCreateWithFlags() failed', dom=self)
> > 2012-05-24 14:11:54 TRACE nova.compute.manager [instance: 
> a5d5b5c7-093a-4eb4-9d70-d6cfa2b30a31] libvirtError: internal error Process 
> exited while reading console log output: char device redirected to /dev/pts/2
> > 2012-05-24 14:11:54 TRACE nova.compute.manager [instance: 
> a5d5b5c7-093a-4eb4-9d70-d6cfa2b30a31] Could not allocate dynamic translator 
> buffer
> 
> That's libvirt specific.
> 
> Do you have "Virtual Machine" support enabled in your BIOS?
> If not the fallback mode is currently disallowed by SELinux,
> and you can enable with: setsebool -P virt_use_execmem=on
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=753589#c36
> 
> It's quite a non specific error so it could be something else.
> 
> I am running the all OpenStack in a virtual machine under VBox.
> nova.conf has libvirt_type = qemu
> 
> Disabling SELinux solved the problem.
> Shouldn't that be reported as a bug?

I think the bug referenced above captures the issue sufficiently.
What I've done is to document the specific SELinux setting above,
to enable this mode of operation, in the fedora/epel wiki where
you said you were following the instructions.

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Nova] Getting error when injecting data into an image

[Pádraig Brady]

On 05/24/2012 01:26 PM, Patrick Petit wrote:
> Hi Pádraig,
> 
> Thank you for your reply.
> 
> I applied the suggested  libvirt_inject_partition = -1
> 
> It does change things because it's not complaining about mount error any more 
> but generates further errors down the path in libvirt.py"

> 2012-05-24 14:11:54 TRACE nova.compute.manager [instance: 
> a5d5b5c7-093a-4eb4-9d70-d6cfa2b30a31]   File 
> "/usr/lib64/python2.7/site-packages/libvirt.py", line 540, in createWithFlags
> 2012-05-24 14:11:54 TRACE nova.compute.manager [instance: 
> a5d5b5c7-093a-4eb4-9d70-d6cfa2b30a31] if ret == -1: raise libvirtError 
> ('virDomainCreateWithFlags() failed', dom=self)
> 2012-05-24 14:11:54 TRACE nova.compute.manager [instance: 
> a5d5b5c7-093a-4eb4-9d70-d6cfa2b30a31] libvirtError: internal error Process 
> exited while reading console log output: char device redirected to /dev/pts/2
> 2012-05-24 14:11:54 TRACE nova.compute.manager [instance: 
> a5d5b5c7-093a-4eb4-9d70-d6cfa2b30a31] Could not allocate dynamic translator 
> buffer

That's libvirt specific.

Do you have "Virtual Machine" support enabled in your BIOS?
If not the fallback mode is currently disallowed by SELinux,
and you can enable with: setsebool -P virt_use_execmem=on

https://bugzilla.redhat.com/show_bug.cgi?id=753589#c36

It's quite a non specific error so it could be something else.

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Nova] Getting error when injecting data into an image

[Pádraig Brady]

On 05/24/2012 11:38 AM, Patrick Petit wrote:
> Hi,
> I am getting the following error when running
> 
> $ nova boot myserver --flavor 2 --key_name mykey --image 
> 661bbe35-ebe5-4614-bdb2-3259ea507934
> 
> +-+--+
> |   Property  |Value |
> +-+--+
> | OS-DCF:diskConfig   | MANUAL   |
> | OS-EXT-SRV-ATTR:host| None |
> | OS-EXT-SRV-ATTR:hypervisor_hostname | None |
> | OS-EXT-SRV-ATTR:instance_name   | instance-0005|
> | OS-EXT-STS:power_state  | 0|
> | OS-EXT-STS:task_state   | scheduling   |
> | OS-EXT-STS:vm_state | building |
> | accessIPv4  |  |
> | accessIPv6  |  |
> | adminPass   | mAnfCStaqWT2 |
> | config_drive|  |
> | created | 2012-05-23T13:49:12Z |
> | flavor  | m1.small |
> | hostId  |  |
> | id  | 18cde301-e8c9-4721-928b-cd0daf63a4f0 |
> | image   | f16-jeos |
> | key_name| mykey|
> | metadata| {}   |
> | name| myserver |
> | progress| 0|
> | status  | BUILD|
> | tenant_id   | 873855379940442797e53f2fa437893f |
> | updated | 2012-05-23T13:49:13Z |
> | user_id | 5677a018b8924cc58f993101c3024794 |
> +-+--+
> 
> 
> The image was obtained from following the Getting Started with OpenStack on 
> Fedora 17 tutorial 
> (http://fedoraproject.org/wiki/Getting_started_with_OpenStack_on_Fedora_17). 
> So, I guess I am not the only one using it.
> 
> $ glance index
> 
> ID   Name   Disk 
> Format  Container Format Size
> 
>  -- 
>   --
> 
> 661bbe35-ebe5-4614-bdb2-3259ea507934 f16-jeos   qcow2 
>ovf   213581824
> 
> c15e90f2-e73e-4987-ad7a-11d87403012e cirros-0.3.0-x86_64-ariari   
>ari 2254249
> 
> 68ad4ece-6a56-4ac8-b112-1dd69283ea83 cirros-0.3.0-x86_64-amiami   
>ami25165824
> 
> 6f5d8022-2dfe-406d-b391-fa0e48c175f3 cirros-0.3.0-x86_64-akiaki   
>aki 4731440
> 
> 
> This is running on Nova Essex on Fedora 16.
> After a while I get:
> 
> $ nova list
> 
> +--+--++--+
> |  ID  |   Name   | Status | Networks 
> |
> +--+--++--+
> | 18cde301-e8c9-4721-928b-cd0daf63a4f0 | myserver | ERROR  | demonet=10.0.0.2 
> |
> +--+--++--+
> 
> 
> And so the log:
> 
> 2012-05-23 15:50:12 INFO nova.virt.libvirt.connection [-] Compute_service 
> record updated for fedora.localdomain
> 
> 2012-05-23 15:50:35 WARNING nova.virt.libvirt.connection 
> [req-dd9a661c-94d3-42e4-b7ba-699c7b41def4 5677a018b8924cc58f993101c3024794 
> 873855379940442797e53f2fa437893f] [instance: 
> 18cde301-e8c9-4721-928b-cd0daf63a4f0] Ignoring error injecting data into 
> image 661bbe35-ebe5-4614-bdb2-3259ea507934 (
> 
> -- 
> 
> Failed to mount filesystem: Unexpected error while running command.
> 
> Command: sudo nova-rootwrap mount /dev/mapper/nbd15p1 /tmp/tmpM9dOLC
> 
> Exit code: 32
> 
> Stdout: ''
> 
> Stderr: 'mount: you must specify the filesystem type\n'
> 
> -- 
> 
> Failed to mount filesystem: Unexpected error while running command.
> 
> Command: sudo nova-rootwrap guestmount --rw -a 
> /var/lib/nova/instances/instance-0005/disk -m /dev/sda1 /tmp/tmpM9dOLC
> 
> Exit code: 1
> 
> Stdout: ''
> 
> Stderr: "libgues

Re: [Openstack] upload file from a specific directory

[Pádraig Brady]

On 05/22/2012 11:43 AM, khabou imen wrote:
> Hi everybody,
> when runnig this  command 
> swift -v -V 2.0 -A http://192.168.1.5:5000/v2.0/  -U service:swift -K 
> swiftpass upload Containera doc1.pdf
> the file  "doc1.pdf" is well uploaded  only if it's placed in the home 
> directory
> How can I upload a file from a different directory
> such as /home/imen/Desktop/img1.jpg?

Referencing arbitrary files works fine here:

$ rpm -qf $(which swift)
openstack-swift-1.4.8-2.el6.noarch

$ swift upload c1 /etc/issue
etc/issue
$ (cd /etc; swift upload c2 resolv.conf)
resolv.conf

$ mkdir t
$ cd t

$ swift download --all
c1/etc/issue
c2/resolv.conf
$ swift download c1
etc/issue
$ swift download c2
resolv.conf

$ find -ls
2770274 drwxrwxr-x   5 padraig  padraig  4096 May 23 00:29 .
2775564 drwxrwxr-x   2 padraig  padraig  4096 May 23 00:29 ./etc
2775584 -rw-rw-r--   1 padraig  padraig85 Mar  8 12:31 ./etc/issue
2774374 drwxrwxr-x   2 padraig  padraig  4096 May 23 00:29 ./c2
2774384 -rw-rw-r--   1 padraig  padraig55 May 22 13:41 
./c2/resolv.conf
2774344 drwxrwxr-x   3 padraig  padraig  4096 May 23 00:29 ./c1
2774354 drwxrwxr-x   2 padraig  padraig  4096 May 23 00:29 ./c1/etc
2774364 -rw-rw-r--   1 padraig  padraig85 Mar  8 12:31 
./c1/etc/issue
2776464 -rw-rw-r--   1 padraig  padraig55 May 22 13:41 ./resolv.conf

Note the pseudo hierarchical directories are supported through:
http://docs.openstack.org/api/openstack-object-storage/1.0/content/pseudo-hierarchical-folders-directories.html

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] centos 6 images

[Pádraig Brady]

On 05/22/2012 05:51 PM, Scott Moser wrote:
> On Tue, 22 May 2012, Pádraig Brady wrote:
> 
>> On 05/22/2012 03:39 PM, Andy Grimm wrote:
>>> On Tue, May 22, 2012 at 9:38 AM, Pádraig Brady  wrote:
>>>> On 05/22/2012 04:07 AM, Jason Ford wrote:
>>>>> I am trying to put together an image for centos 6 that works like 
>>>>> cloud-init on ubuntu does. Currently I have ssh keys getting imported but 
>>>>> having some problems getting the disk to dynamically resize to the flavor 
>>>>> template as well as the hostname set in horizon to be pushed into the 
>>>>> image. Does anyone have any howtos or suggestions on how to get this 
>>>>> done? Is there cloud-init for centos just like ubuntu? I would also be 
>>>>> interested in how to do this with debian as well.
>>>>
>>>> Well I notice there is no cloud-init package for EPEL.
>>>> I took a quick stab at it here:
>>>> http://pbrady.fedorapeople.org/cloud-init-el6/
>>>
>>> I've already responded in IRC, but it wouldn't hurt to have a response
>>> in the mail archive.  In short, the reason there isn't already a
>>> cloud-init for EL6 (or EL5, for that matter) is that upstream has been
>>> using python 2.7-only calls for a while now.  In particular, a couple
>>> of calls to subprocess.check_output need to be replaced, and I think
>>> there are a few other issues as well.  I don't think it's a huge
> 
> It would help if you'd bring that up with upstream :)
> I'm interested in cloud-init working in the most places it can.  I'll try
> to pull in the sysvinit scripts that Pádraig added and grab other changes
> that are there.

Excellent.
I'll submit as much as I can upstream anyway after some more testing.

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] centos 6 images

[Pádraig Brady]

On 05/22/2012 03:39 PM, Andy Grimm wrote:
> On Tue, May 22, 2012 at 9:38 AM, Pádraig Brady  wrote:
>> On 05/22/2012 04:07 AM, Jason Ford wrote:
>>> I am trying to put together an image for centos 6 that works like 
>>> cloud-init on ubuntu does. Currently I have ssh keys getting imported but 
>>> having some problems getting the disk to dynamically resize to the flavor 
>>> template as well as the hostname set in horizon to be pushed into the 
>>> image. Does anyone have any howtos or suggestions on how to get this done? 
>>> Is there cloud-init for centos just like ubuntu? I would also be interested 
>>> in how to do this with debian as well.
>>
>> Well I notice there is no cloud-init package for EPEL.
>> I took a quick stab at it here:
>> http://pbrady.fedorapeople.org/cloud-init-el6/
> 
> I've already responded in IRC, but it wouldn't hurt to have a response
> in the mail archive.  In short, the reason there isn't already a
> cloud-init for EL6 (or EL5, for that matter) is that upstream has been
> using python 2.7-only calls for a while now.  In particular, a couple
> of calls to subprocess.check_output need to be replaced, and I think
> there are a few other issues as well.  I don't think it's a huge
> amount of work to make it functional, but it hasn't been high on
> anyone's list.  It would be cool if you have time to fix / test it,
> though.

Ok I've fixed the check_output calls at the above URL.

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] centos 6 images

[Pádraig Brady]

On 05/22/2012 04:07 AM, Jason Ford wrote:
> I am trying to put together an image for centos 6 that works like cloud-init 
> on ubuntu does. Currently I have ssh keys getting imported but having some 
> problems getting the disk to dynamically resize to the flavor template as 
> well as the hostname set in horizon to be pushed into the image. Does anyone 
> have any howtos or suggestions on how to get this done? Is there cloud-init 
> for centos just like ubuntu? I would also be interested in how to do this 
> with debian as well.

Well I notice there is no cloud-init package for EPEL.
I took a quick stab at it here:
http://pbrady.fedorapeople.org/cloud-init-el6/

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] diablo release rpms

[Pádraig Brady]

On 05/17/2012 01:53 AM, Xin Zhao wrote:
> Hello,
> 
> I have a diablo testbed, for which I have to add more compute nodes. We use 
> RHEL6. But the current EL6 repo looks already upgraded to Essex release. So I 
> wonder if there is any way I can get back to the diablo rpms?

You can get earlier packages from http://kojipkgs.fedoraproject.org/packages/

You could create your own repo from these and give it a higher priority.
For your convenience I've created (but not tested) such a repo here:

  http://pbrady.fedorapeople.org/openstack-diablo-el6/

Details are in the README there.

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Launching vpn image issue

[Pádraig Brady]

On 05/05/2012 02:45 AM, Anton Haldin wrote:
> hi 
> 
> I found guestmount in libguestfs-tools-c package.
> and I had such issue with guestmount in centos.
> 
> in ubuntu I thought by default nova-compute would use nbd .  if you did not 
> change   img_handlers flag.

By default loop,nbd,libguestfs injection methods are tried in that order.
You may need to ensure the nbd module is loaded for that method to work.

If you do need the extra capabilities of libguestfs, then...

According to http://packages.ubuntu.com/, there
is a guestmount package available for precise.
However there may be issues with that:
https://www.redhat.com/archives/libguestfs/2012-April/thread.html#00028

There are also oneric versions available independently:
http://libguestfs.org/download/binaries/ubuntu1004-packages/

For the record, the Fedora and the EPEL (for use with centos etc.)
openstack packages, auto install the libguestfs-mount package.

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Flagfiles vs. .ini files

[Pádraig Brady]

On 05/02/2012 08:32 AM, Ghe Rivero wrote:
> Essex was released with support for both types of configuration files, 
> flagfiles (using the --) and the .ini file style, being the 
> "nova.conf.sample" file included in nova upstream code using the .ini format. 
> 
> From a deployer point of view, this can be really confusing since official 
> docs, use the .ini format, but Ubuntu 12.04 and docs based on ubuntu, use the 
> old deprecated flagfile format.
> 
> I think that we should start using only .ini files and stop supporting the 
> old flagfile behavior. The sooner we remove from the development cycle, the 
> more time the documenters and distributions have to catch up with the "new" 
> format. Anyway, nova-manage can still be used to convert from flagfiles to 
> .ini files: (I wouldn't remove this in folsom cycle)

I agree, having multiple ways to do stuff is problematic.
Is it too late to change the ubuntu configs/docs now?
We should at least output warnings about the old format.

Standardising on the new ini style format make sense,
and that's what has been done in the Essex releases in Fedora and EPEL.
Note to simplify the docs and allow users to cut and paste instructions,
we also include a utility to edit these config files:

https://github.com/fedora-openstack/openstack-utils/blob/master/utils/openstack-config

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] File injection and disk resize not working

[Pádraig Brady]

On 04/25/2012 01:13 PM, Alvaro Lopez wrote:
> Dear.
> 
> I've run into troubles with the resize of partitions and file injection;
> using libvirt and raw images.
> 
> According to https://bugs.launchpad.net/nova/+bug/929005 file injection
> into images only works in the following situations:
>   1.- whole disk image -> inject to partition 1.
>   2.- separate kernel and ramdisk -> inject without partitions.
> 
> In situation 1 (i.e. using whole disk images, with one partition
> inside) the file injection works fine, the disk is resized to the
> actual size, but the partition is not extended. This is because the
> 'extend' function in 'nova/virt/disk/api.py' executes 'qemu-img',
> 'e2fsck' and 'resize2fs' in a row, assuming that there are no
> partitions at all.
> 
> In situation 2 (i.e. using whole disk images) if there is no kernel
> and ramdisk (but the file is a self-contained disk with an appropriate
> kernel) file injection does not work, but the resize works fine (since
> there are no partitions and the fsck and resize work as expected).
> 
> Shouldn't the file injection be extended so as to make it work in any
> situation? That is, add injection without partitions and without
> kernel/ramdisk.

That should be mostly handled with:
https://review.openstack.org/#/c/6668/

> Shouldn't the resize of disk take into account that it might be a disk
> with a (single) partition inside? This would imply dealing with the
> partitions (that is, delete and recreate them to the appropiate size).

Well that would be tricky to do generally, given that
the internal partition structure could be arbitrarily complicated.

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] compute create instance failure

[Pádraig Brady]

On 04/17/2012 03:54 PM, Nathanael Burton wrote:
> I've run into a similar problem when using whole disk (no separate kernel / 
> ramdisk) images with LVM. In my case /dev/sda1 was the /boot file system.  
> What I did is modify the code to let guestmount do its thing by always using 
> the "-i" option to inspect. I don't quite understand why that isn't the 
> default behaviour.

It's not the default because of introspection issues with older versions of 
libguestfs.
You essentially did the same workaround as I did with setting partition=-1
I'll do a patch supporting a configurable partition ASAP.

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] compute create instance failure

[Pádraig Brady]

On 04/11/2012 10:17 PM, Craig Vyvial wrote:
> I've run into a few issues while i have been testing creating and deleting 
> instances on my vm after setting everything up with devstack. I create a new 
> instance and it goes into an error state. the log says it failed to map 
> partitions but this is the same image i have been using without problems 
> before. its a qcow2 image i created with ubuntu-vmbuilder. 
> 
> Anyone else see this?
> 
> i thought maybe i was out of memory but thats not the case.
> ubuntu@ubuntu:/opt/stack$ df
> Filesystem   1K-blocks  Used Available Use% Mounted on
> /dev/sda1 18578172   7169564  10464892  41% /
> udev501644 4501640   1% /dev
> tmpfs   203828   324203504   1% /run
> none  5120 0  5120   0% /run/lock
> none509560 0509560   0% /run/shm
> 
> Excerpt from the nova-compute logs:
> 
> 2012-04-11 13:54:26 DEBUG nova.virt.libvirt.connection 
> [req-91b3855a-1f90-42bb-8c24-6bf546aed826 2e11238b4cf444ba8dcf321a98764050 
> cb0ad32733bb4366962ba76033f4b6fb] [instance: 
> 939d3af8-e7bd-4d4b-b026-c20097e207a6] Finished toXML method from (pid=2720) 
> to_xml /opt/stack/nova/nova/virt/libvirt/connection.py:1662
> 2012-04-11 13:54:26 INFO nova.virt.libvirt.firewall 
> [req-91b3855a-1f90-42bb-8c24-6bf546aed826 2e11238b4cf444ba8dcf321a98764050 
> cb0ad32733bb4366962ba76033f4b6fb] [instance: 
> 939d3af8-e7bd-4d4b-b026-c20097e207a6] Called setup_basic_filtering in nwfilter
> 2012-04-11 13:54:26 INFO nova.virt.libvirt.firewall 
> [req-91b3855a-1f90-42bb-8c24-6bf546aed826 2e11238b4cf444ba8dcf321a98764050 
> cb0ad32733bb4366962ba76033f4b6fb] [instance: 
> 939d3af8-e7bd-4d4b-b026-c20097e207a6] Ensuring static filters
> 2012-04-11 13:54:26 DEBUG nova.virt.firewall 
> [req-91b3855a-1f90-42bb-8c24-6bf546aed826 2e11238b4cf444ba8dcf321a98764050 
> cb0ad32733bb4366962ba76033f4b6fb] Filters added to instance 
> 939d3af8-e7bd-4d4b-b026-c20097e207a6 from (pid=2720) prepare_instance_filter 
> /opt/stack/nova/nova/virt/firewall.py:137
> 2012-04-11 13:54:26 DEBUG nova.utils 
> [req-91b3855a-1f90-42bb-8c24-6bf546aed826 2e11238b4cf444ba8dcf321a98764050 
> cb0ad32733bb4366962ba76033f4b6fb] Attempting to grab semaphore "iptables" for 
> method "_do_refresh_provider_fw_rules"... from (pid=2720) inner 
> /opt/stack/nova/nova/utils.py:929
> 2012-04-11 13:54:26 DEBUG nova.utils 
> [req-91b3855a-1f90-42bb-8c24-6bf546aed826 2e11238b4cf444ba8dcf321a98764050 
> cb0ad32733bb4366962ba76033f4b6fb] Got semaphore "iptables" for method 
> "_do_refresh_provider_fw_rules"... from (pid=2720) inner 
> /opt/stack/nova/nova/utils.py:933
> 2012-04-11 13:54:26 DEBUG nova.utils 
> [req-91b3855a-1f90-42bb-8c24-6bf546aed826 2e11238b4cf444ba8dcf321a98764050 
> cb0ad32733bb4366962ba76033f4b6fb] Attempting to grab file lock "iptables" for 
> method "_do_refresh_provider_fw_rules"... from (pid=2720) inner 
> /opt/stack/nova/nova/utils.py:937
> 2012-04-11 13:54:26 DEBUG nova.utils 
> [req-91b3855a-1f90-42bb-8c24-6bf546aed826 2e11238b4cf444ba8dcf321a98764050 
> cb0ad32733bb4366962ba76033f4b6fb] Got file lock "iptables" for method 
> "_do_refresh_provider_fw_rules"... from (pid=2720) inner 
> /opt/stack/nova/nova/utils.py:944
> 2012-04-11 13:54:26 DEBUG nova.utils 
> [req-91b3855a-1f90-42bb-8c24-6bf546aed826 2e11238b4cf444ba8dcf321a98764050 
> cb0ad32733bb4366962ba76033f4b6fb] Attempting to grab semaphore "iptables" for 
> method "apply"... from (pid=2720) inner /opt/stack/nova/nova/utils.py:929
> 2012-04-11 13:54:26 DEBUG nova.utils 
> [req-91b3855a-1f90-42bb-8c24-6bf546aed826 2e11238b4cf444ba8dcf321a98764050 
> cb0ad32733bb4366962ba76033f4b6fb] Got semaphore "iptables" for method 
> "apply"... from (pid=2720) inner /opt/stack/nova/nova/utils.py:933
> 2012-04-11 13:54:26 DEBUG nova.utils 
> [req-91b3855a-1f90-42bb-8c24-6bf546aed826 2e11238b4cf444ba8dcf321a98764050 
> cb0ad32733bb4366962ba76033f4b6fb] Attempting to grab file lock "iptables" for 
> method "apply"... from (pid=2720) inner /opt/stack/nova/nova/utils.py:937
> 2012-04-11 13:54:26 DEBUG nova.utils 
> [req-91b3855a-1f90-42bb-8c24-6bf546aed826 2e11238b4cf444ba8dcf321a98764050 
> cb0ad32733bb4366962ba76033f4b6fb] Got file lock "iptables" for method 
> "apply"... from (pid=2720) inner /opt/stack/nova/nova/utils.py:944
> 2012-04-11 13:54:26 DEBUG nova.utils 
> [req-91b3855a-1f90-42bb-8c24-6bf546aed826 2e11238b4cf444ba8dcf321a98764050 
> cb0ad32733bb4366962ba76033f4b6fb] Running cmd (subprocess): sudo 
> /usr/local/bin/nova-rootwrap iptables-save -t filter from (pid=2720) execute 
> /opt/stack/nova/nova/utils.py:220
> 2012-04-11 13:54:27 DEBUG nova.utils 
> [req-91b3855a-1f90-42bb-8c24-6bf546aed826 2e11238b4cf444ba8dcf321a98764050 
> cb0ad32733bb4366962ba76033f4b6fb] Running cmd (subprocess): sudo 
> /usr/local/bin/nova-rootwrap iptables-restore from (pid=2720) execute 
> /opt/stack/nova/nova/utils.py:220
> 2012-04-11 13:54:27 DEBUG nova.util

Re: [Openstack] Essex on RHEL 6.2 and Cent OS 6.2

[Pádraig Brady]

On 04/16/2012 04:19 PM, Salman A Baset wrote:
> Hello folks,
> 
> Has anyone tried to install Essex compute, and cloud controller on RHEL 6.2 
> or Cent OS 6.2? Any experiences?
> 
> We tried to install Openstack Essex (2012) release on RHEL 6.2. Since there 
> are no packages ready for it, we tried to use Fedora 16, 17 and 18 packages. 
> In all cases, we found that it not only requires a large number of 
> dependencies, as some of them are only available in the not yet released RHEL 
> 6.3 (like dnsmasq-utils), but that it also requires python 2.7.

Note we're finalizing Essex packages within EPEL.
There is a preview repo available for dev testing,
with a README there detailing setup notes.

http://pbrady.fedorapeople.org/openstack-el6/

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Openstack in centos 6.2 install document

[Pádraig Brady]

On 04/16/2012 05:03 PM, darkfower wrote:
> hi,every one:
> 
> The attachment is I wrote in the installation of openstack centos 6.2 document

That's excellent thanks!

Note we're finalizing Essex packages within EPEL too.
There is a preview repo available for dev testing,
with a README there detailing setup notes.

http://pbrady.fedorapeople.org/openstack-el6/

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] running HA cluster of guests within openstack

[Pádraig Brady]

On 04/13/2012 10:31 AM, ikke wrote:
> I likely am not the first one to ask this, but since I didn't find a
> thread about it I start one.
> 
> Is there any shared experience available what are the capabilities of
> OpenStack to run cluster of guests in the cloud? Do you have
> experience of the following questions, or links to more info? The
> questions relate to running a legacy HA cluster in virtual env, and
> moving it into cloud...

I'll just point out two early stage projects
that used in combination can provide a HA solution.

http://wiki.openstack.org/Heat
http://wiki.openstack.org/ResourceMonitorAlertsandNotifications

These are similar to AWS CloudFormations and CloudWatch respectively.

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Agreeing a common set of Image Properties

[Pádraig Brady]

On 04/08/2012 02:25 AM, Justin Santa Barbara wrote:
> Thanks Padraig & Nathanael - virt-inspector is a great source of inspiration. 
>  Can we put the virt-inspector output into a glance property?  Would all the 
> clouds agree to do that?
> 
> I still would also like simpler metadata, as it just feels like too much 
> information and work required for what is a comparatively simple use case 
> ("launch the official Debian Squeeze image" either "the latest one" or "a 
> known revision").
> 
> Most importantly, I would like to see a solution to this now, rather than 
> with the next release... If we agree these attributes now, we won't have to 
> solve the unification problem in 6 months.
> 
> Padraig: Can you explain the issue with "updated_through"?  I would have 
> thought that would define a unique set of package versions (if I'm using the 
> standard repos)?  I expect that most people will either want to pick the 
> oldest image (with a particular minor vision) or the newest image (with the 
> greatest minor version), so it might be that this simply doesn't matter other 
> than to define an ordering.  It might also be that the creation date is 
> adequate here.

Well "base" + "updated_through" _may_ well define the contents.
Though I'd be worried that depending on config only some
available updates are applied (like only security updates etc.).
It might be worth recording though, especially if the "last update"
date could generally be inspected.

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Agreeing a common set of Image Properties

[Pádraig Brady]

On 04/07/2012 11:13 PM, Justin Santa Barbara wrote:
> Is there a (de-facto) standard for image metadata/properties?  I'd like to be 
> able to able to launch e.g. the Debian Squeeze image provided by the cloud.  
> This is particularly important for clouds that don't allow image upload, but 
> likely this will remain useful because different clouds will have different 
> tweaks needed (e.g installing the right drivers based on the hypervisor).
> 
> I could try "smart"-parsing the names, but it seems like metadata is the 
> right way to do this, and I see no reason why any cloud would gain any 
> advantage from not adopting a common convention.  I know some clouds have 
> started implementing their own approaches, but I don't believe anyone is 
> locked into anything.
> 
> In the interest of efficiency, I'm going to make a proposal for people to 
> attack:
> 
> 3 main pieces of metadata: os:distro, os:version_major, os:version_minor
> 
> I'm thinking of the os prefix as standing for OpenStack, not Operating 
> System.  I'd like to 'reserve' the os prefix for 'agreed' prefixes.  Maybe 
> this should be openstack.org:distro ?
> 
> os:distro would be the primary domain name of the distro, i.e. redhat.com 
> , ubuntu.com , debian.org 
> , centos.org  etc
> 
> os:version_major would be the major version of the release:
> debian.org : 6.0, 5.0, 4.0, 3.1, 3.0, 2.2, 2.1, 2.0
> ubuntu.com : 10.04, 10.10, 11.04, 11.10, 12.04
> 
> Numbers seem more machine-friendly than codenames - 6.0, not squeeze - humans 
> will probably use the image names.
> 
> os:version_minor would be the minor version of the release (because it's a 
> minor revision, hopefully we shouldn't normally have to check this, although 
> we might want to select the latest always).
> 
> So os:distro="debian.org " os:version_major "6.0" 
> os:version_minor "3" would be an image of debian 6.0.3.
> 
> Questions / ideas for other properties:
> 
>   * Some clouds will automatically respin images e.g. weekly with the latest 
> updates.  This could also be exposed through metadata.  os:updated_through= 
> "20120301" ? 
>   * Some clouds will offer only bare images, some will provide a variety e.g. 
> bare, LAMP, Hadoop, etc.  Should we use the native package names to indicate 
> additional packages?  e.g. os:packages="apache2,mysql,php5" ?
> 
> As a (programmatic) consumer of these images, my wishlist:
> 
>   * A cloud will have to put on whatever drivers / agents they need to, but 
> ideally these should otherwise be clean images, with minimal deviation from 
> the stock install.  (Or 'clean' images should be available alongside other 
> images)  It would be great if I could be launch a 'clean' image on any 
> OpenStack cloud and have it behave more or less the same; I shouldn't have to 
> second guess any additional tweaks.
>   * I would like to be able to launch the clean image and install updates 
> myself, in case I don't want a particular update.  Providing a fast apt cache 
> is much better than providing respun images, for my use-case.  I would be 
> great if old images stuck around, therefore!
> 

This overlaps a lot with the output from the virt-inspector component of 
libguestfs,
which might help solidify ideas:

$ virt-inspector /var/lib/libvirt/images/rh63.img


  
/dev/VolGroup/lv_root
linux
x86_64
rhel
Red Hat Enterprise Linux Workstation release 6.3 Beta 
(Santiago)
6
3
rpm
yum
localhost.localdomain
installed
...

  ...
  
coreutils
8.4
18
  
  ...

Note that it doesn't have an "updated_through" element, but that would be fairly
amorphous anyway given the combinations possible through updates.

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] KVM disk performance

[Pádraig Brady]

On 03/26/2012 01:59 PM, Martin van Wilderen - JDN BV wrote:
> Hi List,
> 
> I have a question about KVM disk performance. We are using Openstack Nova on 
> three machines. These machines have a SSD drive with a dd write performance 
> of about 130mb/s
> 
> Within the instance the write performance is down to about 5 mb/s. When using 
> the allocate trick (dd zero to disk before newfs) we get a performance of 20 
> mb/s.

Interesting. I presume this performance difference is when growing the 
allocation.
I.E. without the "dd allocate trick", writes to new files = 5MB/s, but old 
files = 20MB/s ?

What file did you do the `dd` on exactly and what size was it?

I could do a patch to nova to do something like:

fallocate --version || exit 0
fallocate -l $SIZE "$DISKIMG" || { rm -f "$DISKIMG"; exit 1; }

But only when I get a better understand of your setup.
Note it would need to be optional too, akin to a memory
overcommit like flag.

Note the above change would also give the benefit of
immediate feedback of ENOSPC

> 
> Things a have tried but don't give any extra results are:
> - Settings the disklayout from qcow2 to raw

I'm a bit surprised that had no effect.

> - Settings the cache type in libvirt.xml (writeback, writethrough, none)
> - Switching KSM on and off.
> - Tested with different guests OS, Linux, FreeBSD, Windows.
> 
> Is there someone who had some extra info i can check? Or are there more 
> people with this issue?
> 
> Snippet from libvirt.xml
> 
> 
> 

I'll let others more knowledgeable comment on
general virt IO overheads.

cheers,
Pádraig.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] KVM disk performance

[Pádraig Brady]

On 03/26/2012 01:59 PM, Martin van Wilderen - JDN BV wrote:
> Hi List,
> 
> I have a question about KVM disk performance. We are using Openstack Nova on 
> three machines. These machines have a SSD drive with a dd write performance 
> of about 130mb/s
> 
> Within the instance the write performance is down to about 5 mb/s. When using 
> the allocate trick (dd zero to disk before newfs) we get a performance of 20 
> mb/s.

Interesting. I presume this performance difference is when growing the 
allocation.
I.E. without the "dd allocate trick", writes to new files = 5MB/s, but old 
files = 20MB/s ?

What file did you do the `dd` on exactly and what size was it?

I could do a patch to nova to do something like:

fallocate --version || exit 0
fallocate -l $SIZE "$DISKIMG" || { rm -f "$DISKIMG"; exit 1; }

But only when I get a better understand of your setup.
Note it would need to be optional too, akin to a memory
overcommit like flag.

Note the above change would also give the benefit of
immediate feedback of ENOSPC

> Things a have tried but don't give any extra results are:
> - Settings the disklayout from qcow2 to raw

I'm a bit surprised that had no effect.

> - Settings the cache type in libvirt.xml (writeback, writethrough, none)
> - Switching KSM on and off.
> - Tested with different guests OS, Linux, FreeBSD, Windows.
> 
> Is there someone who had some extra info i can check? Or are there more 
> people with this issue?
> 
> Snippet from libvirt.xml
> 
> 
> 

I'll let others more knowledgeable comment on
general virt IO overheads.

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [NOVA] Snapshotting may require significant disk space (in /tmp). How to properly solve disk space issues?

[Pádraig Brady]

On 03/16/2012 11:57 PM, Justin Shepherd wrote:
> 
> 
> On Mar 16, 2012, at 12:26, "Pádraig Brady"  wrote:
> 
>> On 03/16/2012 04:11 PM, Jay Pipes wrote:
>>> Hi Stackers,
>>>
>>> So, in diagnosing a few things on TryStack yesterday, I ran into an 
>>> interesting problem with snapshotting that I'm hoping to get some advice on.
>>>
>>> == The Problem ==
>>>
>>
>>> QEMU was unhelpfully returning a vague error message of "error while 
>>> writing".
>>
>> That could be improved.
>> As an aside, since qemu-img is mainly dealing with large files,
>> it would be a prime candidate to call fallocate() from
>> to get good layout for the files and immediate feedback
>> if there isn't enough space.
>>
>> On a related note, I've a patch pending for after RC1
>> that should auto clean any of these partially written files:
>> https://review.openstack.org/#change,5442
>>
>>> As it turns out, the base operating system we install on our compute nodes 
>>> in TryStack has a (very) small root partition
>>
>>> == Possible Solutions ==
>>>
>>> So, there are a number of solutions that we can work on here, and I'm 
>>> wondering what the preference would be. Here are the solutions I have come 
>>> up with, along with a no-brainer improvement to Nova that would help in 
>>> diagnosing this problem:
>>>
>>> The no-brainer: Detect before attempting a snapshot that there is enough 
>>> space on a device to perform the operation, and if not, throw a useful 
>>> error message up the stack
>>
>> The space can change while writing, so you could still get the same error 
>> above.
>>
>>>
>>> Solutions to the disk space problem:
>>>
>>> (1) Silly Jay, change the damn size of the root partition in your PXE base 
>>> OS install!
>>>
>>> Now, I'm no expert in creating customized base disk images, but from 
>>> looking at the build_pxe_env.sh script in devstack [1], it seems pretty 
>>> trivial to change the ramdisk_size parameter in the startup options to 
>>> something larger than 2109600. We could do this and reimage the compute 
>>> nodes one by one.
>>>
>>> (2) Make the location in which the snapshot is made configurable.
>>>
>>> Right now, as mentioned above, tempfile.mkdtemp() is used, which creates a 
>>> directory in the user's TMPDIR (typically /tmp, which is usually on the 
>>> root partition).
>>>
>>> We could add an option (--libvirt-snapshot-dir?) that would allow 
>>> nova-compute to override where that snapshot is built.
>>>
>>> (3) Change the user (running nova-compute) TMPDIR setting to something 
>>> different than /tmp on the root partition).
>>
>> I'd lean towards (3).
>> That's something that depends on the environment (as you've nicely 
>> demonstrated),
>> and also for security reasons the admin should be able to set TMPDIR.
>> That's the standard way to do it, and it works already (hopefully).
> 
> Actually I would argue that the best way to accomplish this would be option 
> #2. That way an admin/operator has control over the location. Not 
> manipulating this by messing around with a users environment variable.

Well one can set the TMPDIR in the init script for the service.
That's a fairly standard mechanism.

(2) is good though if you would ever want to separate
--libvirt-snapshot-dir from, $TMPDIR

Now I can definitely see the need for changing TMPDIR from /tmp
for Jay's reasons and /tmp being tmpfs by default on debian for example:
http://lists.debian.org/debian-devel/2011/11/msg00281.html
I'm not sure if you'd need to separate them?
Though I'm always biased towards avoiding new config variables.
I suppose one could argue you might want /tmp for small fast accesses,
and something large and separate for manipulating large files.

Now that I look at the existing nova uses of tmp dirs
to store/stage large images, I see existing config vars:

FLAGS.xenapi_sr_base_path  # xens default Storage Repo
FLAGS.image_decryption_dir # nova/image/s3.py

So if you were following that you would implement (2) with:

FLAGS.libvirt_snapshot_dir

There might be opportunity to merge all three to:

FLAGS.nova_image_staging_dir

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [NOVA] Snapshotting may require significant disk space (in /tmp). How to properly solve disk space issues?

[Pádraig Brady]

On 03/16/2012 04:11 PM, Jay Pipes wrote:
> Hi Stackers,
> 
> So, in diagnosing a few things on TryStack yesterday, I ran into an 
> interesting problem with snapshotting that I'm hoping to get some advice on.
> 
> == The Problem ==
> 

> QEMU was unhelpfully returning a vague error message of "error while writing".

That could be improved.
As an aside, since qemu-img is mainly dealing with large files,
it would be a prime candidate to call fallocate() from
to get good layout for the files and immediate feedback
if there isn't enough space.

On a related note, I've a patch pending for after RC1
that should auto clean any of these partially written files:
https://review.openstack.org/#change,5442

> As it turns out, the base operating system we install on our compute nodes in 
> TryStack has a (very) small root partition

> == Possible Solutions ==
> 
> So, there are a number of solutions that we can work on here, and I'm 
> wondering what the preference would be. Here are the solutions I have come up 
> with, along with a no-brainer improvement to Nova that would help in 
> diagnosing this problem:
> 
> The no-brainer: Detect before attempting a snapshot that there is enough 
> space on a device to perform the operation, and if not, throw a useful error 
> message up the stack

The space can change while writing, so you could still get the same error above.

> 
> Solutions to the disk space problem:
> 
> (1) Silly Jay, change the damn size of the root partition in your PXE base OS 
> install!
> 
> Now, I'm no expert in creating customized base disk images, but from looking 
> at the build_pxe_env.sh script in devstack [1], it seems pretty trivial to 
> change the ramdisk_size parameter in the startup options to something larger 
> than 2109600. We could do this and reimage the compute nodes one by one.
> 
> (2) Make the location in which the snapshot is made configurable.
> 
> Right now, as mentioned above, tempfile.mkdtemp() is used, which creates a 
> directory in the user's TMPDIR (typically /tmp, which is usually on the root 
> partition).
> 
> We could add an option (--libvirt-snapshot-dir?) that would allow 
> nova-compute to override where that snapshot is built.
> 
> (3) Change the user (running nova-compute) TMPDIR setting to something 
> different than /tmp on the root partition).

I'd lean towards (3).
That's something that depends on the environment (as you've nicely 
demonstrated),
and also for security reasons the admin should be able to set TMPDIR.
That's the standard way to do it, and it works already (hopefully).

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Can openstack support spice ?

[Pádraig Brady]

On 03/11/2012 02:31 AM, wangsuyi640 wrote:
> Dear all:
> 
> The release of openstack on my server is D3. I tred to run 'qemu-kvm 
> -hda /root/free.img -m 512 -vga qxl -spice port=5930,disable-ticketing '
> 
> without openstack , it worked well.
> 
> However,I have tried to modify the 
> '/opt/stack/nova/nova/virt/libvirt.xml.template' in order to  let the spice 
> work with the openstack. Then it failed.
> 
> Is there anyone tried this? Could you give me some help? Thanks.

Have a look at this Work In Progress:
https://review.openstack.org/5319

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] sqlalchemy version dependencies

[Pádraig Brady]

There are a few inconsistencies currently with
the sqlalchemy version specifications in nova and glance.

tl;dr SQLAlchemy>=0.6.3 should be fine for Essex?

For e.g. the name "useexisting" was deprecated in
sqlalchemy-0.7.0 and changed to "extend_existing".
That name change was also made recently in glance:
  https://review.openstack.org/#change,5031
However glance has this inconsistent pip-requires
(there is no reason given in commit history):
  SQLAlchemy>=0.6.3,<0.7

Nova was changed from sqlalchemy version agnostic to
>=0.7.3 because of this migration bug:
  https://bugs.launchpad.net/nova/+bug/928637
But that sqlalchemy issue was not in 0.6.x,
as can be seen with this similar issue and change:
  https://bugs.launchpad.net/glance/+bug/787296
  http://bazaar.launchpad.net/~hudson-openstack/glance/trunk/revision/138

From experience one should actively try to minimize dependencies.
So it would be better to support 0.6.x if possible.
Even if there are issues with 0.7.[012] upstream,
those could have been patched locally.
So ideally we would not restrict sqlalchemy versions at all.

Pragmatically though since those sqlalchemy versions
don't seem to be commonly distributed, it might
be best to add the following to nova and glance?

  SQLAlchemy>=0.6.3,!=0.7.0,!=0.7.1,!=0.7.2¹

Also that would imply that the glance change 5031
referenced above should be reverted?

Are there other dependencies I'm not aware of?

cheers,
Pádraig.

¹ http://peak.telecommunity.com/DevCenter/setuptools#declaring-dependencies

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] OpenStack Installation Woes - Need re-assurance and help.

[Pádraig Brady]

On 03/13/2012 09:28 AM, Kevin Jackson wrote:
> Dear all,
> I had my first sleepless night last night after a conversation I had in work 
> regarding an OpenStack installation.

> Are my problems OpenStack's or Ubuntu's packaging?  I would love to speak to 
> someone who has this running and direct my questions to the right place.
> My aim is simple: I want a running OpenStack environment.

I'm not familiar with ubuntu's packaging, but to allay your
fears about openstack itself, we've recently had a successful Fedora
test day testing out various functionality of Essex milestone 4
including the new keystone.  I'm not suggesting you switch or
anything, but you might be able to copy some of the configuration
etc. from the steps detailed at:

http://fedoraproject.org/wiki/Test_Day:2012-03-08_OpenStack_Test_Day

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Invitation to work on Essex Deployments, Thursday 3/8

[Pádraig Brady]

On 03/05/2012 06:27 PM, rob_hirschf...@dell.com wrote:
> Stackers,
> 
> The Dell OpenStack team is coordinating a world-wide effort to work on Essex 
> deployments this coming Thursday, 3/8.

Coincidentally Fedora is having an OpenStack test day
on the same date (tomorrow).

http://fedoraproject.org/wiki/Test_Day:2012-03-08_OpenStack_Test_Day

Detailed there are recipes to deploy the OpenStack Essex services
on Fedora 17 (alpha), and it's a good way to quickly learn about OpenStack.
Note the instructions can be followed at any time, but
there will be more people around tomorrow to help out.

Note you don't even need a physical machine to try things out.
You could for example boot this http://goo.gl/S1hPU 64 bit live image
in a VM, and click 'install to "hard disk"', and then follow the
instructions from there (clouds all the way down :))

Hopefully between both of these initiatives we get
lots of new people trying out OpenStack!

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] cfg.py-style config files

[Pádraig Brady]

On 03/05/2012 11:29 PM, Dean Troyer wrote:
> I've got two questions regarding the configuration files used by
> openstack.common.cfg:
> 
> a) Are the option and group names intentionally case sensitive?  I've
> discovered that [default] doesn't work for example.

I'm not sure

> 
> b) Nova-specific: Is there a way to use the INI format config files
> without having to put --config-file on the command-line, i.e. skip the
> compatibility conversion on, say, /etc/nova/nova.conf.  All of the
> default config file locations appear to be assumed to be flagfiles.

I noticed that earlier today and logged:
https://bugs.launchpad.net/bugs/947549

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp