Re: [one-users] What kinds of shared storage are you using?
I've found that NFS is unacceptably slow too. With both the front end and the nodes mounting NFS, copies have to go through the front end, then back out again, which is a bit wasteful. We use a NetApp storage system, which can do flexclones. We can't take advantage of that with OpenNebula because everything is exported via NFS. I have a few coworkers that have experience with a NetApp api, so we plan on writing a tm driver for NetApp soon. I'm interested to hear about other people's setups and solutions. --Michael Brown On Wed, Sep 1, 2010 at 11:48 PM, Huang Zhiteng wrote: > Hi all, > > In my open nebula 2.0b testing, I found NFS performance was unacceptable > (too bad). I haven't done any tuning or optimization to NFS yet but I doubt > if tuning can solve the problem. So I'd like to know what kind of shared > storage you are using. I thought about Global File System v2 (GFSv2). > GFSv2 does performs much better (near native performance) but there's limit > of 32 nodes and setting up GFS is complex. So more important question, how > can shared storage scales to > 100 node cloud? Or this question should be > for > 100 node cloud, what kind of storage system should be used? Please > give any suggestion or comments. If you have already implement/deploy such > an environment, it'd be great if you can share some best practice. > > -- > Below there's some details about my setup and issue: > > 1 front-end, 6 nodes. All machines are two socket Intel Xeon x5570 2.93Ghz > (16 threads in total), with 12GB memory. There's one SATA RAID 0 box (630GB > capacity) connected to front-end. Network is 1Gb Ethernet. > > OpenNebula 2.0b was installed to /srv/cloud/one on front-end and then > exported via NFSv4. Also front-end exports RAID 0 partition to > /srv/cloud/one/var/images. > > The Prolog stage of Creating VM always caused frond-end machine almost > freeze (slow response to input, even OpenNebula command would timeout) in my > setup. I highly suspect the root cause is poor performance NFS. > -- > Regards > Huang Zhiteng > > ___ > Users mailing list > Users@lists.opennebula.org > http://lists.opennebula.org/listinfo.cgi/users-opennebula.org > > ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
[one-users] Register VM for ESX 4.1
Hello, I am trying to register my VM to use with OpenNebula and ESX 4.1. I copied the VM folder to the shared nfs storage and then have tried to create a template for this vm but I am not sure I have it right: r...@:/srv/cloud/one# cat /export/vg0-lv0/WinXP.template NAME=VMwareVM MEMORY=256 CPU=1 NIC=[NETWORK="VMWareNet"] DISK=[ source="/export/vg0-lv0/WinXP_32bit_Dev", clone="yes", save="no"] However when I run the register command I get image not present messages: r...@:/export/vg0-lv0# /srv/cloud/one/bin/oneimage register ./WinXP.template Error: Image not present, aborting. It is not quite clear to me what the template should look like and how to register an existing VMware VM. Thanks, Jon ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
Re: [one-users] Register VM for ESX 4.1
Hi Jonathan, The way to use the image catalog is to upload individual images, each of them requiring an image template [1]. One registered, a VM can use an image by just referencing the image's name (same as the VirtualNetwork from your template, using just the name "VMWareNET"). Regards, -Tino [1] http://opennebula.org/documentation:rel2.0:img_template -- Constantino Vázquez Blanco | dsa-research.org/tinova Virtualization Technology Engineer / Researcher OpenNebula Toolkit | opennebula.org On Thu, Sep 2, 2010 at 4:57 PM, Jonathan Kelley wrote: > Hello, > I am trying to register my VM to use with OpenNebula and ESX 4.1. > I copied the VM folder to the shared nfs storage and then have tried to > create a template for this vm but I am not sure I have it right: > r...@:/srv/cloud/one# cat /export/vg0-lv0/WinXP.template > NAME=VMwareVM > MEMORY=256 > CPU=1 > NIC=[NETWORK="VMWareNet"] > DISK=[ source="/export/vg0-lv0/WinXP_32bit_Dev", > clone="yes", > save="no"] > However when I run the register command I get image not present messages: > r...@:/export/vg0-lv0# /srv/cloud/one/bin/oneimage register > ./WinXP.template > Error: Image not present, aborting. > > It is not quite clear to me what the template should look like and how to > register an existing VMware VM. > Thanks, > Jon > > ___ > Users mailing list > Users@lists.opennebula.org > http://lists.opennebula.org/listinfo.cgi/users-opennebula.org > > ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
Re: [one-users] QEMU Error: could not open disk image
Hello, It looks like the problem is the NFS configuration. KVM runs as root and needs read/write acces to the disk image. It this disk image has been exported by NFS with the root_squash option (by default) the local root user of the worker node won't have the permissions it needs. You might wan't to change your /etc/exports in the frontend (or NFS server) and configure it with no_root_squash. Regards, Jaime On Wed, Sep 1, 2010 at 3:56 PM, Huang Zhiteng wrote: > Hi guys, > > I encountered a qemu error while using ONE 2.0b. Does anyone have any idea > what is wrong? > "error: internal error process exited while connecting to monitor: qemu: > could not open disk image /srv/cloud/one/var//16/images/disk.0: Permission > denied" > > My environment is: > Hardware: > Intel Xeon 2.93GHz x 2 > 12 GB memory > > Software: > RHEL 6 beta (kernel 2.6.32) > qemu-kvm-0.12.1.2 > > SELinux is disabled on all nodes/front-end. So the root cause shouldn't be > SELinux. Below is error log from ONE. > > Wed Sep 1 18:46:47 2010 [DiM][I]: New VM state is ACTIVE. > Wed Sep 1 18:46:50 2010 [LCM][I]: New VM state is PROLOG. > Wed Sep 1 18:46:50 2010 [VM][I]: Virtual Machine has no context > Wed Sep 1 18:47:40 2010 [TM][I]: tm_clone.sh: > rhel6-nhm:/srv/cloud/one/var/images/9c2fea820bf6ec3793dfb6102e5834d41c4bd481 > n5:/srv/cloud/one/var//16/images/disk.0 > Wed Sep 1 18:47:40 2010 [TM][I]: tm_clone.sh: DST: > /srv/cloud/one/var//16/images/disk.0 > Wed Sep 1 18:47:40 2010 [TM][I]: tm_clone.sh: Creating directory > /srv/cloud/one/var//16/images > Wed Sep 1 18:47:40 2010 [TM][I]: tm_clone.sh: Executed "mkdir -p > /srv/cloud/one/var//16/images". > Wed Sep 1 18:47:40 2010 [TM][I]: tm_clone.sh: Executed "chmod a+w > /srv/cloud/one/var//16/images". > Wed Sep 1 18:47:40 2010 [TM][I]: tm_clone.sh: Cloning > /srv/cloud/one/var/images/9c2fea820bf6ec3793dfb6102e5834d41c4bd481 > Wed Sep 1 18:47:40 2010 [TM][I]: tm_clone.sh: Executed "cp > /srv/cloud/one/var/images/9c2fea820bf6ec3793dfb6102e5834d41c4bd481 > /srv/cloud/one/var//16/images/disk.0". > Wed Sep 1 18:47:40 2010 [TM][I]: tm_clone.sh: Executed "chmod a+w > /srv/cloud/one/var//16/images/disk.0". > Wed Sep 1 18:47:55 2010 [LCM][I]: New VM state is BOOT > Wed Sep 1 18:47:55 2010 [VMM][I]: Generating deployment file: > /srv/cloud/one/var/16/deployment.0 > Wed Sep 1 18:47:58 2010 [VMM][I]: Command execution fail: 'mkdir -p > /srv/cloud/one/var//16/images && cat > > /srv/cloud/one/var//16/images/deployment.0 && virsh --connect > qemu+tcp:///system create /srv/cloud/one/var//16/images/deployment.0' > Wed Sep 1 18:47:58 2010 [VMM][I]: STDERR follows. > Wed Sep 1 18:47:58 2010 [VMM][I]: error: Failed to create domain from > /srv/cloud/one/var//16/images/deployment.0 > Wed Sep 1 18:47:58 2010 [VMM][I]: error: internal error process exited > while connecting to monitor: qemu: could not open disk image > /srv/cloud/one/var//16/images/disk.0: Permission denied > Wed Sep 1 18:47:58 2010 [VMM][I]: > Wed Sep 1 18:47:58 2010 [VMM][I]: ExitCode: 1 > Wed Sep 1 18:47:58 2010 [VMM][E]: Error deploying virtual machine > Wed Sep 1 18:48:02 2010 [DiM][I]: New VM state is FAILED > Wed Sep 1 18:48:03 2010 [TM][W]: Ignored: LOG - 16 tm_delete.sh: Deleting > /srv/cloud/one/var//16/images > > Wed Sep 1 18:48:03 2010 [TM][W]: Ignored: LOG - 16 tm_delete.sh: Executed > "rm -rf /srv/cloud/one/var//16/images". > > Wed Sep 1 18:48:03 2010 [TM][W]: Ignored: TRANSFER SUCCESS 16 - > > > -- > Regards > Huang Zhiteng > > ___ > Users mailing list > Users@lists.opennebula.org > http://lists.opennebula.org/listinfo.cgi/users-opennebula.org > > ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
Re: [one-users] Register VM for ESX 4.1
Hi Tino, Thanks for the quick reply. How do you specify a VM PATH though as my VM has 5-6 associated files? Here is the example given: NAME = "Ubuntu Web Development" PATH = /home/one_user/images/ubuntu_desktop.img PUBLIC= YES DESCRIPTION = "Ubuntu 10.04 desktop for Web Development students. Contains the pdf lessons and exercises as well as all the necessary programming tools and testing frameworks." For my Windows XP I have two disk files so how do I specify two files in the PATH field? Or do I specify the vmx file? Thanks again, -Jon On Thu, Sep 2, 2010 at 11:29 AM, Tino Vazquez wrote: > Hi Jonathan, > > The way to use the image catalog is to upload individual images, each > of them requiring an image template [1]. > > One registered, a VM can use an image by just referencing the image's > name (same as the VirtualNetwork from your template, using just the > name "VMWareNET"). > > Regards, > > -Tino > > [1] http://opennebula.org/documentation:rel2.0:img_template > -- > Constantino Vázquez Blanco | dsa-research.org/tinova > Virtualization Technology Engineer / Researcher > OpenNebula Toolkit | opennebula.org > > > > On Thu, Sep 2, 2010 at 4:57 PM, Jonathan Kelley > wrote: > > Hello, > > I am trying to register my VM to use with OpenNebula and ESX 4.1. > > I copied the VM folder to the shared nfs storage and then have tried to > > create a template for this vm but I am not sure I have it right: > > r...@:/srv/cloud/one# cat /export/vg0-lv0/WinXP.template > > NAME=VMwareVM > > MEMORY=256 > > CPU=1 > > NIC=[NETWORK="VMWareNet"] > > DISK=[ source="/export/vg0-lv0/WinXP_32bit_Dev", > >clone="yes", > >save="no"] > > However when I run the register command I get image not present messages: > > r...@:/export/vg0-lv0# /srv/cloud/one/bin/oneimage register > > ./WinXP.template > > Error: Image not present, aborting. > > > > It is not quite clear to me what the template should look like and how to > > register an existing VMware VM. > > Thanks, > > Jon > > > > ___ > > Users mailing list > > Users@lists.opennebula.org > > http://lists.opennebula.org/listinfo.cgi/users-opennebula.org > > > > > ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
Re: [one-users] Open Nebula unable to delete the snapshot of a VM
Hi Italo,
it looks like $VG_NAME contains "/images", but it shouldn't be a path,
it should be the name of the volume group that LVM uses to create LVs.
Try changing that to "images", by editing tm_lvmrc.
cheers,
Jaime
On Wed, Sep 1, 2010 at 5:30 PM, Italo Madalozo wrote:
> Hi,
> I have been trying Open Nebula for a couple of weeks now and so far it
> works great, one of the complex steps of building the cloud is the storage.
> After reading about LVM on the website and then from the mailing list I
> managed to get it working , VG_NAME=/images, and now I can instance many
> virtual machines to the pool within seconds or minutes without having to
> clone (wasting storage) the images or wasting bandwidth transferring from
> the front end to the nodes.
> Now the problem I am facing is that the lvm/tm_delete.sh script can't manage
> to delete the snapshots created by lvm/clone.sh. I have tried onevm $ID
> delete,shutdown and cancel as recommended in a previous thread but none of
> them worked.
> This is the log output:
> Wed Sep 1 16:20:47 2010 [InM][I]: Monitoring host 192.168.1.4 (5)
> Wed Sep 1 16:20:47 2010 [VMM][D]: Message received: LOG - 62 Driver command
> for 62 cancelled
> Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 Command
> execution fail: /srv/cloud/one/lib/tm_commands/lvm/tm_delete.sh
> 192.168.1.4:/srv/cloud/one/var/62/images
> Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 STDERR follows.
> Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 ERROR MESSAGE
> --8<--
> Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 "/images":
> Invalid path for Logical Volume
> Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 "/images/":
> Invalid path for Logical Volume
> Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 ERROR MESSAGE
> -->8--
> Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 ExitCode: 5
> Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 tm_delete.sh:
> Deleting remote LVs
> Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 tm_delete.sh:
> ERROR: Command "ssh 192.168.1.4 sudo lvremove -f $(echo /images/$(sudo lvs
> --noheadings /images|awk '{print $1}'|grep lv-one-62))" failed.
> Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 tm_delete.sh:
> ERROR: "/images": Invalid path for Logical Volume
> Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 "/images/":
> Invalid path for Logical Volume
> Wed Sep 1 16:20:47 2010 [TM][D]: Message received: TRANSFER FAILURE 62
> "/images": Invalid path for Logical Volume
> Wed Sep 1 16:20:47 2010 [TM][D]: Message received: "/images/": Invalid
> path for Logical Volume
> Wed Sep 1 16:20:48 2010 [VMM][D]: Message received: CANCEL SUCCESS 62 -
> Wed Sep 1 16:20:51 2010 [InM][D]: Host 5 successfully monitored.
>
> After checking the script tm_delete.sh I see everything is fine , so I don't
> understand why instead of /dev/images one is looking for /images/.
> Italo
> ___
> Users mailing list
> Users@lists.opennebula.org
> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>
>
___
Users mailing list
Users@lists.opennebula.org
http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
[one-users] Fwd: Register VM for ESX 4.1
Hi Tino, I think I am getting a little farther in that I was trying to use the onevm create instead of registering (which worked but not sure it helps). However it fails to create the vm and I find this message in the vmm_vmware.log: [02.09.2010 11:57:44] Failed deploying VM 1 into 12core.x.com.Reason:/export/vg0-lv0/1/deployment.0 (No such file or directory) with these messages in the oned.log: Thu Sep 2 11:57:43 2010 [DiM][D]: Deploying VM 1 Thu Sep 2 11:57:43 2010 [ReM][D]: VirtualMachinePoolInfo method invoked Thu Sep 2 11:57:43 2010 [TM][D]: Message received: TRANSFER SUCCESS 1 - Thu Sep 2 11:57:44 2010 [VMM][D]: Message received: DEPLOY FAILURE 1 Failed deploying VM in host 12core.boston.vm-labs.com. Thu Sep 2 11:57:44 2010 [TM][D]: Message received: LOG - 1 tm_delete.sh: Deleting Thu Sep 2 11:57:44 2010 [TM][D]: Message received: LOG - 1 tm_delete.sh: Executed "rm -rf /export/vg0-lv0/one-12core.x.com: /export/vg0-lv0/1/images". Thu Sep 2 11:57:44 2010 [TM][D]: Message received: TRANSFER SUCCESS 1 - Any tips with these messages? I have a single NFS mount on the ESX host as datastore1 and is a local path to the OpenNebula front-end. -Jon On Thu, Sep 2, 2010 at 11:39 AM, Jonathan Kelley wrote: > Hi Tino, > > Thanks for the quick reply. > > How do you specify a VM PATH though as my VM has 5-6 associated files? > > Here is the example given: > > NAME = "Ubuntu Web Development" > PATH = /home/one_user/images/ubuntu_desktop.img > PUBLIC= YES > DESCRIPTION = "Ubuntu 10.04 desktop for Web Development students. > Contains the pdf lessons and exercises as well as all the necessary > programming tools and testing frameworks." > > > For my Windows XP I have two disk files so how do I specify two files in > the PATH field? Or do I specify the vmx file? > > Thanks again, > -Jon > > > On Thu, Sep 2, 2010 at 11:29 AM, Tino Vazquez wrote: > >> Hi Jonathan, >> >> The way to use the image catalog is to upload individual images, each >> of them requiring an image template [1]. >> >> One registered, a VM can use an image by just referencing the image's >> name (same as the VirtualNetwork from your template, using just the >> name "VMWareNET"). >> >> Regards, >> >> -Tino >> >> [1] http://opennebula.org/documentation:rel2.0:img_template >> -- >> Constantino Vázquez Blanco | dsa-research.org/tinova >> Virtualization Technology Engineer / Researcher >> OpenNebula Toolkit | opennebula.org >> >> >> >> On Thu, Sep 2, 2010 at 4:57 PM, Jonathan Kelley >> wrote: >> > Hello, >> > I am trying to register my VM to use with OpenNebula and ESX 4.1. >> > I copied the VM folder to the shared nfs storage and then have tried to >> > create a template for this vm but I am not sure I have it right: >> > r...@:/srv/cloud/one# cat /export/vg0-lv0/WinXP.template >> > NAME=VMwareVM >> > MEMORY=256 >> > CPU=1 >> > NIC=[NETWORK="VMWareNet"] >> > DISK=[ source="/export/vg0-lv0/WinXP_32bit_Dev", >> >clone="yes", >> >save="no"] >> > However when I run the register command I get image not present >> messages: >> > r...@:/export/vg0-lv0# /srv/cloud/one/bin/oneimage register >> > ./WinXP.template >> > Error: Image not present, aborting. >> > >> > It is not quite clear to me what the template should look like and how >> to >> > register an existing VMware VM. >> > Thanks, >> > Jon >> > >> > ___ >> > Users mailing list >> > Users@lists.opennebula.org >> > http://lists.opennebula.org/listinfo.cgi/users-opennebula.org >> > >> > >> > > ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
Re: [one-users] Open Nebula unable to delete the snapshot of a VM
Hi Jaime,
thanks for the reply. /images is not a path it is the name of the VG, first
I tried /dev/images and it did not work, when I change to /images it works.
But it can not delete the snapshots afterwards.
Cheers
Italo
On 2 September 2010 15:59, Jaime Melis wrote:
> Hi Italo,
>
> it looks like $VG_NAME contains "/images", but it shouldn't be a path,
> it should be the name of the volume group that LVM uses to create LVs.
>
> Try changing that to "images", by editing tm_lvmrc.
>
> cheers,
> Jaime
>
> On Wed, Sep 1, 2010 at 5:30 PM, Italo Madalozo
> wrote:
> > Hi,
> > I have been trying Open Nebula for a couple of weeks now and so far it
> > works great, one of the complex steps of building the cloud is the
> storage.
> > After reading about LVM on the website and then from the mailing list I
> > managed to get it working , VG_NAME=/images, and now I can instance many
> > virtual machines to the pool within seconds or minutes without having to
> > clone (wasting storage) the images or wasting bandwidth transferring
> from
> > the front end to the nodes.
> > Now the problem I am facing is that the lvm/tm_delete.sh script can't
> manage
> > to delete the snapshots created by lvm/clone.sh. I have tried onevm $ID
> > delete,shutdown and cancel as recommended in a previous thread but none
> of
> > them worked.
> > This is the log output:
> > Wed Sep 1 16:20:47 2010 [InM][I]: Monitoring host 192.168.1.4 (5)
> > Wed Sep 1 16:20:47 2010 [VMM][D]: Message received: LOG - 62 Driver
> command
> > for 62 cancelled
> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 Command
> > execution fail: /srv/cloud/one/lib/tm_commands/lvm/tm_delete.sh
> > 192.168.1.4:/srv/cloud/one/var/62/images
> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 STDERR
> follows.
> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 ERROR
> MESSAGE
> > --8<--
> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 "/images":
> > Invalid path for Logical Volume
> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 "/images/":
> > Invalid path for Logical Volume
> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 ERROR
> MESSAGE
> > -->8--
> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 ExitCode: 5
> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62
> tm_delete.sh:
> > Deleting remote LVs
> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62
> tm_delete.sh:
> > ERROR: Command "ssh 192.168.1.4 sudo lvremove -f $(echo /images/$(sudo
> lvs
> > --noheadings /images|awk '{print $1}'|grep lv-one-62))" failed.
> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62
> tm_delete.sh:
> > ERROR: "/images": Invalid path for Logical Volume
> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 "/images/":
> > Invalid path for Logical Volume
> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: TRANSFER FAILURE 62
> > "/images": Invalid path for Logical Volume
> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: "/images/": Invalid
> > path for Logical Volume
> > Wed Sep 1 16:20:48 2010 [VMM][D]: Message received: CANCEL SUCCESS 62 -
> > Wed Sep 1 16:20:51 2010 [InM][D]: Host 5 successfully monitored.
> >
> > After checking the script tm_delete.sh I see everything is fine , so I
> don't
> > understand why instead of /dev/images one is looking for /images/.
> > Italo
> > ___
> > Users mailing list
> > Users@lists.opennebula.org
> > http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
> >
> >
>
___
Users mailing list
Users@lists.opennebula.org
http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
Re: [one-users] Open Nebula unable to delete the snapshot of a VM
Hi Italo,
that is weird. Can you paste the output of 'lvs' ?
regards,
Jaime
On Thu, Sep 2, 2010 at 6:16 PM, Italo Madalozo wrote:
> Hi Jaime,
> thanks for the reply. /images is not a path it is the name of the VG, first
> I tried /dev/images and it did not work, when I change to /images it works.
> But it can not delete the snapshots afterwards.
> Cheers
> Italo
>
> On 2 September 2010 15:59, Jaime Melis wrote:
>>
>> Hi Italo,
>>
>> it looks like $VG_NAME contains "/images", but it shouldn't be a path,
>> it should be the name of the volume group that LVM uses to create LVs.
>>
>> Try changing that to "images", by editing tm_lvmrc.
>>
>> cheers,
>> Jaime
>>
>> On Wed, Sep 1, 2010 at 5:30 PM, Italo Madalozo
>> wrote:
>> > Hi,
>> > I have been trying Open Nebula for a couple of weeks now and so far it
>> > works great, one of the complex steps of building the cloud is the
>> > storage.
>> > After reading about LVM on the website and then from the mailing list I
>> > managed to get it working , VG_NAME=/images, and now I can instance many
>> > virtual machines to the pool within seconds or minutes without having
>> > to
>> > clone (wasting storage) the images or wasting bandwidth transferring
>> > from
>> > the front end to the nodes.
>> > Now the problem I am facing is that the lvm/tm_delete.sh script can't
>> > manage
>> > to delete the snapshots created by lvm/clone.sh. I have tried onevm $ID
>> > delete,shutdown and cancel as recommended in a previous thread but none
>> > of
>> > them worked.
>> > This is the log output:
>> > Wed Sep 1 16:20:47 2010 [InM][I]: Monitoring host 192.168.1.4 (5)
>> > Wed Sep 1 16:20:47 2010 [VMM][D]: Message received: LOG - 62 Driver
>> > command
>> > for 62 cancelled
>> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 Command
>> > execution fail: /srv/cloud/one/lib/tm_commands/lvm/tm_delete.sh
>> > 192.168.1.4:/srv/cloud/one/var/62/images
>> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 STDERR
>> > follows.
>> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 ERROR
>> > MESSAGE
>> > --8<--
>> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 "/images":
>> > Invalid path for Logical Volume
>> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 "/images/":
>> > Invalid path for Logical Volume
>> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 ERROR
>> > MESSAGE
>> > -->8--
>> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 ExitCode: 5
>> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62
>> > tm_delete.sh:
>> > Deleting remote LVs
>> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62
>> > tm_delete.sh:
>> > ERROR: Command "ssh 192.168.1.4 sudo lvremove -f $(echo /images/$(sudo
>> > lvs
>> > --noheadings /images|awk '{print $1}'|grep lv-one-62))" failed.
>> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62
>> > tm_delete.sh:
>> > ERROR: "/images": Invalid path for Logical Volume
>> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 "/images/":
>> > Invalid path for Logical Volume
>> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: TRANSFER FAILURE 62
>> > "/images": Invalid path for Logical Volume
>> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: "/images/":
>> > Invalid
>> > path for Logical Volume
>> > Wed Sep 1 16:20:48 2010 [VMM][D]: Message received: CANCEL SUCCESS 62 -
>> > Wed Sep 1 16:20:51 2010 [InM][D]: Host 5 successfully monitored.
>> >
>> > After checking the script tm_delete.sh I see everything is fine , so I
>> > don't
>> > understand why instead of /dev/images one is looking for /images/.
>> > Italo
>> > ___
>> > Users mailing list
>> > Users@lists.opennebula.org
>> > http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>> >
>> >
>
>
___
Users mailing list
Users@lists.opennebula.org
http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
Re: [one-users] Open Nebula unable to delete the snapshot of a VM
Can you also paste the output of 'vgs' ?
On Thu, Sep 2, 2010 at 6:23 PM, Jaime Melis wrote:
> Hi Italo,
>
> that is weird. Can you paste the output of 'lvs' ?
>
> regards,
> Jaime
>
> On Thu, Sep 2, 2010 at 6:16 PM, Italo Madalozo
> wrote:
>> Hi Jaime,
>> thanks for the reply. /images is not a path it is the name of the VG, first
>> I tried /dev/images and it did not work, when I change to /images it works.
>> But it can not delete the snapshots afterwards.
>> Cheers
>> Italo
>>
>> On 2 September 2010 15:59, Jaime Melis wrote:
>>>
>>> Hi Italo,
>>>
>>> it looks like $VG_NAME contains "/images", but it shouldn't be a path,
>>> it should be the name of the volume group that LVM uses to create LVs.
>>>
>>> Try changing that to "images", by editing tm_lvmrc.
>>>
>>> cheers,
>>> Jaime
>>>
>>> On Wed, Sep 1, 2010 at 5:30 PM, Italo Madalozo
>>> wrote:
>>> > Hi,
>>> > I have been trying Open Nebula for a couple of weeks now and so far it
>>> > works great, one of the complex steps of building the cloud is the
>>> > storage.
>>> > After reading about LVM on the website and then from the mailing list I
>>> > managed to get it working , VG_NAME=/images, and now I can instance many
>>> > virtual machines to the pool within seconds or minutes without having
>>> > to
>>> > clone (wasting storage) the images or wasting bandwidth transferring
>>> > from
>>> > the front end to the nodes.
>>> > Now the problem I am facing is that the lvm/tm_delete.sh script can't
>>> > manage
>>> > to delete the snapshots created by lvm/clone.sh. I have tried onevm $ID
>>> > delete,shutdown and cancel as recommended in a previous thread but none
>>> > of
>>> > them worked.
>>> > This is the log output:
>>> > Wed Sep 1 16:20:47 2010 [InM][I]: Monitoring host 192.168.1.4 (5)
>>> > Wed Sep 1 16:20:47 2010 [VMM][D]: Message received: LOG - 62 Driver
>>> > command
>>> > for 62 cancelled
>>> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 Command
>>> > execution fail: /srv/cloud/one/lib/tm_commands/lvm/tm_delete.sh
>>> > 192.168.1.4:/srv/cloud/one/var/62/images
>>> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 STDERR
>>> > follows.
>>> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 ERROR
>>> > MESSAGE
>>> > --8<--
>>> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 "/images":
>>> > Invalid path for Logical Volume
>>> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 "/images/":
>>> > Invalid path for Logical Volume
>>> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 ERROR
>>> > MESSAGE
>>> > -->8--
>>> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 ExitCode: 5
>>> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62
>>> > tm_delete.sh:
>>> > Deleting remote LVs
>>> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62
>>> > tm_delete.sh:
>>> > ERROR: Command "ssh 192.168.1.4 sudo lvremove -f $(echo /images/$(sudo
>>> > lvs
>>> > --noheadings /images|awk '{print $1}'|grep lv-one-62))" failed.
>>> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62
>>> > tm_delete.sh:
>>> > ERROR: "/images": Invalid path for Logical Volume
>>> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: LOG - 62 "/images/":
>>> > Invalid path for Logical Volume
>>> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: TRANSFER FAILURE 62
>>> > "/images": Invalid path for Logical Volume
>>> > Wed Sep 1 16:20:47 2010 [TM][D]: Message received: "/images/":
>>> > Invalid
>>> > path for Logical Volume
>>> > Wed Sep 1 16:20:48 2010 [VMM][D]: Message received: CANCEL SUCCESS 62 -
>>> > Wed Sep 1 16:20:51 2010 [InM][D]: Host 5 successfully monitored.
>>> >
>>> > After checking the script tm_delete.sh I see everything is fine , so I
>>> > don't
>>> > understand why instead of /dev/images one is looking for /images/.
>>> > Italo
>>> > ___
>>> > Users mailing list
>>> > Users@lists.opennebula.org
>>> > http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>>> >
>>> >
>>
>>
>
___
Users mailing list
Users@lists.opennebula.org
http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
Re: [one-users] What kinds of shared storage are you using?
Hello, Even if NFS with default configuration is not the most performant shared filesystem we thought it was the most common shared filesystem people could use for virtualization. Maybe you can make it faster adding some parameters when mounting the shared filesystem. "async" will make your VM's run faster as it wont need to synchronously write to the server. Tunning "rsize" and "wsize" will make a difference in performance also. Anyway, I think we made a mistake calling shared filesystem drivers "tm_nfs" as it could be used with other shared filesystem. It will only asume that files in a certain path will be accesible by frontend and nodes using standard fs commands. I encourage you to use other filesystems more performant with "tm_nfs" drivers. In our machines we have been using a hybrid system (described in http://opennebula.org/documentation:rel1.4:sm at "Customizing and Extending" section). This will let you have non cloned images that can livemigrate and local images that will perform better. I also invite you to develop new drivers for other systems if they need changes. I hope that having the transfer commands in shell scripts outside the core will make it easier to change or develop new ones. If you have any doubt or need help to create those new drivers contact us as we are also interested on interacting with other technologies. Bye On Thu, Sep 2, 2010 at 2:03 PM, Michael Brown wrote: > I've found that NFS is unacceptably slow too. With both the front end and > the nodes mounting NFS, copies have to go through the front end, then back > out again, which is a bit wasteful. > > We use a NetApp storage system, which can do flexclones. We can't take > advantage of that with OpenNebula because everything is exported via NFS. I > have a few coworkers that have experience with a NetApp api, so we plan on > writing a tm driver for NetApp soon. > > I'm interested to hear about other people's setups and solutions. > > --Michael Brown > > On Wed, Sep 1, 2010 at 11:48 PM, Huang Zhiteng wrote: >> >> Hi all, >> >> In my open nebula 2.0b testing, I found NFS performance was unacceptable >> (too bad). I haven't done any tuning or optimization to NFS yet but I doubt >> if tuning can solve the problem. So I'd like to know what kind of shared >> storage you are using. I thought about Global File System v2 (GFSv2). >> GFSv2 does performs much better (near native performance) but there's limit >> of 32 nodes and setting up GFS is complex. So more important question, how >> can shared storage scales to > 100 node cloud? Or this question should be >> for > 100 node cloud, what kind of storage system should be used? Please >> give any suggestion or comments. If you have already implement/deploy such >> an environment, it'd be great if you can share some best practice. >> >> -- >> Below there's some details about my setup and issue: >> >> 1 front-end, 6 nodes. All machines are two socket Intel Xeon x5570 >> 2.93Ghz (16 threads in total), with 12GB memory. There's one SATA RAID 0 >> box (630GB capacity) connected to front-end. Network is 1Gb Ethernet. >> >> OpenNebula 2.0b was installed to /srv/cloud/one on front-end and then >> exported via NFSv4. Also front-end exports RAID 0 partition to >> /srv/cloud/one/var/images. >> >> The Prolog stage of Creating VM always caused frond-end machine almost >> freeze (slow response to input, even OpenNebula command would timeout) in my >> setup. I highly suspect the root cause is poor performance NFS. >> -- >> Regards >> Huang Zhiteng >> >> ___ >> Users mailing list >> Users@lists.opennebula.org >> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org >> > > > ___ > Users mailing list > Users@lists.opennebula.org > http://lists.opennebula.org/listinfo.cgi/users-opennebula.org > > -- Javier Fontan, Grid & Virtualization Technology Engineer/Researcher DSA Research Group: http://dsa-research.org Globus GridWay Metascheduler: http://www.GridWay.org OpenNebula Virtual Infrastructure Engine: http://www.OpenNebula.org ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
[one-users] Suggestions sought for private cloud architecture using a central storage server and iSCSI
Hello list, I am new to OpenNebula and in the process of evaluating and testing it for use at our main site. We are currently overhauling our old infrastructure and got some new network hardware and servers. I am looking for suggestions and input on if and how well our hardware will suite a private cloud setup and how the architecture should look like. A bit of info: We have two powerful HP ProLiant DL380 G6 with 36GB RAM each but little local storage (only about 140GB). These would be the cluster nodes. An older but well working DL320 G5 with only 2GB RAM and also little local storage would be well suited for the front-end server. We also have a storage server (DL180 G6) with about 2TB capacity using fast SAS disks. iSCSI would be our preferred SAN protocol. The question is basically: What's a recommended way to integrate a central storage server into a new OpenNebula private cloud setup? Any suggestions are welcome! Looking for experiences, anecdotes and RTFM pointers. Thanks in advance! Andreas -- Andreas Ntaflos Vienna, Austria GPG Fingerprint: 6234 2E8E 5C81 C6CB E5EC 7E65 397C E2A8 090C A9B4 signature.asc Description: This is a digitally signed message part. ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
Re: [one-users] What kinds of shared storage are you using?
On Thursday 02 September 2010 14:03:32 Michael Brown wrote: > I've found that NFS is unacceptably slow too. With both the front end and > the nodes mounting NFS, copies have to go through the front end, then back > out again, which is a bit wasteful. > > We use a NetApp storage system, which can do flexclones. We can't take > advantage of that with OpenNebula because everything is exported via NFS. > I have a few coworkers that have experience with a NetApp api, so we plan > on writing a tm driver for NetApp soon. > > I'm interested to hear about other people's setups and solutions. We're using iSCSI targets directly (one target per vm), automatically created and initialized (cloned) from images on vm deploy. Althogh the target is based on IET behind gigabit links, it works quite well: we haven't done performance benchmarks (yet), but the installation time of virtual machines is kinda same than on real hardware. We developed a custom TM driver for this, because this approach makes live migration trickier, since just before live migration the target host needs to log in to the iSCSI target hosting the disks of the vm, and this is something ONE can't do, so we used libvirt hooks to do this -- works like a charm. Libvirt hooks are also good for reattaching virtual machines to their virtual networks on live migration -- again something ONE doesn't do. Cheers, -- cc ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
Re: [one-users] ttylinux via ssh not accessible any more after migrate / suspend -> resume
hello javier, one more time, I'm now working with OpenNebula 2.0beta with the KVM Hypervisor (head node and cluster nodes are based on 10.04). I tried to play with the supported ttylinux image form the ONE website. Starting the machine and login via ssh is no problem. After Stop -> Resume the machine is still available via network. But if I perform a migration of the the machine or suspend -> resume, the network conncetion is gone (ping / ssh find no aim). I looked via VNC into the VM, and everything looks ok, the eth0 device is still configured right, but also no possibility to ping something others outside the VM. Restarting of the network did not solve the problem. The log files are clear. All physical hosts are in the same switch. Does anyone have an idee what goes wrong?? Greets Viktor Am 30.08.2010 11:32, schrieb Javier Fontan: Hello, Can you try to access it by other means to check if there is something broken in the VM? If you are using xen you should be able to access it using "xm console" from the physical node. With KVM you can add VNC access to it. If the machine seems to be in good shape there can also be some problem in the network relearning where the machine is. Are both physical hosts in the same switch? Bye On Mon, Aug 30, 2010 at 12:56 AM, Viktor Mauch wrote: Hello, I use ONE 1.4 with a shared NFS and play a litle bit with the ttylinux image which automatically configures the eth0 device during booting. The VM is accessible via SSH and everything looks fine. After livemigration, normal migration or suspend->resume the ssh connection is no longer available: $ ssh r...@vm_ip ssh: connect to host VM_IP port 22: No route to host Greets Viktor ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
[one-users] ONE2.0beta, livemigration does not work
hello together, I work with ONE 2.0beta based on ubuntu 10.04 with a NFS shared mount. I try to livemigrate the supported ttylinux image and get the following log output. The VM stays on the origin cluster node. Thu Sep 2 23:38:36 2010 [VMM][I]: Command execution fail: virsh --connect qemu:///system migrate --live one-20 qemu+ssh://xxx.xxx.xxx.xxx/system Thu Sep 2 23:38:36 2010 [VMM][I]: STDERR follows. Thu Sep 2 23:38:36 2010 [VMM][I]: error: Unknown failure Does anyone have an idea, what goes wrong. Greets Viktor ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
Re: [one-users] What kinds of shared storage are you using?
On Friday 03 September 2010 00:28:23 Székelyi Szabolcs wrote: > We're using iSCSI targets directly (one target per vm), automatically > created and initialized (cloned) from images on vm deploy. Althogh > the target is based on IET behind gigabit links, it works quite > well: we haven't done performance benchmarks (yet), but the > installation time of virtual machines is kinda same than on real > hardware. That sounds interesting and very similar to what we hope to achieve using OpenNebula and a central storage server (as I've posted a few hours ago, not realising this thread here is very similar in nature). > We developed a custom TM driver for this, because this approach makes > live migration trickier, since just before live migration the target > host needs to log in to the iSCSI target hosting the disks of the > vm, and this is something ONE can't do, so we used libvirt hooks to > do this -- works like a charm. Libvirt hooks are also good for > reattaching virtual machines to their virtual networks on live > migration -- again something ONE doesn't do. Would you care to go into a little detail regarding your custom TM driver? Maybe even post the sources? I'd be very interested in learning more about your approach to this. Thanks! Andreas -- Andreas Ntaflos GPG Fingerprint: 6234 2E8E 5C81 C6CB E5EC 7E65 397C E2A8 090C A9B4 signature.asc Description: This is a digitally signed message part. ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
