Re: [Ganglia-general] Problem displaying Virtual Machine data with hsflowd and ganglia 3.2.0 in an Openstack Compute node.
On Aug 31, 2011, at 8:15 AM, Emanuele Verga wrote: Hi Neil, thanks a lot for the help! I verified libvirt version, it's 0.8.8 I've dowloaded and compiled hsflowd revision 227: now ganglia correctly receives and processes statistics VM Bytes Written and VM Writes. (http://imageshack.us/f/846/vmstats.png/) Other disk statistics for VM ( VM Bytes Read, VM Disk Errors, Free Vdisk Space, VM Reads, Total Vdisk Space) are not displayed, and hsflowd -dd displays errors similar to the following: libvir: QEMU error : invalid argument in invalid path vda not assigned to domain virDomainGetBlockInfo(vda) failed Oh, sorry. virDomainetBlockInfo needs the path, not the deviceName. I've made the (1-line) change and checked it in. Please svn update and try again. If this works you should start to see the capacity, allocation and available fields. I'm not sure whether disk errors are going to show up or not. It depends on how libvirt implements the virDomainBlockStats() call for KVM. By the way, if you don't want it to even attempt that other call that always fails to find the volume, you can add this somewhere in the src/Linux/Makefile: CFLAGS += -DHSP_VRT_USE_DISKPATH But I think we might change it so that the error message only appears once in future (don't want to fill the logs), so that might be just as good. Neil I've logged a few minutes of hsflowd activity, if it can help you you can download it here: http://www.mediafire.com/?g4jac7dm3mmb662 2011/8/29 Neil Mckee neil.mckee...@gmail.com Sorry, the failure of virStorageLookupByPath() was preventing virDomainBlockStats() from being attempted. I checked in a fix for this, and also code to try the newer virDomainGetBlockInfo() call as a fallback should virStorageLookupByPath() fail. This call only came in with libvirt version 0.8.1. Are you running something newer than that? (see /usr/include/libvirt/libvirt.h) If this works, we should make a new release of hsflowd, so please let me know how it goes. Regards, Neil On Aug 29, 2011, at 7:24 AM, Emanuele Verga wrote: Hi, I downloaded and installe hsflowd trunk revision 226 but using hsflowd I keep seeing virStorageLookupByPath errors, and VM disk statistics aren't displayed. Do I need to tell hsflowd explicitly to use target=vda call? If yes, how? Thanks in advance, Emanuele 2011/8/25 Emanuele Verga verga.emanu...@gmail.com Hi Neil, Yes that's possible, the problem is Nova places each image in a separate folder (/var/lib/nova/instance/INSTANCENAME/), so we would have to create a new pool with the corresponding path each time a new instance is created, and if we start to add more servers it quicly becomes impractical. I've not yet been able to try the hsflowd version you suggested, I'll test it tomorrow and let you know. Thanks for the help! Emanuele -- Special Offer -- Download ArcSight Logger for FREE! Finally, a world-class log management solution at an even better price-free! And you'll get a free Love Thy Logs t-shirt when you download Logger. Secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsisghtdev2dev___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
Re: [Ganglia-general] Problem displaying Virtual Machine data with hsflowd and ganglia 3.2.0 in an Openstack Compute node.
Hi Neil, thanks a lot for the help! I verified libvirt version, it's 0.8.8 I've dowloaded and compiled hsflowd revision 227: now ganglia correctly receives and processes statistics VM Bytes Written and VM Writes. (http://imageshack.us/f/846/vmstats.png/) Other disk statistics for VM ( VM Bytes Read, VM Disk Errors, Free Vdisk Space, VM Reads, Total Vdisk Space) are not displayed, and hsflowd -dd displays errors similar to the following: libvir: QEMU error : invalid argument in invalid path vda not assigned to domain virDomainGetBlockInfo(vda) failed I've logged a few minutes of hsflowd activity, if it can help you you can download it here: http://www.mediafire.com/?g4jac7dm3mmb662 2011/8/29 Neil Mckee neil.mckee...@gmail.com Sorry, the failure of virStorageLookupByPath() was preventing virDomainBlockStats() from being attempted. I checked in a fix for this, and also code to try the newer virDomainGetBlockInfo() call as a fallback should virStorageLookupByPath() fail. This call only came in with libvirt version 0.8.1. Are you running something newer than that? (see /usr/include/libvirt/libvirt.h) If this works, we should make a new release of hsflowd, so please let me know how it goes. Regards, Neil On Aug 29, 2011, at 7:24 AM, Emanuele Verga wrote: Hi, I downloaded and installe hsflowd trunk revision 226 but using hsflowd I keep seeing virStorageLookupByPath errors, and VM disk statistics aren't displayed. Do I need to tell hsflowd explicitly to use target=vda call? If yes, how? Thanks in advance, Emanuele 2011/8/25 Emanuele Verga verga.emanu...@gmail.com Hi Neil, Yes that's possible, the problem is Nova places each image in a separate folder (/var/lib/nova/instance/INSTANCENAME/), so we would have to create a new pool with the corresponding path each time a new instance is created, and if we start to add more servers it quicly becomes impractical. I've not yet been able to try the hsflowd version you suggested, I'll test it tomorrow and let you know. Thanks for the help! Emanuele -- Special Offer -- Download ArcSight Logger for FREE! Finally, a world-class log management solution at an even better price-free! And you'll get a free Love Thy Logs t-shirt when you download Logger. Secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsisghtdev2dev___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
Re: [Ganglia-general] Problem displaying Virtual Machine data with hsflowd and ganglia 3.2.0 in an Openstack Compute node.
Hi, I downloaded and installe hsflowd trunk revision 226 but using hsflowd I keep seeing virStorageLookupByPath errors, and VM disk statistics aren't displayed. Do I need to tell hsflowd explicitly to use target=vda call? If yes, how? Thanks in advance, Emanuele 2011/8/25 Emanuele Verga verga.emanu...@gmail.com Hi Neil, Yes that's possible, the problem is Nova places each image in a separate folder (/var/lib/nova/instance/INSTANCENAME/), so we would have to create a new pool with the corresponding path each time a new instance is created, and if we start to add more servers it quicly becomes impractical. I've not yet been able to try the hsflowd version you suggested, I'll test it tomorrow and let you know. Thanks for the help! Emanuele -- EMC VNX: the world's simplest storage, starting under $10K The only unified storage solution that offers unified management Up to 160% more powerful than alternatives and 25% more efficient. Guaranteed. http://p.sf.net/sfu/emc-vnx-dev2dev___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
Re: [Ganglia-general] Problem displaying Virtual Machine data with hsflowd and ganglia 3.2.0 in an Openstack Compute node.
Sorry, the failure of virStorageLookupByPath() was preventing virDomainBlockStats() from being attempted. I checked in a fix for this, and also code to try the newer virDomainGetBlockInfo() call as a fallback should virStorageLookupByPath() fail. This call only came in with libvirt version 0.8.1. Are you running something newer than that? (see /usr/include/libvirt/libvirt.h) If this works, we should make a new release of hsflowd, so please let me know how it goes. Regards, Neil On Aug 29, 2011, at 7:24 AM, Emanuele Verga wrote: Hi, I downloaded and installe hsflowd trunk revision 226 but using hsflowd I keep seeing virStorageLookupByPath errors, and VM disk statistics aren't displayed. Do I need to tell hsflowd explicitly to use target=vda call? If yes, how? Thanks in advance, Emanuele 2011/8/25 Emanuele Verga verga.emanu...@gmail.com Hi Neil, Yes that's possible, the problem is Nova places each image in a separate folder (/var/lib/nova/instance/INSTANCENAME/), so we would have to create a new pool with the corresponding path each time a new instance is created, and if we start to add more servers it quicly becomes impractical. I've not yet been able to try the hsflowd version you suggested, I'll test it tomorrow and let you know. Thanks for the help! Emanuele -- Special Offer -- Download ArcSight Logger for FREE! Finally, a world-class log management solution at an even better price-free! And you'll get a free Love Thy Logs t-shirt when you download Logger. Secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsisghtdev2dev___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
Re: [Ganglia-general] Problem displaying Virtual Machine data with hsflowd and ganglia 3.2.0 in an Openstack Compute node.
Hi Neil, Yes that's possible, the problem is Nova places each image in a separate folder (/var/lib/nova/instance/INSTANCENAME/), so we would have to create a new pool with the corresponding path each time a new instance is created, and if we start to add more servers it quicly becomes impractical. I've not yet been able to try the hsflowd version you suggested, I'll test it tomorrow and let you know. Thanks for the help! Emanuele -- EMC VNX: the world's simplest storage, starting under $10K The only unified storage solution that offers unified management Up to 160% more powerful than alternatives and 25% more efficient. Guaranteed. http://p.sf.net/sfu/emc-vnx-dev2dev___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
Re: [Ganglia-general] Problem displaying Virtual Machine data with hsflowd and ganglia 3.2.0 in an Openstack Compute node.
On Aug 18, 2011, at 1:35 AM, Emanuele Verga wrote: Ok, I tried linking one of the disk files to the default storage pool folder and it actually detected the linked volume in libvirt: After issuing a virsh pool-refresh default the disk was correctly detected and reported as a volume by virsh vol-list default, but there is a problem: The disk is added as a volume in libvirth with the path parameter corresponding to the soft link ( for example: /var/lib/libvirt/images/instance.002d_disk) instead of the disk path ( /var/lib/nova/instances/instance-002d/disk ), this means that disk lookups performed by hsflowd still fail, because they try to retrieve volume information associated to disk path. (example: virStorageLookupByPath(/var/lib/nova/instances/instance-002d/disk) ). Hsflowd gets path information from the Virtual Machine XML definition (virsh dumpxml instance.002d shows, among many other details, the following line: source file='/var/lib/nova/instances/instance-002d/disk'/), that is generated and stored automatically by openstack into libvirt.xml (ex: /var/lib/nova/instances/instance-002d/libvirt.xml). So, to make it work this way, we should have a way to tell Nova/Openstack which path to look into to retrieve VM disks, and to create the related soft links into the appropriate folder, when provisioning a new instance. I also tried using virt.manager to create the pool but it didn't work, the pool was created but no disk was detected, i suppose because libvirt expect volumes to be located right into the folder specified as pool, and doesn't look in any subfolder (creating the pool manually didn't work for that reason). So, was it not possible to specify the pool directory to be the one where the disk image was actually residing? That worked for me when I tried it here. I have a disk image: /root/not_libvirt_images/test-pool2.img and I was able to add it to a new storage pool called alternative using virt-manager. So in virsh I can ask for pool-dumpxml alternative, like this: virsh # pool-dumpxml alternative pool type='dir' namealternative/name uuid51341d87-d87e-f7ce-6cc0-81ac3967c182/uuid capacity233620566016/capacity allocation37986648064/allocation available195633917952/available source /source target path/root/not_libvirt_images/path permissions mode0700/mode owner0/owner group0/group /permissions /target /pool Something I noticed that may be important is that virt-manager is able to display disk statistics for those VM. I don't really know how it gets those informations, but I believe it accesses disks using the details contained into disk /disk tags located into the VM XML definition, instead of doing a volume lookup starting from the volume path, like hsflowd is trying to do. Example: disk type='file' device='disk' driver name='qemu' type='qcow2'/ source file='/var/lib/nova/instances/INSTANCENAME/disk'/ target dev='vda' bus='virtio'/ alias name='virtio-disk0'/ address type='pci' domain='0x' bus='0x00' slot='0x04' function='0x0'/ /disk Could this be used in some way? There is a libvirt call that takes the target=vda device name and returns reads/writes counters. This was just added to hsflowd. However it's not released yet, so you'd have to check out the trunk using subversion and build from that: svn co https://host-sflow.svn.sourceforge.net/svnroot/host-sflow/trunk host-sflow-trunk I'm not sure if the same handle can be used to retrieve the capacity, allocation and available numbers, though. (If there is a libvirt expert on the list, please jump in and set us straight.) Neil -- Get a FREE DOWNLOAD! and learn more about uberSVN rich system, user administration capabilities and model configuration. Take the hassle out of deploying and managing Subversion and the tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2 ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
Re: [Ganglia-general] Problem displaying Virtual Machine data with hsflowd and ganglia 3.2.0 in an Openstack Compute node.
Ok, I tried linking one of the disk files to the default storage pool folder and it actually detected the linked volume in libvirt: After issuing a *virsh pool-refresh default *the disk was correctly detected and reported as a volume by *virsh vol-list default, *but there is a problem: The disk is added as a volume in libvirth with the path parameter corresponding to the soft link ( for example: /var/lib/libvirt/images/instance.002d_disk) instead of the disk path ( /var/lib/nova/instances/instance-002d/disk ), this means that disk lookups performed by hsflowd still fail, because they try to retrieve volume information associated to disk path. (example: virStorageLookupByPath(/var/lib/nova/instances/instance-002d/disk) ). Hsflowd gets path information from the Virtual Machine XML definition (*virsh dumpxml **instance.002d *shows, among many other details, the following line: source file='/var/lib/nova/instances/instance-002d/disk'/), that is generated and stored automatically by openstack into libvirt.xml (ex: /var/lib/nova/instances/instance-002d/libvirt.xml). So, to make it work this way, we should have a way to tell Nova/Openstack which path to look into to retrieve VM disks, and to create the related soft links into the appropriate folder, when provisioning a new instance. I also tried using virt.manager to create the pool but it didn't work, the pool was created but no disk was detected, i suppose because libvirt expect volumes to be located right into the folder specified as pool, and doesn't look in any subfolder (creating the pool manually didn't work for that reason). Something I noticed that may be important is that virt-manager is able to display disk statistics for those VM. I don't really know how it gets those informations, but I believe it accesses disks using the details contained into disk /disk tags located into the VM XML definition, instead of doing a volume lookup starting from the volume path, like hsflowd is trying to do. Example: disk type='file' device='disk' driver name='qemu' type='qcow2'/ source file='/var/lib/nova/instances/INSTANCENAME/disk'/ target dev='vda' bus='virtio'/ alias name='virtio-disk0'/ address type='pci' domain='0x' bus='0x00' slot='0x04' function='0x0'/ /disk Could this be used in some way? -- Get a FREE DOWNLOAD! and learn more about uberSVN rich system, user administration capabilities and model configuration. Take the hassle out of deploying and managing Subversion and the tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
Re: [Ganglia-general] Problem displaying Virtual Machine data with hsflowd and ganglia 3.2.0 in an Openstack Compute node.
Hi Neil, thanks for the suggestion. I tried to do it. After recompiling hsflowd i checked the files it had open and it showed: hsflowd 30019 nobody mem REG 251,3 84728 927248 /usr/lib/libxenctrl.so.3.2.0 hsflowd 30019 nobody mem REG 251,3 22928 927250 /usr/lib/libxenstore.so.3.0.0 but this solution didn't work. Virtual machines stopped showing up in ganglia and hsflowd gave this error: ERROR Internal error: Could not obtain handle on privileged command interface (2 = No such file or directory) xc_interface_open() failed : No such file or directory then it continued to work normally (hsflowd log file: http://uploading.com/files/get/a3876c8b/ ) but it didn't report anything to ganglia and the VMS were shown as down, the reporting for the physical host instead it's working perfectly, as before. If it can help you, the implementation of Openstack we have uses KVM to virtualize the hosts. Could it be related? Thanks again, Emanuele 2011/8/16 Neil Mckee neil.mckee...@gmail.com Hello, On an OpenStack node you may be able to use libxenstore instead of libvirt. You'll need to recompile hsflowd to try this. Looking at trunk/src/Linux/Makefile it appears to look for libvirt first, but you can override that by compiling hsflowd like this: make clean make LIBVIRT=no The Makefile will then test for libxenstore and libxenctrl. If it finds them it will compile with -DHSF_XEN (instead of -DHSF_VRT), and you may get better results. Please let me know what happens. Neil On Aug 16, 2011, at 2:14 AM, Emanuele Verga wrote: Hi, we have a problem with the following installation: we have a system that’s a compute node in an Openstack test installation. Now on this machine we decided to install Ganglia, to check it’s monitoring capabilities regarding virtual machines hosted on that node by Openstack. We then proceeded to add the repository for ganglia version 3.2 and install: Hsflowd 1.18 Ganglia Monitor Demon 3.2.0.0 Ganglia Meta demon 3.2.0.0 Ganglia Web FronEnd3.2.0.0 Dwoo1.1.1 all on the same machine and to configure ports accordingly. All said and done, the web frontend shown the physical host and all of the VM, but we were unable to: See the hypervisor section in the physical host statistics. It simply is not there. See the graphical preview for VM statistics ( http://imageshack.us/photo/my-images/585/screenshotgangliacomput.png/). The thumbnail are missing but the links do work, clicking on one of the missing thumbnails you are taken to the details page for that VM. See details regarding VM hard disk and I/O. ( http://imageshack.us/photo/my-images/694/screenshotgangliainstan.png/) Debugging hsflowd we found errors similar to the following: Aug 16 06:26:57 eta hsflowd: virStorageLookupByPath(/var/lib/nova/instances/instance-004d/disk.local) failed Aug 16 06:26:57 eta hsflowd: virStorageLookupByPath(/var/lib/nova/instances/instance-004d/disk) failed The strange thing is, the path is correct. We checked libvirt using virsh and found that the Storage Volumes are not reported by libvirt, it seems because libvirt by default search informations in path /var/lib/libvirt/images, instead nova places them inside /var/lib/nova/instances/INSTANCENAME/. Did you have the same problems when testing for Sflow/ Openstack ? How did you manage to resolve it? Any help is appreciated. Thanks in advance for your support! -- uberSVN's rich system and user administration capabilities and model configuration take the hassle out of deploying and managing Subversion and the tools developers use with it. Learn more about uberSVN and get a free download at: http://p.sf.net/sfu/wandisco-dev2dev ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general -- Get a FREE DOWNLOAD! and learn more about uberSVN rich system, user administration capabilities and model configuration. Take the hassle out of deploying and managing Subversion and the tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
Re: [Ganglia-general] Problem displaying Virtual Machine data with hsflowd and ganglia 3.2.0 in an Openstack Compute node.
try having soft links using ln command, may or may not work though :) ln -s On Wed, Aug 17, 2011 at 4:17 PM, Emanuele Verga verga.emanu...@gmail.com wrote: Hi Neil, thanks for the suggestion. I tried to do it. After recompiling hsflowd i checked the files it had open and it showed: hsflowd 30019 nobody mem REG 251,3 84728 927248 /usr/lib/libxenctrl.so.3.2.0 hsflowd 30019 nobody mem REG 251,3 22928 927250 /usr/lib/libxenstore.so.3.0.0 but this solution didn't work. Virtual machines stopped showing up in ganglia and hsflowd gave this error: ERROR Internal error: Could not obtain handle on privileged command interface (2 = No such file or directory) xc_interface_open() failed : No such file or directory then it continued to work normally (hsflowd log file: http://uploading.com/files/get/a3876c8b/ ) but it didn't report anything to ganglia and the VMS were shown as down, the reporting for the physical host instead it's working perfectly, as before. If it can help you, the implementation of Openstack we have uses KVM to virtualize the hosts. Could it be related? Thanks again, Emanuele 2011/8/16 Neil Mckee neil.mckee...@gmail.com Hello, On an OpenStack node you may be able to use libxenstore instead of libvirt. You'll need to recompile hsflowd to try this. Looking at trunk/src/Linux/Makefile it appears to look for libvirt first, but you can override that by compiling hsflowd like this: make clean make LIBVIRT=no The Makefile will then test for libxenstore and libxenctrl. If it finds them it will compile with -DHSF_XEN (instead of -DHSF_VRT), and you may get better results. Please let me know what happens. Neil On Aug 16, 2011, at 2:14 AM, Emanuele Verga wrote: Hi, we have a problem with the following installation: we have a system that’s a compute node in an Openstack test installation. Now on this machine we decided to install Ganglia, to check it’s monitoring capabilities regarding virtual machines hosted on that node by Openstack. We then proceeded to add the repository for ganglia version 3.2 and install: Hsflowd 1.18 Ganglia Monitor Demon 3.2.0.0 Ganglia Meta demon 3.2.0.0 Ganglia Web FronEnd 3.2.0.0 Dwoo 1.1.1 all on the same machine and to configure ports accordingly. All said and done, the web frontend shown the physical host and all of the VM, but we were unable to: See the hypervisor section in the physical host statistics. It simply is not there. See the graphical preview for VM statistics (http://imageshack.us/photo/my-images/585/screenshotgangliacomput.png/). The thumbnail are missing but the links do work, clicking on one of the missing thumbnails you are taken to the details page for that VM. See details regarding VM hard disk and I/O. (http://imageshack.us/photo/my-images/694/screenshotgangliainstan.png/) Debugging hsflowd we found errors similar to the following: Aug 16 06:26:57 eta hsflowd: virStorageLookupByPath(/var/lib/nova/instances/instance-004d/disk.local) failed Aug 16 06:26:57 eta hsflowd: virStorageLookupByPath(/var/lib/nova/instances/instance-004d/disk) failed The strange thing is, the path is correct. We checked libvirt using virsh and found that the Storage Volumes are not reported by libvirt, it seems because libvirt by default search informations in path /var/lib/libvirt/images, instead nova places them inside /var/lib/nova/instances/INSTANCENAME/. Did you have the same problems when testing for Sflow/ Openstack ? How did you manage to resolve it? Any help is appreciated. Thanks in advance for your support! -- uberSVN's rich system and user administration capabilities and model configuration take the hassle out of deploying and managing Subversion and the tools developers use with it. Learn more about uberSVN and get a free download at: http://p.sf.net/sfu/wandisco-dev2dev ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general -- Get a FREE DOWNLOAD! and learn more about uberSVN rich system, user administration capabilities and model configuration. Take the hassle out of deploying and managing Subversion and the tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2 ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general -- Get a FREE DOWNLOAD! and learn more about uberSVN rich system, user
Re: [Ganglia-general] Problem displaying Virtual Machine data with hsflowd and ganglia 3.2.0 in an Openstack Compute node.
I don't think the soft-links will work, but try this: 1) go back to compiling hsflowd for libvirt. 2) using virt-manager or equivalent, tell it about the storage pool of type filesystem directory at /var/lib/nova/instances. See details here: http://virt-manager.org/page/StorageManagement (To get to this virt-manager screen you need to select EditHost Details) Now you should see it with virsh pool-list. If you called it nova then virsh pool-info nova will show something like this: virsh # pool-info nova Name: noca UUID: 51341d87-d87e-f7ce-6cc0-81ac3967c182 State: running Capacity: 217.58 GB Allocation: 35.38 GB Available: 182.20 GB hsflowd should then be able to pull those Capacity, Allocation and Available values and send them to Ganglia. Neil P.S. It looks as though hsflowd is not filling in the disk reads/writes/errors counters for KVM VMs yet. If you know of an efficient way to do that, please suggest it on the hsflowd mailing list: https://lists.sourceforge.net/lists/listinfo/host-sflow-discuss On Aug 17, 2011, at 7:16 AM, santosh gangwani wrote: try having soft links using ln command, may or may not work though :) ln -s On Wed, Aug 17, 2011 at 4:17 PM, Emanuele Verga verga.emanu...@gmail.com wrote: Hi Neil, thanks for the suggestion. I tried to do it. After recompiling hsflowd i checked the files it had open and it showed: hsflowd 30019 nobody mem REG 251,3 84728 927248 /usr/lib/libxenctrl.so.3.2.0 hsflowd 30019 nobody mem REG 251,3 22928 927250 /usr/lib/libxenstore.so.3.0.0 but this solution didn't work. Virtual machines stopped showing up in ganglia and hsflowd gave this error: ERROR Internal error: Could not obtain handle on privileged command interface (2 = No such file or directory) xc_interface_open() failed : No such file or directory then it continued to work normally (hsflowd log file: http://uploading.com/files/get/a3876c8b/ ) but it didn't report anything to ganglia and the VMS were shown as down, the reporting for the physical host instead it's working perfectly, as before. If it can help you, the implementation of Openstack we have uses KVM to virtualize the hosts. Could it be related? Thanks again, Emanuele 2011/8/16 Neil Mckee neil.mckee...@gmail.com Hello, On an OpenStack node you may be able to use libxenstore instead of libvirt. You'll need to recompile hsflowd to try this. Looking at trunk/src/Linux/Makefile it appears to look for libvirt first, but you can override that by compiling hsflowd like this: make clean make LIBVIRT=no The Makefile will then test for libxenstore and libxenctrl. If it finds them it will compile with -DHSF_XEN (instead of -DHSF_VRT), and you may get better results. Please let me know what happens. Neil On Aug 16, 2011, at 2:14 AM, Emanuele Verga wrote: Hi, we have a problem with the following installation: we have a system that’s a compute node in an Openstack test installation. Now on this machine we decided to install Ganglia, to check it’s monitoring capabilities regarding virtual machines hosted on that node by Openstack. We then proceeded to add the repository for ganglia version 3.2 and install: Hsflowd 1.18 Ganglia Monitor Demon 3.2.0.0 Ganglia Meta demon 3.2.0.0 Ganglia Web FronEnd3.2.0.0 Dwoo1.1.1 all on the same machine and to configure ports accordingly. All said and done, the web frontend shown the physical host and all of the VM, but we were unable to: See the hypervisor section in the physical host statistics. It simply is not there. See the graphical preview for VM statistics (http://imageshack.us/photo/my-images/585/screenshotgangliacomput.png/). The thumbnail are missing but the links do work, clicking on one of the missing thumbnails you are taken to the details page for that VM. See details regarding VM hard disk and I/O. (http://imageshack.us/photo/my-images/694/screenshotgangliainstan.png/) Debugging hsflowd we found errors similar to the following: Aug 16 06:26:57 eta hsflowd: virStorageLookupByPath(/var/lib/nova/instances/instance-004d/disk.local) failed Aug 16 06:26:57 eta hsflowd: virStorageLookupByPath(/var/lib/nova/instances/instance-004d/disk) failed The strange thing is, the path is correct. We checked libvirt using virsh and found that the Storage Volumes are not reported by libvirt, it seems because libvirt by default search informations in path /var/lib/libvirt/images, instead nova places them inside /var/lib/nova/instances/INSTANCENAME/. Did you have the same problems when testing for Sflow/ Openstack ? How did you manage to resolve it? Any help is appreciated. Thanks in advance for your support!
[Ganglia-general] Problem displaying Virtual Machine data with hsflowd and ganglia 3.2.0 in an Openstack Compute node.
Hi, we have a problem with the following installation: we have a system that’s a compute node in an Openstack test installation. Now on this machine we decided to install Ganglia, to check it’s monitoring capabilities regarding virtual machines hosted on that node by Openstack. We then proceeded to add the repository for ganglia version 3.2 and install: Hsflowd 1.18 Ganglia Monitor Demon 3.2.0.0 Ganglia Meta demon 3.2.0.0 Ganglia Web FronEnd3.2.0.0 Dwoo1.1.1 all on the same machine and to configure ports accordingly. All said and done, the web frontend shown the physical host and all of the VM, but we were unable to: See the hypervisor section in the physical host statistics. It simply is not there. See the graphical preview for VM statistics ( http://imageshack.us/photo/my-images/585/screenshotgangliacomput.png/). The thumbnail are missing but the links do work, clicking on one of the missing thumbnails you are taken to the details page for that VM. See details regarding VM hard disk and I/O. ( http://imageshack.us/photo/my-images/694/screenshotgangliainstan.png/) Debugging hsflowd we found errors similar to the following: Aug 16 06:26:57 eta hsflowd: virStorageLookupByPath(/var/lib/nova/instances/instance-004d/disk.local) failed Aug 16 06:26:57 eta hsflowd: virStorageLookupByPath(/var/lib/nova/instances/instance-004d/disk) failed The strange thing is, the path is correct. We checked libvirt using virsh and found that the Storage Volumes are not reported by libvirt, it seems because libvirt by default search informations in path /var/lib/libvirt/images, instead nova places them inside /var/lib/nova/instances/INSTANCENAME/. Did you have the same problems when testing for Sflow/ Openstack ? How did you manage to resolve it? Any help is appreciated. Thanks in advance for your support! -- uberSVN's rich system and user administration capabilities and model configuration take the hassle out of deploying and managing Subversion and the tools developers use with it. Learn more about uberSVN and get a free download at: http://p.sf.net/sfu/wandisco-dev2dev ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
Re: [Ganglia-general] Problem displaying Virtual Machine data with hsflowd and ganglia 3.2.0 in an Openstack Compute node.
Hello, On an OpenStack node you may be able to use libxenstore instead of libvirt. You'll need to recompile hsflowd to try this. Looking at trunk/src/Linux/Makefile it appears to look for libvirt first, but you can override that by compiling hsflowd like this: make clean make LIBVIRT=no The Makefile will then test for libxenstore and libxenctrl. If it finds them it will compile with -DHSF_XEN (instead of -DHSF_VRT), and you may get better results. Please let me know what happens. Neil On Aug 16, 2011, at 2:14 AM, Emanuele Verga wrote: Hi, we have a problem with the following installation: we have a system that’s a compute node in an Openstack test installation. Now on this machine we decided to install Ganglia, to check it’s monitoring capabilities regarding virtual machines hosted on that node by Openstack. We then proceeded to add the repository for ganglia version 3.2 and install: Hsflowd 1.18 Ganglia Monitor Demon 3.2.0.0 Ganglia Meta demon 3.2.0.0 Ganglia Web FronEnd3.2.0.0 Dwoo1.1.1 all on the same machine and to configure ports accordingly. All said and done, the web frontend shown the physical host and all of the VM, but we were unable to: See the hypervisor section in the physical host statistics. It simply is not there. See the graphical preview for VM statistics (http://imageshack.us/photo/my-images/585/screenshotgangliacomput.png/). The thumbnail are missing but the links do work, clicking on one of the missing thumbnails you are taken to the details page for that VM. See details regarding VM hard disk and I/O. (http://imageshack.us/photo/my-images/694/screenshotgangliainstan.png/) Debugging hsflowd we found errors similar to the following: Aug 16 06:26:57 eta hsflowd: virStorageLookupByPath(/var/lib/nova/instances/instance-004d/disk.local) failed Aug 16 06:26:57 eta hsflowd: virStorageLookupByPath(/var/lib/nova/instances/instance-004d/disk) failed The strange thing is, the path is correct. We checked libvirt using virsh and found that the Storage Volumes are not reported by libvirt, it seems because libvirt by default search informations in path /var/lib/libvirt/images, instead nova places them inside /var/lib/nova/instances/INSTANCENAME/. Did you have the same problems when testing for Sflow/ Openstack ? How did you manage to resolve it? Any help is appreciated. Thanks in advance for your support! -- uberSVN's rich system and user administration capabilities and model configuration take the hassle out of deploying and managing Subversion and the tools developers use with it. Learn more about uberSVN and get a free download at: http://p.sf.net/sfu/wandisco-dev2dev ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general -- Get a FREE DOWNLOAD! and learn more about uberSVN rich system, user administration capabilities and model configuration. Take the hassle out of deploying and managing Subversion and the tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2 ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
