Re: [Ganglia-developers] Ganglia libvirt and other virtualization technologies
Hi Semy: The more elegant solution is to use gmetad-python and write plugins to manipulate the metric data collected however way you want. For more information, please see the README: https://github.com/ganglia/monitor-core/blob/master/gmetad-python/README And Peter is right, ganglia-general is the more appropriate list for your questions. This list should be reserved for code development discussions, not so much user questions. Thanks, Bernard On Wed, Jun 20, 2012 at 12:40 PM, Simon G. wrote: > I think I've found answer for my question. I've just read few files from > source code of ganglia-web and as i can see they are reading RRD to draw > everything so i think I have to write my own wrapper around RRD to do what i > need. Anyway thx for informations. > > Cheers, > > > On Wed, Jun 20, 2012 at 8:57 PM, Peter Phaal wrote: >> >> There are people on this list who should be able to answer questions >> relating to gmetad (although you might have better luck on the >> ganglia-general list). My expertise is limited to sFlow. >> >> >> On Jun 20, 2012, at 11:39 AM, Simon G. wrote: >> >> In next 2-5 days I'll try ganglia + sFlow again, so I'll definitely let >> you know about my problems. >> >> If you can't answer my previous question to whom should I send it? >> >> Cheers, >> >> On Wed, Jun 20, 2012 at 8:24 PM, Peter Phaal >> wrote: >>> >>> I can't help you with your questions about gmetad, but I would be very >>> interested in hearing where you ran into trouble during the sFlow/Ganglia >>> configuration. I am currently working on documentation and would like to >>> make sure that I fill any gaps. >>> >>> >>> On Jun 20, 2012, at 11:09 AM, Simon G. wrote: >>> >>> I've already seen sFlow and ganglia about month ago, and I had some >>> problems during configuration, but maybe I should try it again. >>> >>> So maybe I have different question now. Is there any way to query gmetad >>> about historical data? I'd like to retrieve some data in my external piece >>> of software (python module) to analyze those data and do some actions if >>> proper conditions are fulfilled. I don't want store data twice (in my module >>> for example in my Mysql database and in RRD files created by gmetad), so my >>> true question is: can i query gmetad to get those data, or should i just >>> read gmetad's RRD file and do not connect with gmetad itself or maybe should >>> i get data through tcp from gmetad and store it in my own way and own >>> database? I don't know much about RRD and I'd like to avoid reading RRD >>> files myself, but also I don't want to store them twice. It would be perfect >>> if I could get them through ganglia (from gmetad after proper request), >>> because mostly, I'm interested in historical data. >>> >>> Btw. what differences are between gmetad and gmetad-python? Beside >>> language of course. Both implementations have the same features? >>> >>> >>> On Tue, Jun 19, 2012 at 12:04 AM, Peter Phaal >>> wrote: Simon, Ganglia 3.2+ can collect and display sFlow metrics: http://blog.sflow.com/2012/01/using-ganglia-to-monitor-virtual.html You can monitor the virtual machines by installing Host sFlow agents (http://host-sflow.sourceforge.net/ ) on the hypervisors. The sFlow agents export libvirt metrics using the sFlow protocol. Currently, Host sFlow supports KVM, Xen and Hyper-V. Cheers, Peter On Jun 18, 2012, at 2:50 PM, Simon G. wrote: Hello, Is there any way to use Ganglia to monitor virtual machines? Right now I'd like to get data from libvirt's api (CPU usage, IO, Network etc), but I don't want to reinvent the wheel and I'd like to use already mature tools to do that, for example Ganglia. I know collectd has plugins which support libvirt, but from what I've read, ganglia could be better solution. 1st question is: how can I monitor virtual machines in Ganglia. Especially using libvirt, but later also Xen, VMware, Hyper-V. Is it possible? Can ganglia monitor virtual machines? What can I monitor without additional agent on guest and what can I monitor with such agent? 2nd question: I've just found out gmetad-python, but before I'll try to use it I'd like to ask about possibilities to use python scripts to collect data from gmond. What is the easiest way to do that? I'm interested in python, because I'd like to try create simple Openstack and Ganglia proof of concept. I was going to use existing simple monitoring around libvirt from openstack and implement storing data in RRD files, but as I've mentioned above, I don't want to reinvent the wheel and I think a better way is to integrate already existing solutions. The question is how to integrate those two projects. My first step is to run gmond on each host, collect data from it via python script, save those data into RRD file and use this data in Open
Re: [Ganglia-developers] Ganglia libvirt and other virtualization technologies
TCP is not supported. Ganglia's binary protocol and sFlow both use UDP. Loosing a packet is no big deal - a new measurement will be sent next polling interval and the benefits of a stateless protocol are considerable. Peter On Jun 20, 2012, at 3:30 PM, "Simon G." wrote: > On Wed, Jun 20, 2012 at 8:57 PM, Peter Phaal wrote: > There are people on this list who should be able to answer questions relating > to gmetad (although you might have better luck on the ganglia-general list). > My expertise is limited to sFlow. > > Btw. sFlow can use TCP as well as UDP? I'm worried about UDP unreliability. > > -- > Simon > > -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers
Re: [Ganglia-developers] Ganglia libvirt and other virtualization technologies
On Wed, Jun 20, 2012 at 8:57 PM, Peter Phaal wrote: > There are people on this list who should be able to answer questions > relating to gmetad (although you might have better luck on the > ganglia-general list). My expertise is limited to sFlow. > Btw. sFlow can use TCP as well as UDP? I'm worried about UDP unreliability. -- *Simon* -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers
Re: [Ganglia-developers] Ganglia libvirt and other virtualization technologies
On Wed, Jun 20, 2012 at 10:08 PM, Alex Dean wrote: > gweb allows downloading RRD data as either csv or json. Might that help? > > It could definitely help even if I'm not good at PHP. I'll try it. Thx. -- *Simon* -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers
Re: [Ganglia-developers] Ganglia libvirt and other virtualization technologies
gweb allows downloading RRD data as either csv or json. Might that help? On Jun 20, 2012, at 2:40 PM, Simon G. wrote: > I think I've found answer for my question. I've just read few files from > source code of ganglia-web and as i can see they are reading RRD to draw > everything so i think I have to write my own wrapper around RRD to do what i > need. Anyway thx for informations. > > Cheers, > > On Wed, Jun 20, 2012 at 8:57 PM, Peter Phaal wrote: > There are people on this list who should be able to answer questions relating > to gmetad (although you might have better luck on the ganglia-general list). > My expertise is limited to sFlow. > > > On Jun 20, 2012, at 11:39 AM, Simon G. wrote: > >> In next 2-5 days I'll try ganglia + sFlow again, so I'll definitely let you >> know about my problems. >> >> If you can't answer my previous question to whom should I send it? >> >> Cheers, >> >> On Wed, Jun 20, 2012 at 8:24 PM, Peter Phaal wrote: >> I can't help you with your questions about gmetad, but I would be very >> interested in hearing where you ran into trouble during the sFlow/Ganglia >> configuration. I am currently working on documentation and would like to >> make sure that I fill any gaps. >> >> >> On Jun 20, 2012, at 11:09 AM, Simon G. wrote: >> >>> I've already seen sFlow and ganglia about month ago, and I had some >>> problems during configuration, but maybe I should try it again. >>> >>> So maybe I have different question now. Is there any way to query gmetad >>> about historical data? I'd like to retrieve some data in my external piece >>> of software (python module) to analyze those data and do some actions if >>> proper conditions are fulfilled. I don't want store data twice (in my >>> module for example in my Mysql database and in RRD files created by >>> gmetad), so my true question is: can i query gmetad to get those data, or >>> should i just read gmetad's RRD file and do not connect with gmetad itself >>> or maybe should i get data through tcp from gmetad and store it in my own >>> way and own database? I don't know much about RRD and I'd like to avoid >>> reading RRD files myself, but also I don't want to store them twice. It >>> would be perfect if I could get them through ganglia (from gmetad after >>> proper request), because mostly, I'm interested in historical data. >>> >>> Btw. what differences are between gmetad and gmetad-python? Beside language >>> of course. Both implementations have the same features? >>> >>> >>> On Tue, Jun 19, 2012 at 12:04 AM, Peter Phaal wrote: >>> Simon, >>> >>> Ganglia 3.2+ can collect and display sFlow metrics: >>> http://blog.sflow.com/2012/01/using-ganglia-to-monitor-virtual.html >>> >>> You can monitor the virtual machines by installing Host sFlow agents >>> (http://host-sflow.sourceforge.net/ ) on the hypervisors. The sFlow agents >>> export libvirt metrics using the sFlow protocol. Currently, Host sFlow >>> supports KVM, Xen and Hyper-V. >>> >>> Cheers, >>> Peter >>> >>> On Jun 18, 2012, at 2:50 PM, Simon G. wrote: >>> Hello, Is there any way to use Ganglia to monitor virtual machines? Right now I'd like to get data from libvirt's api (CPU usage, IO, Network etc), but I don't want to reinvent the wheel and I'd like to use already mature tools to do that, for example Ganglia. I know collectd has plugins which support libvirt, but from what I've read, ganglia could be better solution. 1st question is: how can I monitor virtual machines in Ganglia. Especially using libvirt, but later also Xen, VMware, Hyper-V. Is it possible? Can ganglia monitor virtual machines? What can I monitor without additional agent on guest and what can I monitor with such agent? 2nd question: I've just found out gmetad-python, but before I'll try to use it I'd like to ask about possibilities to use python scripts to collect data from gmond. What is the easiest way to do that? I'm interested in python, because I'd like to try create simple Openstack and Ganglia proof of concept. I was going to use existing simple monitoring around libvirt from openstack and implement storing data in RRD files, but as I've mentioned above, I don't want to reinvent the wheel and I think a better way is to integrate already existing solutions. The question is how to integrate those two projects. My first step is to run gmond on each host, collect data from it via python script, save those data into RRD file and use this data in Openstack's environment. If gmetad-python can do that, it's great. Then second step will be to get that data from gmetad-python (via python script) and use it in Openstack's environment. Cheers, -- Simon -- Live Security Virtual Conference Exclu
Re: [Ganglia-developers] Ganglia libvirt and other virtualization technologies
I think I've found answer for my question. I've just read few files from source code of ganglia-web and as i can see they are reading RRD to draw everything so i think I have to write my own wrapper around RRD to do what i need. Anyway thx for informations. Cheers, On Wed, Jun 20, 2012 at 8:57 PM, Peter Phaal wrote: > There are people on this list who should be able to answer questions > relating to gmetad (although you might have better luck on the > ganglia-general list). My expertise is limited to sFlow. > > > On Jun 20, 2012, at 11:39 AM, Simon G. wrote: > > In next 2-5 days I'll try ganglia + sFlow again, so I'll definitely let > you know about my problems. > > If you can't answer my previous question to whom should I send it? > > Cheers, > > On Wed, Jun 20, 2012 at 8:24 PM, Peter Phaal wrote: > >> I can't help you with your questions about gmetad, but I would be very >> interested in hearing where you ran into trouble during the sFlow/Ganglia >> configuration. I am currently working on documentation and would like to >> make sure that I fill any gaps. >> >> >> On Jun 20, 2012, at 11:09 AM, Simon G. wrote: >> >> I've already seen sFlow and ganglia about month ago, and I had some >> problems during configuration, but maybe I should try it again. >> >> So maybe I have different question now. Is there any way to query gmetad >> about historical data? I'd like to retrieve some data in my external piece >> of software (python module) to analyze those data and do some actions if >> proper conditions are fulfilled. I don't want store data twice (in my >> module for example in my Mysql database and in RRD files created by >> gmetad), so my true question is: can i query gmetad to get those data, or >> should i just read gmetad's RRD file and do not connect with gmetad itself >> or maybe should i get data through tcp from gmetad and store it in my own >> way and own database? I don't know much about RRD and I'd like to avoid >> reading RRD files myself, but also I don't want to store them twice. It >> would be perfect if I could get them through ganglia (from gmetad after >> proper request), because mostly, I'm interested in historical data. >> >> Btw. what differences are between gmetad and gmetad-python? Beside >> language of course. Both implementations have the same features? >> >> >> On Tue, Jun 19, 2012 at 12:04 AM, Peter Phaal wrote: >> >>> Simon, >>> >>> Ganglia 3.2+ can collect and display sFlow metrics: >>> http://blog.sflow.com/2012/01/using-ganglia-to-monitor-virtual.html >>> >>> You can monitor the virtual machines by installing Host sFlow agents ( >>> http://host-sflow.sourceforge.net/ ) on the hypervisors. The sFlow >>> agents export libvirt metrics using the sFlow protocol. Currently, Host >>> sFlow supports KVM, Xen and Hyper-V. >>> >>> Cheers, >>> Peter >>> >>> On Jun 18, 2012, at 2:50 PM, Simon G. wrote: >>> >>> Hello, >>> >>> Is there any way to use Ganglia to monitor virtual machines? Right now >>> I'd like to get data from libvirt's api (CPU usage, IO, Network etc), but I >>> don't want to reinvent the wheel and I'd like to use already mature tools >>> to do that, for example Ganglia. I know collectd has plugins which support >>> libvirt, but from what I've read, ganglia could be better solution. >>> >>> 1st question is: how can I monitor virtual machines in Ganglia. >>> Especially using libvirt, but later also Xen, VMware, Hyper-V. Is it >>> possible? Can ganglia monitor virtual machines? What can I monitor without >>> additional agent on guest and what can I monitor with such agent? >>> 2nd question: I've just found out gmetad-python, but before I'll try to >>> use it I'd like to ask about possibilities to use python scripts to collect >>> data from gmond. What is the easiest way to do that? >>> >>> I'm interested in python, because I'd like to try create simple >>> Openstack and Ganglia proof of concept. I was going to use existing simple >>> monitoring around libvirt from openstack and implement storing data in RRD >>> files, but as I've mentioned above, I don't want to reinvent the wheel and >>> I think a better way is to integrate already existing solutions. The >>> question is how to integrate those two projects. My first step is to run >>> gmond on each host, collect data from it via python script, save those data >>> into RRD file and use this data in Openstack's environment. If >>> gmetad-python can do that, it's great. Then second step will be to get that >>> data from gmetad-python (via python script) and use it in Openstack's >>> environment. >>> >>> Cheers, >>> -- >>> *Simon* >>> >>> >>> >>> -- >>> Live Security Virtual Conference >>> Exclusive live event will cover all the ways today's security and >>> threat landscape has changed and how IT managers can respond. >>> Discussions >>> will include endpoint security, mobile security and the latest in >>> malware >>> threats. >>> http://www.accelacom
Re: [Ganglia-developers] Ganglia libvirt and other virtualization technologies
There are people on this list who should be able to answer questions relating to gmetad (although you might have better luck on the ganglia-general list). My expertise is limited to sFlow. On Jun 20, 2012, at 11:39 AM, Simon G. wrote: > In next 2-5 days I'll try ganglia + sFlow again, so I'll definitely let you > know about my problems. > > If you can't answer my previous question to whom should I send it? > > Cheers, > > On Wed, Jun 20, 2012 at 8:24 PM, Peter Phaal wrote: > I can't help you with your questions about gmetad, but I would be very > interested in hearing where you ran into trouble during the sFlow/Ganglia > configuration. I am currently working on documentation and would like to make > sure that I fill any gaps. > > > On Jun 20, 2012, at 11:09 AM, Simon G. wrote: > >> I've already seen sFlow and ganglia about month ago, and I had some problems >> during configuration, but maybe I should try it again. >> >> So maybe I have different question now. Is there any way to query gmetad >> about historical data? I'd like to retrieve some data in my external piece >> of software (python module) to analyze those data and do some actions if >> proper conditions are fulfilled. I don't want store data twice (in my module >> for example in my Mysql database and in RRD files created by gmetad), so my >> true question is: can i query gmetad to get those data, or should i just >> read gmetad's RRD file and do not connect with gmetad itself or maybe should >> i get data through tcp from gmetad and store it in my own way and own >> database? I don't know much about RRD and I'd like to avoid reading RRD >> files myself, but also I don't want to store them twice. It would be perfect >> if I could get them through ganglia (from gmetad after proper request), >> because mostly, I'm interested in historical data. >> >> Btw. what differences are between gmetad and gmetad-python? Beside language >> of course. Both implementations have the same features? >> >> >> On Tue, Jun 19, 2012 at 12:04 AM, Peter Phaal wrote: >> Simon, >> >> Ganglia 3.2+ can collect and display sFlow metrics: >> http://blog.sflow.com/2012/01/using-ganglia-to-monitor-virtual.html >> >> You can monitor the virtual machines by installing Host sFlow agents >> (http://host-sflow.sourceforge.net/ ) on the hypervisors. The sFlow agents >> export libvirt metrics using the sFlow protocol. Currently, Host sFlow >> supports KVM, Xen and Hyper-V. >> >> Cheers, >> Peter >> >> On Jun 18, 2012, at 2:50 PM, Simon G. wrote: >> >>> Hello, >>> >>> Is there any way to use Ganglia to monitor virtual machines? Right now I'd >>> like to get data from libvirt's api (CPU usage, IO, Network etc), but I >>> don't want to reinvent the wheel and I'd like to use already mature tools >>> to do that, for example Ganglia. I know collectd has plugins which support >>> libvirt, but from what I've read, ganglia could be better solution. >>> >>> 1st question is: how can I monitor virtual machines in Ganglia. Especially >>> using libvirt, but later also Xen, VMware, Hyper-V. Is it possible? Can >>> ganglia monitor virtual machines? What can I monitor without additional >>> agent on guest and what can I monitor with such agent? >>> 2nd question: I've just found out gmetad-python, but before I'll try to use >>> it I'd like to ask about possibilities to use python scripts to collect >>> data from gmond. What is the easiest way to do that? >>> >>> I'm interested in python, because I'd like to try create simple Openstack >>> and Ganglia proof of concept. I was going to use existing simple monitoring >>> around libvirt from openstack and implement storing data in RRD files, but >>> as I've mentioned above, I don't want to reinvent the wheel and I think a >>> better way is to integrate already existing solutions. The question is how >>> to integrate those two projects. My first step is to run gmond on each >>> host, collect data from it via python script, save those data into RRD file >>> and use this data in Openstack's environment. If gmetad-python can do that, >>> it's great. Then second step will be to get that data from gmetad-python >>> (via python script) and use it in Openstack's environment. >>> >>> Cheers, >>> -- >>> Simon >>> >>> >>> -- >>> Live Security Virtual Conference >>> Exclusive live event will cover all the ways today's security and >>> threat landscape has changed and how IT managers can respond. Discussions >>> will include endpoint security, mobile security and the latest in malware >>> threats. >>> http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ >>> Ganglia-developers mailing list >>> Ganglia-developers@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/ganglia-developers >> >> >> >> >> -- >> Semy >> >> > > > > > -- > Semy > >
Re: [Ganglia-developers] Ganglia libvirt and other virtualization technologies
In next 2-5 days I'll try ganglia + sFlow again, so I'll definitely let you know about my problems. If you can't answer my previous question to whom should I send it? Cheers, On Wed, Jun 20, 2012 at 8:24 PM, Peter Phaal wrote: > I can't help you with your questions about gmetad, but I would be very > interested in hearing where you ran into trouble during the sFlow/Ganglia > configuration. I am currently working on documentation and would like to > make sure that I fill any gaps. > > > On Jun 20, 2012, at 11:09 AM, Simon G. wrote: > > I've already seen sFlow and ganglia about month ago, and I had some > problems during configuration, but maybe I should try it again. > > So maybe I have different question now. Is there any way to query gmetad > about historical data? I'd like to retrieve some data in my external piece > of software (python module) to analyze those data and do some actions if > proper conditions are fulfilled. I don't want store data twice (in my > module for example in my Mysql database and in RRD files created by > gmetad), so my true question is: can i query gmetad to get those data, or > should i just read gmetad's RRD file and do not connect with gmetad itself > or maybe should i get data through tcp from gmetad and store it in my own > way and own database? I don't know much about RRD and I'd like to avoid > reading RRD files myself, but also I don't want to store them twice. It > would be perfect if I could get them through ganglia (from gmetad after > proper request), because mostly, I'm interested in historical data. > > Btw. what differences are between gmetad and gmetad-python? Beside > language of course. Both implementations have the same features? > > > On Tue, Jun 19, 2012 at 12:04 AM, Peter Phaal wrote: > >> Simon, >> >> Ganglia 3.2+ can collect and display sFlow metrics: >> http://blog.sflow.com/2012/01/using-ganglia-to-monitor-virtual.html >> >> You can monitor the virtual machines by installing Host sFlow agents ( >> http://host-sflow.sourceforge.net/ ) on the hypervisors. The sFlow >> agents export libvirt metrics using the sFlow protocol. Currently, Host >> sFlow supports KVM, Xen and Hyper-V. >> >> Cheers, >> Peter >> >> On Jun 18, 2012, at 2:50 PM, Simon G. wrote: >> >> Hello, >> >> Is there any way to use Ganglia to monitor virtual machines? Right now >> I'd like to get data from libvirt's api (CPU usage, IO, Network etc), but I >> don't want to reinvent the wheel and I'd like to use already mature tools >> to do that, for example Ganglia. I know collectd has plugins which support >> libvirt, but from what I've read, ganglia could be better solution. >> >> 1st question is: how can I monitor virtual machines in Ganglia. >> Especially using libvirt, but later also Xen, VMware, Hyper-V. Is it >> possible? Can ganglia monitor virtual machines? What can I monitor without >> additional agent on guest and what can I monitor with such agent? >> 2nd question: I've just found out gmetad-python, but before I'll try to >> use it I'd like to ask about possibilities to use python scripts to collect >> data from gmond. What is the easiest way to do that? >> >> I'm interested in python, because I'd like to try create simple Openstack >> and Ganglia proof of concept. I was going to use existing simple monitoring >> around libvirt from openstack and implement storing data in RRD files, but >> as I've mentioned above, I don't want to reinvent the wheel and I think a >> better way is to integrate already existing solutions. The question is how >> to integrate those two projects. My first step is to run gmond on each >> host, collect data from it via python script, save those data into RRD file >> and use this data in Openstack's environment. If gmetad-python can do that, >> it's great. Then second step will be to get that data from gmetad-python >> (via python script) and use it in Openstack's environment. >> >> Cheers, >> -- >> *Simon* >> >> >> >> -- >> Live Security Virtual Conference >> Exclusive live event will cover all the ways today's security and >> threat landscape has changed and how IT managers can respond. Discussions >> will include endpoint security, mobile security and the latest in malware >> threats. >> http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ >> Ganglia-developers mailing list >> Ganglia-developers@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/ganglia-developers >> >> >> > > > -- > *Semy* > > > > -- *Semy* -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___
Re: [Ganglia-developers] Ganglia libvirt and other virtualization technologies
I can't help you with your questions about gmetad, but I would be very interested in hearing where you ran into trouble during the sFlow/Ganglia configuration. I am currently working on documentation and would like to make sure that I fill any gaps. On Jun 20, 2012, at 11:09 AM, Simon G. wrote: > I've already seen sFlow and ganglia about month ago, and I had some problems > during configuration, but maybe I should try it again. > > So maybe I have different question now. Is there any way to query gmetad > about historical data? I'd like to retrieve some data in my external piece of > software (python module) to analyze those data and do some actions if proper > conditions are fulfilled. I don't want store data twice (in my module for > example in my Mysql database and in RRD files created by gmetad), so my true > question is: can i query gmetad to get those data, or should i just read > gmetad's RRD file and do not connect with gmetad itself or maybe should i get > data through tcp from gmetad and store it in my own way and own database? I > don't know much about RRD and I'd like to avoid reading RRD files myself, but > also I don't want to store them twice. It would be perfect if I could get > them through ganglia (from gmetad after proper request), because mostly, I'm > interested in historical data. > > Btw. what differences are between gmetad and gmetad-python? Beside language > of course. Both implementations have the same features? > > > On Tue, Jun 19, 2012 at 12:04 AM, Peter Phaal wrote: > Simon, > > Ganglia 3.2+ can collect and display sFlow metrics: > http://blog.sflow.com/2012/01/using-ganglia-to-monitor-virtual.html > > You can monitor the virtual machines by installing Host sFlow agents > (http://host-sflow.sourceforge.net/ ) on the hypervisors. The sFlow agents > export libvirt metrics using the sFlow protocol. Currently, Host sFlow > supports KVM, Xen and Hyper-V. > > Cheers, > Peter > > On Jun 18, 2012, at 2:50 PM, Simon G. wrote: > >> Hello, >> >> Is there any way to use Ganglia to monitor virtual machines? Right now I'd >> like to get data from libvirt's api (CPU usage, IO, Network etc), but I >> don't want to reinvent the wheel and I'd like to use already mature tools to >> do that, for example Ganglia. I know collectd has plugins which support >> libvirt, but from what I've read, ganglia could be better solution. >> >> 1st question is: how can I monitor virtual machines in Ganglia. Especially >> using libvirt, but later also Xen, VMware, Hyper-V. Is it possible? Can >> ganglia monitor virtual machines? What can I monitor without additional >> agent on guest and what can I monitor with such agent? >> 2nd question: I've just found out gmetad-python, but before I'll try to use >> it I'd like to ask about possibilities to use python scripts to collect data >> from gmond. What is the easiest way to do that? >> >> I'm interested in python, because I'd like to try create simple Openstack >> and Ganglia proof of concept. I was going to use existing simple monitoring >> around libvirt from openstack and implement storing data in RRD files, but >> as I've mentioned above, I don't want to reinvent the wheel and I think a >> better way is to integrate already existing solutions. The question is how >> to integrate those two projects. My first step is to run gmond on each host, >> collect data from it via python script, save those data into RRD file and >> use this data in Openstack's environment. If gmetad-python can do that, it's >> great. Then second step will be to get that data from gmetad-python (via >> python script) and use it in Openstack's environment. >> >> Cheers, >> -- >> Simon >> >> >> -- >> Live Security Virtual Conference >> Exclusive live event will cover all the ways today's security and >> threat landscape has changed and how IT managers can respond. Discussions >> will include endpoint security, mobile security and the latest in malware >> threats. >> http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ >> Ganglia-developers mailing list >> Ganglia-developers@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/ganglia-developers > > > > > -- > Semy > > -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers
Re: [Ganglia-developers] Ganglia libvirt and other virtualization technologies
I've already seen sFlow and ganglia about month ago, and I had some problems during configuration, but maybe I should try it again. So maybe I have different question now. Is there any way to query gmetad about historical data? I'd like to retrieve some data in my external piece of software (python module) to analyze those data and do some actions if proper conditions are fulfilled. I don't want store data twice (in my module for example in my Mysql database and in RRD files created by gmetad), so my true question is: can i query gmetad to get those data, or should i just read gmetad's RRD file and do not connect with gmetad itself or maybe should i get data through tcp from gmetad and store it in my own way and own database? I don't know much about RRD and I'd like to avoid reading RRD files myself, but also I don't want to store them twice. It would be perfect if I could get them through ganglia (from gmetad after proper request), because mostly, I'm interested in historical data. Btw. what differences are between gmetad and gmetad-python? Beside language of course. Both implementations have the same features? On Tue, Jun 19, 2012 at 12:04 AM, Peter Phaal wrote: > Simon, > > Ganglia 3.2+ can collect and display sFlow metrics: > http://blog.sflow.com/2012/01/using-ganglia-to-monitor-virtual.html > > You can monitor the virtual machines by installing Host sFlow agents ( > http://host-sflow.sourceforge.net/ ) on the hypervisors. The sFlow agents > export libvirt metrics using the sFlow protocol. Currently, Host sFlow > supports KVM, Xen and Hyper-V. > > Cheers, > Peter > > On Jun 18, 2012, at 2:50 PM, Simon G. wrote: > > Hello, > > Is there any way to use Ganglia to monitor virtual machines? Right now I'd > like to get data from libvirt's api (CPU usage, IO, Network etc), but I > don't want to reinvent the wheel and I'd like to use already mature tools > to do that, for example Ganglia. I know collectd has plugins which support > libvirt, but from what I've read, ganglia could be better solution. > > 1st question is: how can I monitor virtual machines in Ganglia. Especially > using libvirt, but later also Xen, VMware, Hyper-V. Is it possible? Can > ganglia monitor virtual machines? What can I monitor without additional > agent on guest and what can I monitor with such agent? > 2nd question: I've just found out gmetad-python, but before I'll try to > use it I'd like to ask about possibilities to use python scripts to collect > data from gmond. What is the easiest way to do that? > > I'm interested in python, because I'd like to try create simple Openstack > and Ganglia proof of concept. I was going to use existing simple monitoring > around libvirt from openstack and implement storing data in RRD files, but > as I've mentioned above, I don't want to reinvent the wheel and I think a > better way is to integrate already existing solutions. The question is how > to integrate those two projects. My first step is to run gmond on each > host, collect data from it via python script, save those data into RRD file > and use this data in Openstack's environment. If gmetad-python can do that, > it's great. Then second step will be to get that data from gmetad-python > (via python script) and use it in Openstack's environment. > > Cheers, > -- > *Simon* > > > > -- > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. > http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ > Ganglia-developers mailing list > Ganglia-developers@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/ganglia-developers > > > -- *Semy* -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers
Re: [Ganglia-developers] Ganglia libvirt and other virtualization technologies
Simon, Ganglia 3.2+ can collect and display sFlow metrics: http://blog.sflow.com/2012/01/using-ganglia-to-monitor-virtual.html You can monitor the virtual machines by installing Host sFlow agents (http://host-sflow.sourceforge.net/ ) on the hypervisors. The sFlow agents export libvirt metrics using the sFlow protocol. Currently, Host sFlow supports KVM, Xen and Hyper-V. Cheers, Peter On Jun 18, 2012, at 2:50 PM, Simon G. wrote: > Hello, > > Is there any way to use Ganglia to monitor virtual machines? Right now I'd > like to get data from libvirt's api (CPU usage, IO, Network etc), but I don't > want to reinvent the wheel and I'd like to use already mature tools to do > that, for example Ganglia. I know collectd has plugins which support libvirt, > but from what I've read, ganglia could be better solution. > > 1st question is: how can I monitor virtual machines in Ganglia. Especially > using libvirt, but later also Xen, VMware, Hyper-V. Is it possible? Can > ganglia monitor virtual machines? What can I monitor without additional agent > on guest and what can I monitor with such agent? > 2nd question: I've just found out gmetad-python, but before I'll try to use > it I'd like to ask about possibilities to use python scripts to collect data > from gmond. What is the easiest way to do that? > > I'm interested in python, because I'd like to try create simple Openstack and > Ganglia proof of concept. I was going to use existing simple monitoring > around libvirt from openstack and implement storing data in RRD files, but as > I've mentioned above, I don't want to reinvent the wheel and I think a better > way is to integrate already existing solutions. The question is how to > integrate those two projects. My first step is to run gmond on each host, > collect data from it via python script, save those data into RRD file and use > this data in Openstack's environment. If gmetad-python can do that, it's > great. Then second step will be to get that data from gmetad-python (via > python script) and use it in Openstack's environment. > > Cheers, > -- > Simon > > > -- > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. > http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ > Ganglia-developers mailing list > Ganglia-developers@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/ganglia-developers -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers
[Ganglia-developers] Ganglia libvirt and other virtualization technologies
Hello, Is there any way to use Ganglia to monitor virtual machines? Right now I'd like to get data from libvirt's api (CPU usage, IO, Network etc), but I don't want to reinvent the wheel and I'd like to use already mature tools to do that, for example Ganglia. I know collectd has plugins which support libvirt, but from what I've read, ganglia could be better solution. 1st question is: how can I monitor virtual machines in Ganglia. Especially using libvirt, but later also Xen, VMware, Hyper-V. Is it possible? Can ganglia monitor virtual machines? What can I monitor without additional agent on guest and what can I monitor with such agent? 2nd question: I've just found out gmetad-python, but before I'll try to use it I'd like to ask about possibilities to use python scripts to collect data from gmond. What is the easiest way to do that? I'm interested in python, because I'd like to try create simple Openstack and Ganglia proof of concept. I was going to use existing simple monitoring around libvirt from openstack and implement storing data in RRD files, but as I've mentioned above, I don't want to reinvent the wheel and I think a better way is to integrate already existing solutions. The question is how to integrate those two projects. My first step is to run gmond on each host, collect data from it via python script, save those data into RRD file and use this data in Openstack's environment. If gmetad-python can do that, it's great. Then second step will be to get that data from gmetad-python (via python script) and use it in Openstack's environment. Cheers, -- *Simon* -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers
