-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 guys-
i have a new demo for you to take a look at http://matt-massie.com/g3/ganglia-3.0.0.tar.gz % gunzip < ganglia-3.0.0.tar.gz % cd ganglia-3.0.0 % ./configure && make % cd apps in the "apps" directory you'll see an application called "gfilter". % ./gfilter --help gfilter 3.0.0 Purpose: An example application for g3 filtering Usage: gfilter [OPTIONS]... -h --help Print help and exit -V --version Print version and exit -fSTRING --data_file=STRING XML file containing data to filter -dSTRING --data_source=STRING I.P. address of gmond/gmetad (default='127.0.0.1') -pINT --data_port=INT Port number to connect to (default=8649) -oSTRING --ou_filter=STRING Regex for organizational units (default='.*') -mSTRING --mu_filter=STRING Regex for metric units (default='.*') this is a simple program that will work on both ganglia 2 and ganglia 3 xml streams.. however .. it'll only output ganglia 3 streams. gfilter collects data from a local file or a network socket and outputs g3 xml based on the filters you specify. if you want to use a local file.. i put some samples in the distribution.. (in ./tests there is sample1.xml and sample2.xml). here are some samples... % ./gfilter -f ../tests/sample2.xml -o "/Webserver Cluster/" -m "/cpu_num" <?xml version="1.0" encoding="US-ASCII" standalone="yes"?> <ganglia_root version="3.0.0"> <ou id="WebServer Cluster"> <host id="touch.Millennium.Berkeley.EDU"> <metric id="cpu_num" value="1" type="number" units=""/> </host> <host id="see.Millennium.Berkeley.EDU"> <metric id="cpu_num" value="1" type="number" units=""/> </host> <host id="smell.Millennium.Berkeley.EDU"> <metric id="cpu_num" value="1" type="number" units=""/> </host> <host id="taste.Millennium.Berkeley.EDU"> <metric id="cpu_num" value="1" type="number" units=""/> </host> <metric id="cpu_num" value="4" type="number" samples="4"/> </ou> </ganglia_root> sample2.xml is a real g2 xml file collected from our machines here at berkeley. one of the groups of machines is the webserver cluster.. as you can see... it's made up of 4 machines with 1 processor each. you can see the last metric entry is really a summary entry since it's inside the <ou> and outside of eny hosts. the regular expressions are extended regular expressions that are not case-sensitive. say we want to learn more about one of the web server machines % ./gfilter -f ../tests/sample2.xml -o "/Webserver Cluster/taste" -m "/cpu_num" <?xml version="1.0" encoding="US-ASCII" standalone="yes"?> <ganglia_root version="3.0.0"> <ou id="WebServer Cluster"> <host id="taste.Millennium.Berkeley.EDU"> <metric id="cpu_num" value="1" type="number" units=""/> </host> <metric id="cpu_num" value="4" type="number" samples="4"/> </ou> </ganglia_root> or.. let's say we want to see all the machines that start with "t" (silly example i know). % ./gfilter -f ../tests/sample2.xml -o "/Webserver Cluster/t" -m "/cpu_num" <?xml version="1.0" encoding="US-ASCII" standalone="yes"?> <ganglia_root version="3.0.0"> <ou id="WebServer Cluster"> <host id="touch.Millennium.Berkeley.EDU"> <metric id="cpu_num" value="1" type="number" units=""/> </host> <host id="taste.Millennium.Berkeley.EDU"> <metric id="cpu_num" value="1" type="number" units=""/> </host> <metric id="cpu_num" value="4" type="number" samples="4"/> </ou> </ganglia_root> since it's extended regular expressions.. we can look at one or more groups of machines/clusters/grids etc... glued together with |. e.g. Say I want CPU info about our web server cluster and one machine from our CITRIS cluster. % ./gfilter -f ../tests/sample2.xml -o "/Webserver Cluster/|/Citris.*/lemon" -m "/cpu_num" <?xml version="1.0" encoding="US-ASCII" standalone="yes"?> <ganglia_root version="3.0.0"> <ou id="WebServer Cluster"> <host id="touch.Millennium.Berkeley.EDU"> <metric id="cpu_num" value="1" type="number" units=""/> </host> <host id="see.Millennium.Berkeley.EDU"> <metric id="cpu_num" value="1" type="number" units=""/> </host> <host id="smell.Millennium.Berkeley.EDU"> <metric id="cpu_num" value="1" type="number" units=""/> </host> <host id="taste.Millennium.Berkeley.EDU"> <metric id="cpu_num" value="1" type="number" units=""/> </host> <metric id="cpu_num" value="4" type="number" samples="4"/> </ou> <ou id="CITRIS Itanium II Cluster"> <host id="lemon.Millennium.Berkeley.EDU"> <metric id="cpu_num" value="2" type="number" units=""/> </host> <metric id="cpu_num" value="2" type="number"/> </ou> </ganglia_root> .. so now i know that we have 4 CPUs in the web server cluster and 2 CPUs on the single machine in the CITRIS cluster. cool.. but say I want a total of all the totals... i would wrap this request inside of a virtual ou. % ./gfilter -f ../tests/sample2.xml -o "/Webserver Cluster/|/Citris.*/lemon" -m "/cpu_num" -v "gfilter query group" <?xml version="1.0" encoding="US-ASCII" standalone="yes"?> <ganglia_root version="3.0.0"> <ou id="gfilter query group"> <ou id="WebServer Cluster"> <host id="touch.Millennium.Berkeley.EDU"> <metric id="cpu_num" value="1" type="number" units=""/> </host> <host id="see.Millennium.Berkeley.EDU"> <metric id="cpu_num" value="1" type="number" units=""/> </host> <host id="smell.Millennium.Berkeley.EDU"> <metric id="cpu_num" value="1" type="number" units=""/> </host> <host id="taste.Millennium.Berkeley.EDU"> <metric id="cpu_num" value="1" type="number" units=""/> </host> <metric id="cpu_num" value="4" type="number" samples="4"/> </ou> <ou id="CITRIS Itanium II Cluster"> <host id="lemon.Millennium.Berkeley.EDU"> <metric id="cpu_num" value="2" type="number" units=""/> </host> <metric id="cpu_num" value="2" type="number"/> </ou> <metric id="cpu_num" value="6" type="number" samples="8"/> </ou> </ganglia_root> now all the data is wrapped inside a "gfilter query group" organizational unit. now we can see at the end that we have 6 CPUs total in this group. we can also link together metrics with |. % ./gfilter -f ../tests/sample2.xml -o "/Webserver Cluster/|/Citris.*/lemon" -m "/cpu_num|/load_one" -v "gfilter query group" <?xml version="1.0" encoding="US-ASCII" standalone="yes"?> <ganglia_root version="3.0.0"> <ou id="gfilter query group"> <ou id="WebServer Cluster"> <host id="touch.Millennium.Berkeley.EDU"> <metric id="load_one" value="0.58" type="float" units=""/> <metric id="cpu_num" value="1" type="number" units=""/> </host> <host id="see.Millennium.Berkeley.EDU"> <metric id="load_one" value="0.36" type="float" units=""/> <metric id="cpu_num" value="1" type="number" units=""/> </host> <host id="smell.Millennium.Berkeley.EDU"> <metric id="load_one" value="0.68" type="float" units=""/> <metric id="cpu_num" value="1" type="number" units=""/> </host> <host id="taste.Millennium.Berkeley.EDU"> <metric id="load_one" value="0.16" type="float" units=""/> <metric id="cpu_num" value="1" type="number" units=""/> </host> <metric id="load_one" value="177999999999999999992E1" type="float" samples="4"/> <metric id="cpu_num" value="4" type="number" samples="4"/> </ou> <ou id="CITRIS Itanium II Cluster"> <host id="lemon.Millennium.Berkeley.EDU"> <metric id="load_one" value="0.0" type="float" units=""/> <metric id="cpu_num" value="2" type="number" units=""/> </host> <metric id="load_one" value="0.0" type="float"/> <metric id="cpu_num" value="2" type="number"/> </ou> <metric id="load_one" value="177999999999999999992E1" type="float" samples="8"/> <metric id="cpu_num" value="6" type="number" samples="8"/> </ou> </ganglia_root> now we know that overall.. we have 6 CPUs with a 1-minute load of 1.78. i'll be adding two more features to the filtering next... a maximum depth number (i.e. only recurse down n levels) and the ability to display only summary records, detail records or both. once i finish that.. i'm going to call the filtering side finished (for now). this basic filtering will probably be enough to get a strong 3.0.0 release out. later, we can write a value processor to handle the values we receive after filtering.. (e.g. only print records with a value > 4). that will come later. again.. i really want to get g3 out the door soon. i hope you guys have a kickaz weekend - -matt -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.7 (GNU/Linux) iD8DBQE+6joeVmIXr0CKtmERAoTVAKCCErpRwhXfnim63cXgTOqKNkkuSACdE8Dv Al1/dnck/nWfNO+ORtL8DDY= =5CFJ -----END PGP SIGNATURE-----
