I think a JSON option would make the gmond data more accessible. If the hood is
going to be open on the transport, an HTTP option would further open up access
to the data. HTTP would provide a way to pass query arguments and return
specific metrics, or metrics for a particular host, tag etc. I don't thing a
full blown web server would be necessary, just a simple embedded HTTP parser,
e.g. https://github.com/joyent/http-parser
Currently Ganglia has a strict containment hierarchy. I can imagine
environments, large virtual machine pools for example, where you might want to
cluster hosts based on their attributes, or create logical clusters for
different services rather than cluster based on the gmond instance that happens
to be collecting the data. Filtering in gmond is more efficient that having to
retrieve the entire database and filter at the receiver.
Another point worth considering. It is my understanding that gmond is single
threaded. Locking up the daemon while it exports a large xml document risks
losing data. A json document would be more compact and improve scalability
while filtered queries would further reduce the time needed to export data on a
large site.
Peter
On Jan 27, 2012, at 2:59 PM, Vladimir Vuksan wrote:
> I don't get it. JSON is a notation that has nothing to do with Linux.
>
> I think addition of JSON will be fantastic and look forward to including it.
>
> Vladimir
>
> On Fri, 27 Jan 2012, Im Root wrote:
>
>> I forgot to add that by adding json, you will be restricting the types of
>> Linux that this platform runs on. Although
>> it may be a "nice to have" feature for a few developers who may want to
>> customize things, this puts an additional
>> dependency on the platform. You will be restricted to using only the flavors
>> of linux that the keepers of json feel
>> like using. Sorry, but adding the json dependency is a completely boneheaded
>> move.
>> ____________________________________________________________________________________________________________________
>> From: Im Root <imr...@rocketmail.com>
>> To: Dave Rawks <d...@pandora.com>; "ganglia-developers@lists.sourceforge.net"
>> <ganglia-developers@lists.sourceforge.net>
>> Sent: Friday, January 27, 2012 9:59 AM
>> Subject: Re: [Ganglia-developers] Protocol Efficiency Ideas
>> I believe that adding json would be a mistake. The reason is that when users
>> install the main package there would be
>> now a dependency on having json installed. It just adds to the complexity
>> and helps to perpetuate RPM hell. I've had
>> to deal with installing json in the past and it's been awful. It may be nice
>> for a developer but not so nice for the
>> end users.
>> ____________________________________________________________________________________________________________________
>> From: Dave Rawks <d...@pandora.com>
>> To: ganglia-developers@lists.sourceforge.net
>> Sent: Thursday, January 26, 2012 2:34 PM
>> Subject: [Ganglia-developers] Protocol Efficiency Ideas
>> Hey All,
>> We've been talking about adding json in addition to xml for the tcp
>> listen port exchange format. And I was curious if the "EXTRA_DATA"
>> subtree to the XML ever contains something aside from "EXTRA_ELEMENTS"
>> and if the "EXTRA_ELEMENTS" ever have attributes aside from "NAME" and
>> "VAL".
>> Just doing some back of napkin calculations it looks like reducing this
>> portion of the xml from:
>> <METRIC NAME="swap_free" VAL="47872928" TYPE="float" UNITS="KB" TN="24"
>> TMAX="180" DMAX="0" SLOPE="both">
>> <EXTRA_DATA>
>> <EXTRA_ELEMENT NAME="GROUP" VAL="memory"/>
>> <EXTRA_ELEMENT NAME="DESC" VAL="Amount of available swap memory"/>
>> <EXTRA_ELEMENT NAME="TITLE" VAL="Free Swap Space"/>
>> </EXTRA_DATA>
>> </METRIC>
>> to:
>> <METRIC NAME="swap_free" VAL="47872928" TYPE="float" UNITS="KB" TN="24"
>> TMAX="180" DMAX="0" SLOPE="both">
>> <EXTRA_DATA GROUP="memory" DESC="Amount of available swap memory"
>> TITLE="Free Swap Space"/>
>> </METRIC>
>> would be quite a savings over the wire. Of course this would break
>> compatibility with anything that currently exchanges xml with ganglia
>> monitor. But... That gets me back to json...
>> The current data structure from xml to json is something like this:
>> {metric:{name:'swap_free',val:47872928,type:'float',units:'KB',tn:24,tmax:180,dmax:0,slope:'both',extra_data:{extra
>> _element:[{name:'GROUP',val:'memory'},{name:'DESC',val:'Amount
>> of available swap memory'},{name:'TITLE',val:'Free Swap Space'}]}}}
>> While the collapsed version ends up being this tiny json blob:
>> {metric:{name:'swap_free',val:47872928,type:'float',units:'KB',tn:24,tmax:180,dmax:0,slope:'both',extra_data:{group
>> :'memory',desc:'Amount
>> of available swap memory',title:'Free Swap Space'}}}
>> On a relatively small cluster with a dozen metrics and a handful of
>> hosts the savings are minor. However on a cluster of hundreds of hosts
>> with perhaps dozens of metrics the savings would equate to MBs of data
>> per tcp fetch. And the parse speed of the json /should/ be much faster
>> as well.
>> Any comments/questions/ideas?
>> -Dave
>> ------------------------------------------------------------------------------
>> Keep Your Developer Skills Current with LearnDevNow!
>> The most comprehensive online learning library for Microsoft developers
>> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
>> Metro Style Apps, more. Free future releases when you subscribe now!
>> http://p.sf.net/sfu/learndevnow-d2d
>> _______________________________________________
>> Ganglia-developers mailing list
>> Ganglia-developers@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/ganglia-developers
>> ------------------------------------------------------------------------------
>> Try before you buy = See our experts in action!
>> The most comprehensive online learning library for Microsoft developers
>> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
>> Metro Style Apps, more. Free future releases when you subscribe now!
>> http://p.sf.net/sfu/learndevnow-dev2
>> _______________________________________________
>> Ganglia-developers mailing list
>> Ganglia-developers@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/ganglia-developers
> ------------------------------------------------------------------------------
> Try before you buy = See our experts in action!
> The most comprehensive online learning library for Microsoft developers
> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
> Metro Style Apps, more. Free future releases when you subscribe now!
> http://p.sf.net/sfu/learndevnow-dev2_______________________________________________
> Ganglia-developers mailing list
> Ganglia-developers@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/ganglia-developers
------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
_______________________________________________
Ganglia-developers mailing list
Ganglia-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ganglia-developers
