Dave, There is undoubtably fat in the XML. However, I think that you would get far more savings by giving the client a means to request which data they are interested in. Right now it sends everything each time, right? It would be nice if you could provide a regex for what you needed. One option would be to put up an http end point which allowed query parameters that describe the data of interest. I've done this in a similar system in Java. Not sure what options are out there in C for embedded httpd.
Bryan > -----Original Message----- > From: Dave Rawks [mailto:d...@pandora.com] > Sent: Thursday, January 26, 2012 2:34 PM > To: ganglia-developers@lists.sourceforge.net > Subject: [Ganglia-developers] Protocol Efficiency Ideas > > Hey All, > We've been talking about adding json in addition to xml > for the tcp listen port exchange format. And I was curious if > the "EXTRA_DATA" > subtree to the XML ever contains something aside from "EXTRA_ELEMENTS" > and if the "EXTRA_ELEMENTS" ever have attributes aside from > "NAME" and "VAL". > > Just doing some back of napkin calculations it looks like > reducing this portion of the xml from: > > <METRIC NAME="swap_free" VAL="47872928" TYPE="float" > UNITS="KB" TN="24" > TMAX="180" DMAX="0" SLOPE="both"> > <EXTRA_DATA> > <EXTRA_ELEMENT NAME="GROUP" VAL="memory"/> <EXTRA_ELEMENT > NAME="DESC" VAL="Amount of available swap memory"/> > <EXTRA_ELEMENT NAME="TITLE" VAL="Free Swap Space"/> > </EXTRA_DATA> </METRIC> > > to: > > <METRIC NAME="swap_free" VAL="47872928" TYPE="float" > UNITS="KB" TN="24" > TMAX="180" DMAX="0" SLOPE="both"> > <EXTRA_DATA GROUP="memory" DESC="Amount of available swap memory" > TITLE="Free Swap Space"/> > </METRIC> > > would be quite a savings over the wire. Of course this would > break compatibility with anything that currently exchanges > xml with ganglia monitor. But... That gets me back to json... > > The current data structure from xml to json is something like this: > > {metric:{name:'swap_free',val:47872928,type:'float',units:'KB' > ,tn:24,tmax:180,dmax:0,slope:'both',extra_data:{extra_element: > [{name:'GROUP',val:'memory'},{name:'DESC',val:'Amount > of available swap memory'},{name:'TITLE',val:'Free Swap Space'}]}}} > > While the collapsed version ends up being this tiny json blob: > > {metric:{name:'swap_free',val:47872928,type:'float',units:'KB' > ,tn:24,tmax:180,dmax:0,slope:'both',extra_data:{group:'memory' > ,desc:'Amount > of available swap memory',title:'Free Swap Space'}}} > > > On a relatively small cluster with a dozen metrics and a > handful of hosts the savings are minor. However on a cluster > of hundreds of hosts with perhaps dozens of metrics the > savings would equate to MBs of data per tcp fetch. And the > parse speed of the json /should/ be much faster as well. > > Any comments/questions/ideas? > > -Dave > > -------------------------------------------------------------- > ---------------- > Keep Your Developer Skills Current with LearnDevNow! > The most comprehensive online learning library for Microsoft > developers is just $99.99! Visual Studio, SharePoint, SQL - > plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future > releases when you subscribe now! > http://p.sf.net/sfu/learndevnow-d2d > _______________________________________________ > Ganglia-developers mailing list > Ganglia-developers@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/ganglia-developers > ------------------------------------------------------------------------------ Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d _______________________________________________ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers
