>>> On 10/7/2010 at 3:56 PM, in message <6b7a9ac5-465d-4311-ac85-fd42c219d...@inmon.com>, Peter Phaal <peter.ph...@inmon.com> wrote: > Brad, > > Thanks for the feedback. My comments are in-line. > > On Oct 7, 2010, at 12:27 PM, Brad Nicholes wrote: > >> Sorry to jump into this thread so late but I thought that I would throw my 2 > cents in. >> >> I finally got a chance to take a look at the code. I was able to compile it > but ran into some C99 issues with variable declarations. Once I got the code > to compile, I was able to take a closer look at what it was doing. From what > I could tell, it looks like the sflow integration is based around reading XDR > packets from an sflow agent and turning them into gmond spoofing metrics. My > first question after seeing this is why does this code have to be built into > gmond.c? Why can't it just do the same thing in a module that would be > plugged into gmond? >> >> The reason why I ask this is because we went to a lot of work to pull all of > the metric gathering out of gmond and into modules (including all of the > standard metrics). Some of the main reasons for this is so that metric > gathering could be pluggable without having to affect the gmond code itself. > That way if a bug was ever found and fixed for a specific metrics, we > wouldn't have to re-release all of Ganglia just for one metric fix. Also, > modules give the user the ability to customize each gmond agent to conform to > the specific needs of the node where gmond is running. Regarding sflow, it > seems that in order to integrate the sflow metrics into the Ganglia > monitoring system, only a single gmond node needs to be configured to gather > the sflow metrics. All of the other gmond agents can continue to be > configured and run as they were. Given that, it would make more sense to > integrate sflow as a module that could be loaded under a single gmond agent > rather than replacing all the gmond agents or even upgrading just a single > agent. It would also seem to follow the way that other metric modules and > spoofing modules have been implemented as well. > > > I am not very familiar with gmod modules, but it looks like they are > designed around a polling model and used to retrieve metrics from the server > that the particular instance of gmond is running on: > 1. a module is loaded in the modules section of the gmond.conf file and > registers a set of metrics it can provide > 2. metrics are then included in collection_group sections and polled at the > specified intervals > > With sFlow, the counters are being pushed by remote servers. There may be > hundreds of sFlow agents sending XDR packets to the single gmond instance. > Our code acts as a gateway, translating the metrics from the remote hosts and > presenting them as if they had arrived in the form of Ganglia XDR datagrams > from remote gmond instances. This function needs to be part of the main > datagram processing loop. I don't see a way for a module to inject code into > the packet processing loop(?) >
The way that I would envision an sflow module working would be similar to the spoofing example module that is currently checked into the Ganglia SVN repository. The spoofing module can be found at http://ganglia.svn.sourceforge.net/viewvc/ganglia/trunk/monitor-core/gmond/python_modules/example/spfexample.py?revision=1895&view=markup . Unfortunately it is a python module example rather than a C module but hopefully you can get the idea of what I am talking about from the code. One application of this kind of spoofing module would be to load it under a gmond instance running on a VM host. It would then query each VM running on the box and register a set of spoofed metrics for each VM. From that point on, the module just reports the metrics for each spoofed VM and returns them as if gmond were running on each of the VMs. I actually have another python module that does exactly that, but I haven't been able to release the source code for it yet. You can also look at the modpython.c module to get an idea of how to do the spoofing in C code. But then you guys have already worked with the spoofing code as part of the patch that you already did so you probably already know how that works. Basically an sflow module would be loaded like any other module and a collection interval would be set in the configuration file. In the sflow module itself, register a spoofed metric for each managed sflow monitored node. How you get the list of nodes to register is up to you. It could be from the gmond.conf file, some other configuration file or by listening to the sflow data packets themselves. The module would then start a thread that would read the XDR packets in exactly the same way that you are doing now in the gmond code. The thread would then store the current metric data temporarily for each monitored node that it knows about until the next collection cycle happens. Then when gmond requests specific metric data for a specific spoofed node, you just return the last metric read from your temporary storage. Hopefully this helps. The whole intent of the modular interface is to move all of the metric gathering out of gmond itself in order to make gmond more flexible and upgradable for the user without having to depend on all of us as Ganglia developers to get around to releasing a new version every time a bug is found or someone wants to collect a metric in a different way. By implementing sflow as a module rather than within gmond itself, it would help to make sure gmond remains flexible and upgradable for the user. Brad ------------------------------------------------------------------------------ Beautiful is writing same markup. Internet Explorer 9 supports standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3. Spend less time writing and rewriting code and more time creating great experiences on the web. Be a part of the beta today. http://p.sf.net/sfu/beautyoftheweb _______________________________________________ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers
