Chris,
I am facing the exact issue but I chose a different approach:
1. I added support for "custom time range" in the web frontend. While
I don't have 1-hour granularity a year back, I doubt we'd need to
debug year-old (or even month-old) NFS issues. This kind of
information is best served hot.
2. I've added gmetrics showing NFS ops.
3. I'm working on an interface which would allow generating "custom
graphs". Here the user would choose whatever metrics she wants in
the graph, whether builtin gmetrics, and any CDEF-based metrics,
and a graph would be generated based on that information. I'm
implementing this for cluster/grid views, as this is where I
believe the information would be most relevant.
Another helpful feature would be an interface grouping a list of select
hosts (logical cluster, if you will) and displaying the requested
metrics for them. I'll look into this once I'm done implementing custom
graphs.
I think the main question here is "would I need to debug month-old
problems?"
Cheers,
Alex
Chris Croswhite wrote:
> Alex et al,
>
> Yeah, I am rethinking what stats I acutally need to keep. The issue
> here is to be able to go back to a particular period of time and try to
> understand what hosts were doing at that given period in hopes that it
> will point to a potential problem (things like what is beating up the
> NFS file servers at X time of the day on this week).
>
> I am currently rethinking what might be the best least amount of data.
>
> Thanks for the suggestions.
>
> Chris
> On Thu, 2006-02-23 at 08:48, Alex Balk wrote:
> Hi,
>
> Keep in mind that you'd also need modify the web frontend, which may
> more of a headache than you think.
>
> It current relies on rrdtool for generating the graphs.
>
> If disk capacity and update time is not an issue, perhaps you could
> just change the granularity of data stored in the RRDs? I'm don't know
> how long back you want to keep 1-hour stats (or 1-minute stats, but
> then you're just sick ;-) ), but if you go down this path you could
> end up with pretty huge RRD files, so it's not a "cool, lets do that"
> kind of solution.
>
>
> Cheers,
>
> Alex
>
> Chris Croswhite wrote:
>
> >>> Matin,
> >>>
> >>> Hmm, that might work for what I am looking to do. Thanks for the
> >>> suggestion. Though, I think (still pondering), for the long term, I
> >>> would like to see a write out naitively to a db (I think....).
> >>>
> >>> Chris
> >>>
> >>> On Wed, 2006-02-22 at 23:30, Martin Knoblauch wrote:
> >>>> Arjun,
> >>>>
> >>>> instead of hacking "gmetad", another option could be to just write a
> >>>> script/program that queries a running "gmetad" tcp-port and feed the
> >>>> data from the received XML stream into your DB. That way you could
> >>>> avoid touching the existing ganglia code alltogether.
> >>>>
> >>>> Just for curiosity, how would you use the data in the DB? Any
> initial
> >>>> ideas about the table layout?
> >>>>
> >>>> Cheers
> >>>> Martin
> >>>>
> >>>> --- Arjun <[EMAIL PROTECTED]> wrote:
> >>>>
> >>>>> Martin,
> >>>>>
> >>>>> Thanks for the tip. Yes, it seems that the rrd_helpers.c file is
> >>>>> where I
> >>>>> will have to start.
> >>>>> I think that maybe we should have a separate helper file for MySql
> >>>>> insertion
> >>>>> (or any DB insertion)
> >>>>> and have that as a separate option for ganglia. So people will
> have a
> >>>>> choice
> >>>>> of where to store data.
> >>>>>
> >>>>> Yes, storing data in a DB will introduce the capacity issue
> which rrd
> >>>>> avoided but in my case
> >>>>> I need the data to be stored in a DB and capacity is not an issue.
> >>>>> For cases
> >>>>> where capacity is an issue,
> >>>>> it should be fairly easy to write a cron-type script for
> >>>>> truncating/deleting
> >>>>> data older than a particular date.
> >>>>>
> >>>>> I'll keep you updated on my progress in this area. Unfortunately I'm
> >>>>> new to
> >>>>> C programming so its slow going.
> >>>>> Do let me know if you have any further suggestions/ideas/advice.
> >>>>>
> >>>>> Thanks and regards
> >>>>> Arjun
> >>>>>
> >>>>> On 2/22/06, Martin Knoblauch <[EMAIL PROTECTED]> wrote:
> >>>>>> Arjun,
> >>>>>>
> >>>>>> only comments I have:
> >>>>>>
> >>>>>> - yes pleaase. Having the metrics stored in a "real" DB is
> >>>>> something I
> >>>>>> am very interested in. Although this may have some issues like
> >>>>>> capacity, being able to view/search 1-hour stats over the last year
> >>>>>> would be great.
> >>>>>>
> >>>>>> - modifying "gmetad" seems to be the most likely way. Have a look
> >>>>> at
> >>>>>> the stuff provided by the "rrd_helpers.c" source and how its used.
> >>>>>>
> >>>>>> Cheers
> >>>>>> Martin
> >>>>>>
> >>>>>> --- Arjun <[EMAIL PROTECTED]> wrote:
> >>>>>>
> >>>>>>> Chris,
> >>>>>>>
> >>>>>>> Sure thing. Will let you know if I get anything or write
> >>>>> anything.
> >>>>>>> Yes, I guess it will be a gmetad modification.
> >>>>>>>
> >>>>>>> In my case, I'm not interested in transferring data from RRD to
> >>>>> mysql
> >>>>>>> offline but
> >>>>>>> rather have gmetad write to the MySql DB directly instead of RRD.
> >>>>>>>
> >>>>>>> I think there are probably already some tools available for
> >>>>>>> transferring or
> >>>>>>> importing data
> >>>>>>> from RRD to MySql. If I come across any I'll let you know.
> >>>>>>>
> >>>>>>> Regards
> >>>>>>> Arjun
> >>>>>>>
> >>>>>>>
> >>>>>>> On 2/22/06, Chris Croswhite <[EMAIL PROTECTED]> wrote:
> >>>>>>>> it will be part of gmetad, I believe it writes out the data
> >>>>> into
> >>>>>>> the rrd
> >>>>>>>> db's.
> >>>>>>>>
> >>>>>>>> As another option, you could script out a dump call on all the
> >>>>> rrd
> >>>>>>> db's
> >>>>>>>> and plug the values into your sql table, but this would give
> >>>>> you
> >>>>>>> numbers
> >>>>>>>> you would have to translate (e.g. map the numbers to what they
> >>>>>>>> represent).
> >>>>>>>>
> >>>>>>>> I am VERY interested in getting the same functionality, so any
> >>>>> code
> >>>>>>> or
> >>>>>>>> information you get, could you please share with me. I will do
> >>>>> the
> >>>>>>>> same.
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> Chris
> >>>>>>>>
> >>>>>>>> On Wed, 2006-02-22 at 10:37, Arjun wrote:
> >>>>>>>>> Hi Chris,
> >>>>>>>>>
> >>>>>>>>> Any ideas on where (in the codebase) I can start ? I don't
> >>>>> mind
> >>>>>>> doing
> >>>>>>>>> the code modifications myself as long as I can get a
> >>>>> jumpstart.
> >>>>>>> (I'm a
> >>>>>>>>> bit pressed for time so I'd rather have an existing solution)
> >>>>>>>>>
> >>>>>>>>> regds
> >>>>>>>>> Arjun
> >>>>>>>>>
> >>>>>>>>> On 2/22/06, Chris Croswhite <[EMAIL PROTECTED]> wrote:
> >>>>>>>>> I am looking to do something like this as well.
> >>>>>>>>>
> >>>>>>>>> TIA,
> >>>>>>>>> Chris
> >>>>>>>>>
> >>>>>>>>> On Wed, 2006-02-22 at 10:02, Arjun wrote:
> >>>>>>>>> > Hi Folks,
> >>>>>>>>> >
> >>>>>>>>> > I have a requirement to integrate ganglia to a DB
> >>>>> such
> >>>>>>> as
> >>>>>>>>> MySql so
> >>>>>>>>> > that gmetad should store retreived data in
> >>>>>>>>> > MySql instead of an RRD.
> >>>>>>>>> >
> >>>>>>>>> > Has anyone already done this and if so can someone
> >>>>>>> please
> >>>>>>>>> point me to
> >>>>>>>>> > the code/docs etc ?
> >>>>>>>>> >
> >>>>>>>>> > If it has not been done then can someone guide as
> >>>>> to
> >>>>>>> where I
> >>>>>>>>> should
> >>>>>>>>> > start modifying code so that gmetad writes
> >>>>>>>>> > to MySql rather than RRD ? Some detailed insight
> >>>>> would
> >>>>>>> be
> >>>>>>>>> very
> >>>>>>>>> > welcome.
> >>>>>>>>> >
> >>>>>>>>> > Thanks and regards
> >>>>>>>>> > Arjun
> >>>>>>>>> > PennState University
> >>>>>>>>>
> >>>>>>>>>
> >>>>>> ------------------------------------------------------
> >>>>>> Martin Knoblauch
> >>>>>> email: k n o b i AT knobisoft DOT de
> >>>>>> www: http://www.knobisoft.de
> >>>>>>
> >>>> ------------------------------------------------------
> >>>> Martin Knoblauch
> >>>> email: k n o b i AT knobisoft DOT de
> >>>> www: http://www.knobisoft.de
> >>>>
> >>>>
> >>>> -------------------------------------------------------
> >>>> This SF.Net email is sponsored by xPML, a groundbreaking scripting
> language
> >>>> that extends applications into web and mobile media. Attend the live
> webcast
> >>>> and join the prime developer group breaking into this new coding
> territory!
> >>>>
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
> >>>> _______________________________________________
> >>>> Ganglia-developers mailing list
> >>>> Ganglia-developers@lists.sourceforge.net
> >>>> https://lists.sourceforge.net/lists/listinfo/ganglia-developers
> >>>
> >>>
> >>> -------------------------------------------------------
> >>> This SF.Net email is sponsored by xPML, a groundbreaking scripting
> language
> >>> that extends applications into web and mobile media. Attend the live
> webcast
> >>> and join the prime developer group breaking into this new coding
> territory!
> >>>
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
> >>> _______________________________________________
> >>> Ganglia-developers mailing list
> >>> Ganglia-developers@lists.sourceforge.net
> >>> https://lists.sourceforge.net/lists/listinfo/ganglia-developers
>>
