We just use the Splunk forwarder to read the reporting file as it's generated (turns out I mis-spoke - we only forward reporting since it's a superset of the accounting log), classify it with a custom source type, and then do automatic field extraction on the Splunk server. Splunk makes this easy since it's just a colon-delimited file, and we assign names to each field based on the reporting(5) man page.
Some of the more general dashboards we have: * Job start delay (difference between submission and start times) * Average job wallclock time * Number of successful/failed jobs On Thu, Feb 21, 2019 at 02:18:45PM +1100, Derrick Lin wrote: > Hi Skylar, > > Thanks for the info. I will look into the way you suggested. > > BTW, can you tell me more about sending the accounting and reporting files > to Splunk? We also want to do some dashboards too. > > Cheers, > > On Thu, Feb 21, 2019 at 3:20 AM Skylar Thompson <skyl...@u.washington.edu> > wrote: > > > We actually don't have a shared $SGE_ROOT, so that in the event of > > network/storage trouble the binaries and libraries are still accessible. We > > do have $SGE_ROOT/$SGE_CELL on network storage via a symlink, so the > > accounting and reporting files are accessible. We also send the accounting > > and reporting files to Splunk so we can generate some dashboards for our > > useres. > > > > On Wed, Feb 20, 2019 at 10:12:08AM +0000, Tina Friedrich wrote: > > > I was about to ask that :) > > > > > > $SGE_ROOT ought to be accessible from (the) submit host(s), at least. So > > > in general, you should be able to access it from there? > > > > > > (Note that you can also tell qacct where the accounting file lives - it > > > assumes a default location, but the file does not have be in that > > location.) > > > > > > Tina > > > > > > On 20/02/2019 07:09, Reuti wrote: > > > > Hi, > > > > > > > >> Am 20.02.2019 um 05:31 schrieb Derrick Lin <klin...@gmail.com>: > > > >> > > > >> Hi guys, > > > >> > > > >> On our SGE cluster, the accounting file stored on the qmaster node > > and is not accessible outside. qmaster node is not accessible by any user > > either. > > > >> > > > >> Now we have users request to obtain accounting info via qacct. I am > > wondering what is the common way to achieve this without giving access to > > the qmaster node? > > > > > > > > You mean, $SGE_ROOT is not shared in your cluster? > > > > > > > > -- Reuti > > > > > > > > > > > > _______________________________________________ > > > > users mailing list > > > > users@gridengine.org > > > > https://gridengine.org/mailman/listinfo/users > > > > > > > > > > _______________________________________________ > > > users mailing list > > > users@gridengine.org > > > https://gridengine.org/mailman/listinfo/users > > > > -- > > -- Skylar Thompson (skyl...@u.washington.edu) > > -- Genome Sciences Department, System Administrator > > -- Foege Building S046, (206)-685-7354 > > -- University of Washington School of Medicine > > _______________________________________________ > > users mailing list > > users@gridengine.org > > https://gridengine.org/mailman/listinfo/users > > -- -- Skylar Thompson (skyl...@u.washington.edu) -- Genome Sciences Department, System Administrator -- Foege Building S046, (206)-685-7354 -- University of Washington School of Medicine _______________________________________________ users mailing list users@gridengine.org https://gridengine.org/mailman/listinfo/users
