Perhaps not a new thought, but a simple DBM (or NoSQL I suppose) database
to store job accounting information could work really well here. I'm not
suggesting that SGE do that natively (although that'd be cool too),
but a simple script to post-process the logs as they come in could work
really well.
On Fri, Jan 23, 2015 at 08:23:46AM +0000, James Abbott wrote:
Do you rotate your accounting file at all? qacct performance suffers
badly when the accounting file gets too big. I drop a file containing
the following into /etc/logrotate.d on the qmaster (replacing the path
to the accounting file to it's location on the system...) :
/opt/sge/current/bss/common/accounting {
compress
nocreate
dateext
delaycompress
ifempty
# keep logs essentually "forever"
rotate 5000
monthly
}
which gives me a date-stamped, gzipped monthly accounting file, so
qacct continues to perform in a sensible way.
James
On 23/01/15 01:55, David Chin wrote:
Dear GE users:
Is it possible to have an epilog script which prints a summary of
resource usage (wallclock time, mem, cpu-seconds)?
I have tried an epilog bash script which calls qacct, but it took a
long time to generate that output.
Thanks,
Dave Chin
--
David Chin, Ph.D.
[email protected] <mailto:[email protected]> Sr. Systems
Administrator, URCF, Drexel U.
http://www.drexel.edu/research/urcf/ https://linuxfollies.blogspot.com/
215.221.4747 (mobile) https://github.com/prehensilecode
_______________________________________________
users mailing list
[email protected]
https://gridengine.org/mailman/listinfo/users
--
Dr. James Abbott
Lead Bioinformatician
Bioinformatics Support Service
Imperial College, London
_______________________________________________
users mailing list
[email protected]
https://gridengine.org/mailman/listinfo/users
--
Jesse Becker (Contractor)
_______________________________________________
users mailing list
[email protected]
https://gridengine.org/mailman/listinfo/users
