Interface for SGE has already been developed by Rocks (part of the SGE
Roll). Rocks also has interface for PBS within PBS Roll. It's written in
Python. However, Rocks's interfaces are implemented as part of their
greceptor service and they don't support archiving. I think that their
SGE interface can be easily integrated in your part. I'll try to do that
and get back to you ...
Also, a while ago I extended Ganglia job web interface so that I can
gather information from 3 types of interface: Rocks SGE, Rocks PBS and
Sara PBS (which I use on OSCAR clusters).
Cheers,
emir
Martin Knoblauch wrote:
Ramon,
if I had the time, I could do some development for it on one of our
SGE clusters. Testing is definitely possible.
LSF support would also be great.
Martin
--- Ramon Bastiaans <[EMAIL PROTECTED]> wrote:
Hi Bernard,
It seems there is a Python module available for DRMAA, here:
http://gridengine.sunsource.net/servlets/ProjectDocumentList?folderID=15&expandFolder=15&folderID=15
But I have no SGE/DRMAA system to test/develop on. ;)
Any patches/contributions are welcome, so if you would be able to
test/develop something let me know.
Or maybe I could try to find/get access to some SGE system, who
knows.
Cheers,
- Ramon.
Bernard Li wrote:
Hi Ramon:
SGE uses the DRMAA interface, which I believe someone is working on
bindings for TORQUE, so if you can modify your tool to support
DRMAA,
then it will be easy enough to support both:
http://www.drmaa.org/
Perhaps I'll take a look at implementing support for SGE later on.
I
will keep you posted.
Cheers,
Bernard
------------------------------------------------------------------------
*From:* Ramon Bastiaans [mailto:[EMAIL PROTECTED]
*Sent:* Sat 11/03/2006 06:28
*To:* Bernard Li
*Cc:* Ganglia General; Ganglia Developers
*Subject:* Re: [Ganglia-developers] ANNOUNCE: Public release of
Ganglia Job Monarch v0.1.0
Hi Bernard,
I have no experience with SGE but if I could find a
good/appropriate
interface to the SGE libraries for Python, I would certainly be
willing
to add support for it. Should you be aware of or find such a
package let
me know.
Cheers,
- Ramon.
Bernard Li wrote:
Hi Ramon:
Have you thought of supporting other workload management systems
such
as Sun Grid Engine (SGE)?
Cheers,
Bernard
------------------------------------------------------------------------
*From:* [EMAIL PROTECTED] on behalf
of
Ramon Bastiaans
*Sent:* Fri 10/03/2006 09:33
*To:* [EMAIL PROTECTED]; Ganglia General
*Cc:* Ganglia Developers
*Subject:* [Ganglia-developers] ANNOUNCE: Public release of
Ganglia
Job Monarch v0.1.0
This is the first initial and public open source release of:
"Ganglia Job Monarch", the Job Monitoring and Archiving tool
and is
a addon to Ganglia.
DOWNLOAD
==========
This release is: ganglia_jobmonarch-0.1.0
It is available here:
ftp://ftp.sara.nl/pub/outgoing/ganglia_jobmonarch.tar.gz
See the INSTALL file on how to set it up.
DESCRIPTION
===========
Job Monarch is a set of tools to monitor and optionally
archive
(batch)job information.
It is a addon for the Ganglia monitoring system and plugs in
to a
existing Ganglia setup.
To view a operational setup with Job Monarch, have a look
here:
http://ganglia.sara.nl/
Job Monarch stands for 'Job Monitoring and Archiving' tool
and
consists of three (3) components:
* jobmond
The Job Monitoring Daemon.
Gathers PBS/Torque batch statistics on jobs/nodes and
submits
them into
Ganglia's XML stream.
Through this daemon, users are able to view the
PBS/Torque batch
system and the
jobs/nodes that are in it (be it either running or
queued).
* jobarchived (optionally)
The Job Archiving Daemon.
Listens to Ganglia's XML stream and archives the job and
node
statistics.
It stores the job statistics in a Postgres SQL database
and the
node statistics
in RRD files.
Through this daemon, users are able to lookup a
old/finished job
and view all it's statistics.
Optionally: You can either choose to use this daemon if
your
users have use for it.
As it can be a heavy application to run and not everyone
may
have a need for it.
- Multithreaded: Will not miss any data regardless of
(slow)
storage
- Staged writing: Spread load over bigger time periods
- High precision RRDs: Allow for zooming on old
periods with
large precision
- Timeperiod RRDs: Allow for smaller number of files
while
still keeping advantage of small disk space
* web
The Job Monarch web interface.
This interfaces with the jobmond data and (optionally)
the
jobarchived and presents the
data and graphs.
It does this in a similar layout/setup as Ganglia itself,
so the
navigation and usage is intuitive.
- Graphical usage: Displays graphical cluster overview
so you
can see the cluster (job) state
in one view/image and additional pie chart
with
relevant information on your
current view
- Filters: Ability to filter output to limit
information
displayed (usefull for those
clusters with 500+ jobs). This also filters
the
graphical overview images output
and pie chart so you only see the filter
relevant data
- Archive: When enabling jobarchived, users can go
back
as far as recorded in the database
or archived RRDs to find out what happened to
a
crashed or old job
- Zoom ability: Users can zoom into a timepriod as
small
as the smallest grain of the RRDS
(typically up to 10 seconds) when a
jobarchived is
present
EXAMPLE
========
You can view a operational Ganglia Job Monarch setup here:
http://ganglia.sara.nl/
CONTACT
========
Any information/suggestions/hatemail/bugreports/whatever to:
Ramon Bastiaans
<bastiaans ( a t ) sara ( d o t ) nl>
-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking
scripting
language
that extends applications into web and mobile media. Attend the
live
webcast
and join the prime developer group breaking into this new coding
territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
<http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642>
<http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
<http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642>>
_______________________________________________
Ganglia-developers mailing list
Ganglia-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ganglia-developers
--
.--------------------------------------------.
| ing. Ramon Bastiaans |
| HPC - Systems Programmer |
|--------------------------------------------|
| SARA - Computing and Networking Services |
| Kruislaan 415 PO Box 194613 |
| 1098 SJ Amsterdam 1090 GP Amsterdam |
|--------------------------------------------|
| Mail: bastiaans ( a t ) sara ( d o t ) nl |
| Web: http://www.sara.nl/ |
| Phone: +31 (0)20 592 80 19 |
| Fax: +31 (0)20 668 31 67 |
`--------------------------------------------'
-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting
language
that extends applications into web and mobile media. Attend the live
webcast
and join the prime developer group breaking into this new coding
territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
_______________________________________________
Ganglia-developers mailing list
Ganglia-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ganglia-developers
------------------------------------------------------
Martin Knoblauch
email: k n o b i AT knobisoft DOT de
www: http://www.knobisoft.de
-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
_______________________________________________
Ganglia-developers mailing list
Ganglia-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ganglia-developers
