Hi, Am 13.01.2012 um 01:03 schrieb Michael Coffman:
> We are in the process of developing a gridwatcher utility that is launched > in the background from the prolog script. The intent is to have a > process monitor various aspects of the job and store or report on them. this is of course an interesting goal. What are you missing right now? > It currently determines the pid of the shepherd process then watches all > the children processes. I think it's easier to use the additional group ID, which is attached to all kids by SGE, whether they jump out of the process tree or not. This one is recorded in $SGE_JOB_SPOOL_DIR in the file "addgrpid". > Initially it will be watching memory usage and if a job begins using more > physical memory than requested, the user will be notified. That's where > my question comes from. What about setting a soft limit for h_vmem and prepare the job script to handle ithe signal to send an email. How will they request memory - by virtual_free? > Is there any way in the prolog to get access to the hard_request options > besides using qstat? > > What I'm currently doing: > > cmd = "bash -c '. #{@sge_root}/default/common/settings.sh && qstat > -xml -j #{@number}'" > > I have thought of possibly setting an environment variable via a jsv script > that can be queried by the prolog script. Is this a good idea? How much > impact > on submission time does jsv_send_env() add? You can use either a JSV or a `qsub` wrapper for it. > Any one else doing anything like this have any suggestions? > > > The end goal is to have a utility that users can also interact with to > monitor their jobs. By either setting environment variables or grid > complexes Complexes are only handled internally by SGE. There is no user command to change them for a non-admin. > to affect the behavior of what is being watched and how they > are notified. AFAIK you can't change the content of an already inherited variable, as the process got a copy of the value. Also /proc/12345/environ is only readable. And your "observation daemon" will run on all nodes - one for each job from the prolog if I get you right? But a nice solution could be the usage of the job context. This can be set by the user on the command line, and your job can access this by issuing a similar command like you did already. If the exechosts are submit hosts, the job can also change this by using `qalter` like the user has to use on the command line. We use the job context only for documentation purpose, to record the issued command and append it to the email which is send after the job. http://gridengine.org/pipermail/users/2011-September/001629.html $ qstat -j 12345 ... context: COMMAND=subturbo -v 631 -g -m 3500 -p 8 -t infinity -s aoforce,OUTPUT=/home/foobar/carbene/gecl4_2carb228/trans_tzvp_3.out It's only one long line, and I split it later on to inidividual entries. In your case you have to watch out for commas, as they are used already to separate entries. -- Reuti > Thanks. > > -- > -MichaelC > _______________________________________________ > users mailing list > users@gridengine.org > https://gridengine.org/mailman/listinfo/users _______________________________________________ users mailing list users@gridengine.org https://gridengine.org/mailman/listinfo/users