Re: [gridengine users] how to override boolean attributes set in .sge_request
On 29 March 2012 19:11, Reuti re...@staff.uni-marburg.de wrote: Am 29.03.2012 um 19:33 schrieb berg...@merctech.com: In the message dated: Thu, 29 Mar 2012 08:30:35 BST, The pithy ruminations from William Hay on Re: [gridengine users] how to override boolean attributes set in .sge_request were: = On 29 March 2012 02:07, berg...@merctech.com berg...@merctech.com wrote: = We're running SGE 6.2u5 and seem to be having a problem with using = command-line options to disable boolean attributes set in the .sge_request = file. = = We've got both CentOS4 and CentOS5 nodes. There are complex entries = for each: = = = qconf -sc|egrep #|centos = #name shortcut type relop requestable consumable default urgency = #-- = centos4 c4 BOOL == YES NO 0 1000 = centos5 c5 BOOL == YES NO 0 1000 = = = Compute nodes have the appropriate attributes for those resources. = = Without a .sge_request file, users can direct jobs to either type of node by = specifying -l centos4 or -l centos5. = = By default, users have a ~/.sge_request file that contains: = = -l centos5 = = This works fine, except for the rare times when someone needs an = interactive session on a CentOS4 machine. = = Since the centos5 complex is a boolean, I hoped to be able to run: = = qlogin -l centos4 -l centos5=0 = or = qlogin -l centos4,centos5=0 = = In order for the above to work your centos4 nodes would need = centos5=false or centos5=0 in their complex values configuration. Do = they? Requesting false with an == RELOP = is not the same as not caring about the value. Yep. Interesting. Are you saying that there is no way to unset a boolean, once it has been set...and there is no implicit attribute set to FALSE if an attribute is not assigned to a host or queue? There is the option -clear for qsub, but it resets all. I can easily update my CentOS4 nodes to set centos5=false (and the reverse) [PAUSE WHILE TYPING] Done. That solved the problem. This: qlogin -l centos4,centos5=FALSE now works. Another way could have been a RESTRING and you request -l os=centos4 or -l os=centos5 I did think of suggesting that but from his description he would have had to poke around in each user's home directory since the default was set in their personal .sge_request files rather than the global equivalent. Or make centos5=true for all hosts even the centos4 ones. ___ users mailing list users@gridengine.org https://gridengine.org/mailman/listinfo/users
[gridengine users] re-submitting finished taskes
hey everyone, is it possible to resubmit a task that is listed in qmon as finished? or has anyone build a system to allow for such functionality? Lars ___ users mailing list users@gridengine.org https://gridengine.org/mailman/listinfo/users
Re: [gridengine users] re-submitting finished taskes
http://arc.liv.ac.uk/pipermail/gridengine-users/2010-November/032986.html found my answer. On 30 March 2012 10:19, Lars van der bijl l...@realisestudio.com wrote: hey everyone, is it possible to resubmit a task that is listed in qmon as finished? or has anyone build a system to allow for such functionality? Lars ___ users mailing list users@gridengine.org https://gridengine.org/mailman/listinfo/users
Re: [gridengine users] Fwd: Gridengine and Hadoop
I'm registering my interest here. Reuti -- if you could pass my email along to Ralph I'd appreciate it. I have several consulting customers using EMC Isilon storage on Grid Engine HPC clusters and we've been getting pinged from EMC/Greenplum sales reps pushing to show off the combination of native HDFS support in Isilon + the greenplum hadoop appliance integration. Basically I have a few largish sites that could test provide feedback if things work out. Some are commercial, some are .gov all are interested in SGE + Hadoop enhancements. -dag Reuti wrote: on behalf of Ralph Castain who you may know from the Open MPI mailing list I want to forward this eMail to your attention. -- Reuti I have a question for the Gridengine community, but thought I'd run it through you as I believe you work in that area? As you may know, I am now employed by Greenplum/EMC to work on resource management for Hadoop as well as MPI. The main concern frankly is that the current Hadoop RM (yarn) scales poorly in terms of launch and provides no support for MPI wireup, thus causing MPI jobs to exhibit quadratic scaling of startup times. The only reason for using yarn is that it has the HDFS interface required to determine file locality, thus allowing users to place processes network-near to the files they will use. I have initiated an effort here at GP to create a C-library for accessing HDFS to obtain that locality info, and expect to have it completed in the next few weeks. Armed with that capability, it would be possible to extend more capable RMs such as Gridengine so that users could obtain HDFS-based allocations for their MapReduce applications. This would allow Gridengine to support Hadoop operations, and make Hadoop clusters that used Gridengine as their RM be multi-use. Would this be of interest to the community? I can contribute the C-lib code for their use under a BSD-like license structure, if that would help. Regards, Ralph ___ users mailing list users@gridengine.org https://gridengine.org/mailman/listinfo/users
