Been chatting off-list with the SGE folks - can you tell us what version of SGE you are using?
On Jul 26, 2012, at 9:02 AM, Christoph van Wüllen wrote: > It is a long-standing problem that due to a bug in Sun GridEngine > (setting the stack size limit equal to the address space limit) > using qrsh from within OpenMPI fails if a large memory is requested > but the stack size not explicitly set to a reasonably small value. > > The best solution were if SGE just would not touch the stack > size limit and leave it at INFINITY. > > However I have tested that just reducing the stack size limit in > file orte/mca/plm/rsh/plm_rsh_module.c, function ssh_child() before > execv'ing qrsh circumvents the problem, so just after exec_patch is set > by strdup(...) I inserted the lines > > { > struct rlimit rlim; > int l; > > l=strlen(exec_path); > if (l > 5 && !strcmp("/qrsh", exec_path + (l-5))) { > getrlimit(RLIMIT_STACK, &rlim); > if (rlim.rlim_max > 10000000L) rlim.rlim_max=10000000L; > if (rlim.rlim_cur > 10000000L) rlim.rlim_cur=10000000L; > setrlimit(RLIMIT_STACK, &rlim); > } > } > > > It looks quick-and-dirty and it certainly is, but it solves a severe > problem many users have with OpenMPI and SGE. Feel free to use this > information as you like. Note that MPI worker jobs eventually > spawned off on "distant" nodes do not suffer from the reduced stack > size limit, it is only the qrsh command. > > Is this (still) of interest? > > +---------------------------------+----------------------------------+ > | Prof. Christoph van Wüllen | Tele-Phone (+49) (0)631 205 2749 | > | TU Kaiserslautern, FB Chemie | Tele-Fax (+49) (0)631 205 2750 | > | Erwin-Schrödinger-Str. | | > | D-67663 Kaiserslautern, Germany | vanwul...@chemie.uni-kl.de | > | | > | HomePage: http://www.chemie.uni-kl.de/vanwullen | > +---------------------------------+----------------------------------+ > > > _______________________________________________ > devel mailing list > de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/devel