Maybe this should go to the devel list but I'll start here.
In tracking the way the PBS tm API propagates error information
back to clients, I noticed that Open MPI is making an incorrect
assumption. (I'm looking 1.3.2.) The relevant code in
orte/mca/plm/tm/plm_tm_module.c is:
/* TM poll for all the spawns */
for (i = 0; i < launched; ++i) {
rc = tm_poll(TM_NULL_EVENT, &event, 1, &local_err);
if (TM_SUCCESS != rc) {
errno = local_err;
opal_output(0, "plm:tm: failed to poll for a spawned daemon,"
" return status = %d", rc);
goto cleanup;
}
}
My reading of the way the tm API works is that tm_poll() can (will)
return TM_SUCCESS(0) even when the tm_spawn event being waited on failed,
i.e. local_err needs to be checked even if rc=0. It looks like TM_
errors (rc values) are from tm protocol failures or incorrect calls
to tm. local_err is to do with why the actual requested action failed
and is usually some sort of internal PBSE_ error code. In fact it's
probably always PBSE_SYSTEM (15010) - I think it is for tm_spawn().
Something like the following is probably closer to what is needed.
/* TM poll for all the spawns */
for (i = 0; i < launched; ++i) {
rc = tm_poll(TM_NULL_EVENT, &event, 1, &local_err);
if (TM_SUCCESS != rc) {
errno = local_err;
opal_output(0, "plm:tm: failed to poll for a spawned daemon,"
" return status = %d", rc);
goto cleanup;
}
if (local_err!=0) {
errno = local_err;
opal_output(0, "plm:tm: failed to spawn daemon,"
" error code = %d", errno );
goto cleanup;
}
}
I checked torque 2.3.3 to confirm that it's tm behaviour is the same as
OpenPBS in this respect. No idea about PBSPro.
David
