Hi David You are quite correct. IIRC, we didn't bother checking the local_err because we found it to be unreliable - all Torque checks is that the program exec's. It doesn't report back an error if it segfaults instantly, for example, or aborts because it fails to find a required library. So we added a simple timer that declares the launch a failure if the daemon(s) fail to report back in a specified time. Hi David
This didn't cause any problems, so I went ahead and put it in our devel trunk. Barring any subsequent error reports, I'll move it over to the 1.3 series. Thanks! Ralph > However, it can't hurt to check the flag as well. I'll test it out first > just to ensure we don't get false failures. > > Thanks > Ralph > > On Aug 12, 2009, at 11:33 PM, David Singleton wrote: >> >> >> Maybe this should go to the devel list but I'll start here. >> >> In tracking the way the PBS tm API propagates error information >> back to clients, I noticed that Open MPI is making an incorrect >> assumption. (I'm looking 1.3.2.) The relevant code in >> orte/mca/plm/tm/plm_tm_module.c is: >> >> /* TM poll for all the spawns */ >> for (i = 0; i < launched; ++i) { >> rc = tm_poll(TM_NULL_EVENT, &event, 1, &local_err); >> if (TM_SUCCESS != rc) { >> errno = local_err; >> opal_output(0, "plm:tm: failed to poll for a spawned daemon," >> " return status = %d", rc); >> goto cleanup; >> } >> } >> >> My reading of the way the tm API works is that tm_poll() can (will) >> return TM_SUCCESS(0) even when the tm_spawn event being waited on failed, >> i.e. local_err needs to be checked even if rc=0. It looks like TM_ >> errors (rc values) are from tm protocol failures or incorrect calls >> to tm. local_err is to do with why the actual requested action failed >> and is usually some sort of internal PBSE_ error code. In fact it's >> probably always PBSE_SYSTEM (15010) - I think it is for tm_spawn(). >> >> Something like the following is probably closer to what is needed. >> >> /* TM poll for all the spawns */ >> for (i = 0; i < launched; ++i) { >> rc = tm_poll(TM_NULL_EVENT, &event, 1, &local_err); >> if (TM_SUCCESS != rc) { >> errno = local_err; >> opal_output(0, "plm:tm: failed to poll for a spawned daemon," >> " return status = %d", rc); >> goto cleanup; >> } >> if (local_err!=0) { >> errno = local_err; >> opal_output(0, "plm:tm: failed to spawn daemon," >> " error code = %d", errno ); >> goto cleanup; >> } >> } >> >> I checked torque 2.3.3 to confirm that it's tm behaviour is the same as >> OpenPBS in this respect. No idea about PBSPro. >> >> >> David >> _______________________________________________ >> users mailing list >> us...@open-mpi.org >> http://www.open-mpi.org/mailman/listinfo.cgi/users >> >