Has anyone noticed their sge_execd proceses suddenly taking up a lot of CPU, possibly since around July 2nd this year? I think it might be to do with the Linux "leap second" bug, which affects processes that use "futexes". It doesn't happen to all nodes on a queue, just some. The only way I know to resolve this is to reboot the machine.
If you do "strace -p <pid>" on the sge_execd process, you'll see output like the following. BTW, I think I am not on this list right now. Dan futex(0x7fb381006b94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 2027276699, {1342229400, 776755000}, ffffffff) = -1 ETIMEDOUT (Connection timed out) futex(0x7fb381006b30, FUTEX_WAKE_PRIVATE, 1) = 0 futex(0x7fb381006b94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 2027276701, {1342229400, 776871000}, ffffffff) = -1 ETIMEDOUT (Connection timed out) futex(0x7fb381006b30, FUTEX_WAKE_PRIVATE, 1) = 0 futex(0x7fb381006b94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 2027276703, {1342229400, 776988000}, ffffffff) = -1 ETIMEDOUT (Connection timed out) futex(0x7fb381006b30, FUTEX_WAKE_PRIVATE, 1) = 0 futex(0x7fb381006b94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 2027276705, {1342229400, 777102000}, ffffffff) = -1 ETIMEDOUT (Connection timed out) futex(0x7fb381006b30, FUTEX_WAKE_PRIVATE, 1) = 0 futex(0x7fb381006b94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 2027276707, {1342229400, 777219000}, ffffffff) = -1 ETIMEDOUT (Connection timed out) futex(0x7fb381006b30, FUTEX_WAKE_PRIVATE, 1) = 0 futex(0x7fb381006b94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 2027276709, {1342229400, 777335000}, ffffffff) = -1 ETIMEDOUT (Connection timed out) futex(0x7fb381006b30, FUTEX_WAKE_PRIVATE, 1) = 0 futex(0x7fb381006b94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 2027276711, {1342229400, 777452000}, ffffffff) = -1 ETIMEDOUT (Connection timed out) futex(0x7fb381006b30, FUTEX_WAKE_PRIVATE, 1) = 0 futex(0x7fb381006b94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 2027276713, {1342229400, 777567000}, ffffffff) = -1 ETIMEDOUT (Connection timed out) futex(0x7fb381006b30, FUTEX_WAKE_PRIVATE, 1) = 0 futex(0x7fb381006b94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 2027276715, {1342229400, 777683000}, ffffffff) = -1 ETIMEDOUT (Connection timed out) futex(0x7fb381006b30, FUTEX_WAKE_PRIVATE, 1) = 0 futex(0x7fb381006b94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 2027276717, {1342229400, 777799000}, ffffffff) = -1 ETIMEDOUT (Connection timed out)
_______________________________________________ users mailing list users@gridengine.org https://gridengine.org/mailman/listinfo/users