> That makes no sense. Fix your quoting (it's atrocious) and then run > the function with "set -x". Don't throw strace at it, at this point. > It's really not the right tool at this level of debugging.
Okay, I simplified the script slightly, fixed the quoting and re-ran with set -x. Here is the script and the output for the failure scenario. # $1 mntpoint fsaccesstest() { set -x local RETRY_MAX=3 local RETRY_INTERVAL=3 local CHK_RESULT=1 local RETVAL=0 TMP=`grep "$1" /proc/mounts|awk '{print $1}'` if [ "$TMP" != "" ]; then for i in `seq 1 ${RETRY_MAX}` do touch "$1"/.accesstest CHK_RESULT=$? if [ ${CHK_RESULT} -eq 0 ] ; then break fi logger -s -t ${LOGTAG} -p ${LOGFACILITY} "*** fsaccesstest test $1 failed CHK_RESULT=$CHK_RESULT. retrying... (${i}) ***" sleep ${RETRY_INTERVAL} done if [ $CHK_RESULT -ne 0 ]; then RETVAL=$CHK_RESULT fi fi set +x return $RETVAL } + local RETRY_MAX=3 + local RETRY_INTERVAL=3 + local CHK_RESULT=1 + local RETVAL=0 ++ grep /mnt/array1 /proc/mounts ++ awk '{print $1}' + TMP=/dev/md10 + '[' /dev/md10 '!=' '' ']' ++ seq 1 3 + for i in '`seq 1 ${RETRY_MAX}`' + touch /mnt/array1/.accesstest + CHK_RESULT=1 + '[' 1 -eq 0 ']' + logger -s -t diskmon -p local0.info '*** fsaccesstest test /mnt/array1 failed CHK_RESULT=1. retrying... (1) ***' diskmon: *** fsaccesstest test /mnt/array1 failed CHK_RESULT=1. retrying... (1) *** + sleep 3 + for i in '`seq 1 ${RETRY_MAX}`' + touch /mnt/array1/.accesstest + CHK_RESULT=0 + '[' 0 -eq 0 ']' + break + '[' 0 -ne 0 ']' + set +x > I don't even know what your actual *symptom* is. I can't deduce it from > a spew of strace output. You haven't described the reason for the touch > command, so I can only presume this is some sort of autofs environment, > hence my attempt to solve the issue, above. The purpose of this function is simply to try and create or modify a test file on a RAID share. It's just a periodic integrity check. The set -x output shows that the 'touch' command failed, but it doesn't show why. I certainly acknowledge the strace output is a lot of stuff to untangle, but it does show what is happening at a process level and the return status for every system call. What I'm seeing is that there are lots of concurrent processes running at the time this particular shell script function is executed. The other processes seem to have an adverse effect on the process that is executing 'touch'. The three system calls that touch makes are all successful. So my question is why is $? getting set to 1? It looks to me like the bash interpreter is returning early from this call to 'touch'. I can't prove this yet though. Here's another example of an unsuccessful touch execution. Notice how the execution keeps getting interrupted and then resumed. [pid 31400] execve("/tmp/test-touch", ["/tmp/test-touch", "/mnt/array1/.accesstest"], ["HOSTNAME=TS-2RVED8", "TERM=linux", "SHELL=/bin/sh", "LIBRARY_PATH=/usr/lib:/usr/local"..., "HUSHLOGIN=FALSE", "USER=root", "PATH=/bin:/sbin:/usr/bin:/usr/sb"..., "MAIL=/var/spool/mail/root", "C_INCLUDE_PATH=/usr/include:/usr"..., "PWD=/usr/local/sbin", "HOME=/root", "SHLVL=2", "LOGNAME=root", "_=/tmp/test-touch"]Process 31401 attached (waiting for parent) [pid 31400] <... execve resumed> ) = 0 [pid 31400] brk(0 <unfinished ...> [pid 31400] <... brk resumed> ) = 0x2369000 [pid 31400] open("/mnt/array1/.accesstest", O_RDWR|O_CREAT, 0666 <unfinished ...> [pid 31400] <... open resumed> ) = 3 [pid 31400] close(3 <unfinished ...> [pid 31400] <... close resumed> ) = 0 [pid 31400] utimes("/mnt/array1/.accesstest", {4196394, 0} <unfinished ...> [pid 31400] <... utimes resumed> ) = 0 [pid 31400] exit_group(0) = ? Process 23756 resumed Process 31400 detached [pid 23756] <... wait4 resumed> [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 31400 [pid 23756] --- SIGCHLD (Child exited) @ 0 (0) --- [pid 23756] wait4(-1, Process 23756 suspended