Re: utrace-ptrace gdb testsuite tesults
Followed the differences found by Qian and verified none of them (did not verify the ppc suspicious one) has any regression in GDB testsuite. I did not reproduce the original possible regression seen on ppc64 RHEL6 systems. The kernel was build directly from roland's git tree with and without CONFIG_UTRACE. $ diff -u noutrace/gdb-64.sum utrace/gdb-64.sum --- noutrace/gdb-64.sum 2009-11-30 16:29:16.793769391 +0800 +++ utrace/gdb-64.sum 2009-11-30 16:28:06.249892712 +0800 @@ -1,4 +1,4 @@ -Test Run By root on Mon Nov 30 02:27:53 2009 +Test Run By root on Mon Nov 30 02:27:59 2009 Native configuration is powerpc64-unknown-linux-gnu === gdb tests === @@ -4046,7 +4046,7 @@ PASS: gdb.base/foll-fork.exp: default parent follow, no catchpoints PASS: gdb.base/foll-fork.exp: set follow parent PASS: gdb.base/foll-fork.exp: explicit show parent follow, no catchpoints -PASS: gdb.base/foll-fork.exp: explicit parent follow, no catchpoints +FAIL: gdb.base/foll-fork.exp: (timeout) explicit parent follow, no catchpoints PASS: gdb.base/foll-fork.exp: set follow child PASS: gdb.base/foll-fork.exp: explicit show child follow, no catchpoints PASS: gdb.base/foll-fork.exp: explicit child follow, no catchpoints @@ -14764,7 +14764,7 @@ PASS: gdb.threads/attach-stopped.exp: threaded: set file, before attach1 to stopped process (re-read) PASS: gdb.threads/attach-stopped.exp: threaded: attach1 to stopped, after setting file PASS: gdb.threads/attach-stopped.exp: threaded: attach1 to stopped bt -FAIL: gdb.threads/attach-stopped.exp: threaded: attach1, exit leaves process stopped +PASS: gdb.threads/attach-stopped.exp: threaded: attach1, exit leaves process stopped PASS: gdb.threads/attach-stopped.exp: threaded: attach2 to stopped, after setting file PASS: gdb.threads/attach-stopped.exp: threaded: attach2 to stopped bt PASS: gdb.threads/attach-stopped.exp: continue (threaded: attach2 continue) I'll try to schedule it on a few other ppc64 systems to see if make any difference. CAI Qian
Re: utrace-ptrace gdb testsuite tesults
On Wed, 25 Nov 2009 23:30:37 +0100, Jan Kratochvil wrote: Please point at some built or easily buildable kernel .rpm first. http://kojipkgs.fedoraproject.org/scratch/roland/task_1825649/ OK, taken for reverification. Followed the differences found by Qian and verified none of them (did not verify the ppc suspicious one) has any regression in GDB testsuite. Regards, Jan
Re: utrace-ptrace gdb testsuite tesults
On Sun, 29 Nov 2009 23:39:59 +0100, Jan Kratochvil wrote: Followed the differences found by Qian and verified none of them (did not verify the ppc suspicious one) has any regression in GDB testsuite. Forgot the log FYI. Regards, Jan -result-2.6.31.5-127.fc12.x86_64/gdb +result-2.6.32-0.53.rc8.496.fc13.x86_64/gdb *attach*stop* generally unchecked, it should be covered by ptrace-testsuite and the GDB testcases are currently racy. -FAIL: gdb.base/follow-child.exp: break +PASS: gdb.base/follow-child.exp: break = unstable testcase, RH-specific, dropped as redundant to other testcases /root/jkratoch/redhat/gdb-7.0-3.fc12.src/gdb-7.0-m64/gdb/testsuite -PASS: gdb.threads/attachstop-mt.exp: attach4 stop by interrupt -PASS: gdb.threads/attachstop-mt.exp: attach4, exit leaves process sleeping +FAIL: gdb.threads/attachstop-mt.exp: attach4 stop by interrupt (timeout) +FAIL: gdb.threads/attachstop-mt.exp: attach4, exit leaves process sleeping = racy, ignored -PASS: gdb.base/foll-fork.exp: default parent follow, no catchpoints +FAIL: gdb.base/foll-fork.exp: (timeout) default parent follow, no catchpoints = racy, fixed the testcase upstream /root/jkratoch/redhat/gdb-7.0-3.fc12.src/gdb-7.0-m64/gdb/testsuite.unix.-m32 -FAIL: gdb.threads/attach-stopped.exp: threaded: attach1, exit leaves process stopped +PASS: gdb.threads/attach-stopped.exp: threaded: attach1, exit leaves process stopped = racy, ignored -FAIL: gdb.base/interrupt.exp: continue +PASS: gdb.base/interrupt.exp: continue FAIL: gdb.base/interrupt.exp: echo data (timeout) ERROR: Undefined command . UNRESOLVED: gdb.base/interrupt.exp: Send Control-C, second time FAIL: gdb.base/interrupt.exp: signal SIGINT (the program is no longer running) -FAIL: gdb.base/interrupt.exp: echo more data (timeout) -FAIL: gdb.base/interrupt.exp: send end of file +PASS: gdb.base/interrupt.exp: echo more data +FAIL: gdb.base/interrupt.exp: send end of file (eof) = both kernels behave the same - correctly, updated erestart* tests set, for x86_64-x86_64-i386 (kernel-debugger-inferior) GDB needs a fix: http://sourceware.org/ml/gdb-patches/2009-11/msg00592.html -FAIL: gdb.server/ext-run.exp: get process list +PASS: gdb.server/ext-run.exp: get process list = upstream gdbserver data corruption -FAIL: gdb.java/jnpe.exp: next over NPE +PASS: gdb.java/jnpe.exp: next over NPE = fixed the testcase in archer /root/jkratoch/redhat/gdb-7.0-3.fc12.src/gdb-7.0-m32/gdb/testsuite.unix.-m32 -ERROR: Couldn't send info inferior 16 to GDB. -UNRESOLVED: gdb.base/multi-forks.exp: Did kill 16 +PASS: gdb.base/multi-forks.exp: Run to exit 11 = always ignored by me, IMO racy -PASS: gdb.threads/attachstop-mt.exp: attach4 stop by interrupt -PASS: gdb.threads/attachstop-mt.exp: attach4, exit leaves process sleeping +FAIL: gdb.threads/attachstop-mt.exp: attach4 stop by interrupt (timeout) +FAIL: gdb.threads/attachstop-mt.exp: attach4, exit leaves process sleeping = racy, ignored -FAIL: gdb.cp/constructortest.exp: running to main in runto PASS: gdb.cp/constructortest.exp: breaking on A::A -FAIL: gdb.cp/constructortest.exp: continue to breakpoint: First line A -FAIL: gdb.cp/constructortest.exp: Verify in in-charge A::A -FAIL: gdb.cp/constructortest.exp: continue to breakpoint: First line A -FAIL: gdb.cp/constructortest.exp: Verify in not-in-charge A::A +PASS: gdb.cp/constructortest.exp: continue to breakpoint: First line A +PASS: gdb.cp/constructortest.exp: Verify in in-charge A::A +PASS: gdb.cp/constructortest.exp: continue to breakpoint: First line A +PASS: gdb.cp/constructortest.exp: Verify in not-in-charge A::A -FAIL: gdb.pie/break.exp: run until function breakpoint -FAIL: gdb.pie/break.exp: run until breakpoint set at a line number -FAIL: gdb.pie/break.exp: run until file:function(6) breakpoint -FAIL: gdb.pie/break.exp: run until file:function(5) breakpoint (the program is no longer running) -FAIL: gdb.pie/break.exp: run until file:function(4) breakpoint (the program is no longer running) -FAIL: gdb.pie/break.exp: run until file:function(3) breakpoint (the program is no longer running) -FAIL: gdb.pie/break.exp: run until file:function(2) breakpoint (the program is no longer running) -FAIL: gdb.pie/break.exp: run until file:function(1) breakpoint (the program is no longer running) -FAIL: gdb.pie/break.exp: run until quoted breakpoint (the program is no longer running) -FAIL: gdb.pie/break.exp: run until file:linenum breakpoint (the program is no longer running) -FAIL: gdb.pie/break.exp: breakpoint offset +1 -FAIL: gdb.pie/break.exp: step onto breakpoint (the program is no longer running) +PASS: gdb.pie/break.exp: run until function breakpoint +PASS: gdb.pie/break.exp: run until breakpoint set at a line number +PASS: gdb.pie/break.exp: run until file:function(6) breakpoint +PASS: gdb.pie/break.exp: run until file:function(5) breakpoint +PASS: gdb.pie/break.exp: run until file:function(4) breakpoint +PASS:
Re: utrace-ptrace gdb testsuite tesults
On Wed, Nov 25, 2009 at 01:17:15PM -0800, Roland McGrath wrote: That's certainly good to hear. If you are pretty confident about that, then I am quite happy to consider nonregression on all of ptrace-tests the sole gating test for kernel changes. We just don't want to wind up having other upstream reviewers notice a regression using gdb that we didn't notice before we submitted a kernel change. I've just done 'make check' twice on unpatched kernel, and found that the results are not stable: --- gdb.sum 2009-11-27 09:54:14.0 +0100 +++ gdb.sum22009-11-27 10:51:42.0 +0100 @@ -1,4 +1,4 @@ -Test Run By root on Thu Nov 26 18:52:09 2009 +Test Run By root on Fri Nov 27 09:54:33 2009 Native configuration is i686-pc-linux-gnu === gdb tests === @@ -3537,12 +3537,12 @@ PASS: gdb.base/foll-fork.exp: unpatch ch PASS: gdb.base/foll-fork.exp: unpatch child, catch fork PASS: gdb.base/foll-fork.exp: unpatch child, breakpoint at exit call PASS: gdb.base/foll-fork.exp: unpatch child, set follow child -FAIL: gdb.base/foll-fork.exp: unpatch child, unpatched parent breakpoints from child (timeout) +PASS: gdb.base/foll-fork.exp: unpatch child, unpatched parent breakpoints from child PASS: gdb.base/foll-fork.exp: explicit parent follow, set tcatch fork PASS: gdb.base/foll-fork.exp: explicit parent follow, tcatch fork PASS: gdb.base/foll-fork.exp: set follow parent PASS: gdb.base/foll-fork.exp: set follow parent, tbreak -PASS: gdb.base/foll-fork.exp: set follow parent, hit tbreak +FAIL: gdb.base/foll-fork.exp: (timeout) set follow parent, hit tbreak PASS: gdb.base/foll-fork.exp: set follow parent, cleanup Running ./gdb.base/foll-vfork.exp ... PASS: gdb.base/foll-vfork.exp: set verbose @@ -12499,7 +12499,7 @@ PASS: gdb.mi/mi-nsmoribund.exp: thread s PASS: gdb.mi/mi-nsmoribund.exp: resume all, thread specific breakpoint PASS: gdb.mi/mi-nsmoribund.exp: hit thread specific breakpoint PASS: gdb.mi/mi-nsmoribund.exp: thread state: all running except the breakpoint thread -PASS: gdb.mi/mi-nsmoribund.exp: resume all, program exited normally +FAIL: gdb.mi/mi-nsmoribund.exp: unexpected stop Running ./gdb.mi/mi-nsthrexec.exp ... PASS: gdb.mi/mi-nsthrexec.exp: successfully compiled posix threads test case PASS: gdb.mi/mi-nsthrexec.exp: breakpoint at main @@ -14507,7 +14507,7 @@ PASS: gdb.threads/watchthreads2.exp: bre PASS: gdb.threads/watchthreads2.exp: all threads started PASS: gdb.threads/watchthreads2.exp: watch x PASS: gdb.threads/watchthreads2.exp: set var test_ready = 1 -KFAIL: gdb.threads/watchthreads2.exp: gdb can drop watchpoints in multithreaded app (PRMS: gdb/10116) +PASS: gdb.threads/watchthreads2.exp: all threads incremented x Running ./gdb.threads/watchthreads.exp ... PASS: gdb.threads/watchthreads.exp: successfully compiled posix threads test case PASS: gdb.threads/watchthreads.exp: watch args[0] @@ -14672,7 +14672,7 @@ UNSUPPORTED: gdb.xml/tdesc-xinclude.exp: === gdb Summary === # of expected passes 13854 -# of unexpected failures 75 +# of unexpected failures 76 # of expected failures 43 # of untested testcases7 # of unsupported tests 59 -- Veaceslav
Re: utrace-ptrace gdb testsuite tesults
On Fri, 27 Nov 2009 15:11:09 +0100, Veaceslav Falico wrote: -FAIL: gdb.base/foll-fork.exp: unpatch child, unpatched parent breakpoints from child (timeout) +PASS: gdb.base/foll-fork.exp: unpatch child, unpatched parent breakpoints from child -PASS: gdb.base/foll-fork.exp: set follow parent, hit tbreak +FAIL: gdb.base/foll-fork.exp: (timeout) set follow parent, hit tbreak To be ignored, fixed upstream: http://sourceware.org/ml/gdb-patches/2009-11/msg00573.html -PASS: gdb.mi/mi-nsmoribund.exp: resume all, program exited normally +FAIL: gdb.mi/mi-nsmoribund.exp: unexpected stop -KFAIL: gdb.threads/watchthreads2.exp: gdb can drop watchpoints in multithreaded app (PRMS: gdb/10116) +PASS: gdb.threads/watchthreads2.exp: all threads incremented x These are known to be unstable but there some known watch and non-stop problems so it may not even be a testcase-side bug. Therefore this test shows no changes/regressions. Regards, Jan
Re: utrace-ptrace gdb testsuite tesults
On 11/27, Veaceslav Falico wrote: On Wed, Nov 25, 2009 at 01:17:15PM -0800, Roland McGrath wrote: That's certainly good to hear. If you are pretty confident about that, then I am quite happy to consider nonregression on all of ptrace-tests the sole gating test for kernel changes. We just don't want to wind up having other upstream reviewers notice a regression using gdb that we didn't notice before we submitted a kernel change. I've just done 'make check' twice on unpatched kernel, and found that the results are not stable: --- gdb.sum 2009-11-27 09:54:14.0 +0100 +++ gdb.sum22009-11-27 10:51:42.0 +0100 @@ -1,4 +1,4 @@ -Test Run By root on Thu Nov 26 18:52:09 2009 +Test Run By root on Fri Nov 27 09:54:33 2009 Native configuration is i686-pc-linux-gnu === gdb tests === @@ -3537,12 +3537,12 @@ PASS: gdb.base/foll-fork.exp: unpatch ch PASS: gdb.base/foll-fork.exp: unpatch child, catch fork PASS: gdb.base/foll-fork.exp: unpatch child, breakpoint at exit call PASS: gdb.base/foll-fork.exp: unpatch child, set follow child -FAIL: gdb.base/foll-fork.exp: unpatch child, unpatched parent breakpoints from child (timeout) +PASS: gdb.base/foll-fork.exp: unpatch child, unpatched parent breakpoints from child PASS: gdb.base/foll-fork.exp: explicit parent follow, set tcatch fork PASS: gdb.base/foll-fork.exp: explicit parent follow, tcatch fork PASS: gdb.base/foll-fork.exp: set follow parent PASS: gdb.base/foll-fork.exp: set follow parent, tbreak -PASS: gdb.base/foll-fork.exp: set follow parent, hit tbreak +FAIL: gdb.base/foll-fork.exp: (timeout) set follow parent, hit tbreak PASS: gdb.base/foll-fork.exp: set follow parent, cleanup Running ./gdb.base/foll-vfork.exp ... PASS: gdb.base/foll-vfork.exp: set verbose @@ -12499,7 +12499,7 @@ PASS: gdb.mi/mi-nsmoribund.exp: thread s PASS: gdb.mi/mi-nsmoribund.exp: resume all, thread specific breakpoint PASS: gdb.mi/mi-nsmoribund.exp: hit thread specific breakpoint PASS: gdb.mi/mi-nsmoribund.exp: thread state: all running except the breakpoint thread -PASS: gdb.mi/mi-nsmoribund.exp: resume all, program exited normally +FAIL: gdb.mi/mi-nsmoribund.exp: unexpected stop Running ./gdb.mi/mi-nsthrexec.exp ... PASS: gdb.mi/mi-nsthrexec.exp: successfully compiled posix threads test case PASS: gdb.mi/mi-nsthrexec.exp: breakpoint at main @@ -14507,7 +14507,7 @@ PASS: gdb.threads/watchthreads2.exp: bre PASS: gdb.threads/watchthreads2.exp: all threads started PASS: gdb.threads/watchthreads2.exp: watch x PASS: gdb.threads/watchthreads2.exp: set var test_ready = 1 -KFAIL: gdb.threads/watchthreads2.exp: gdb can drop watchpoints in multithreaded app (PRMS: gdb/10116) +PASS: gdb.threads/watchthreads2.exp: all threads incremented x Running ./gdb.threads/watchthreads.exp ... PASS: gdb.threads/watchthreads.exp: successfully compiled posix threads test case PASS: gdb.threads/watchthreads.exp: watch args[0] @@ -14672,7 +14672,7 @@ UNSUPPORTED: gdb.xml/tdesc-xinclude.exp: === gdb Summary === # of expected passes 13854 -# of unexpected failures 75 +# of unexpected failures 76 # of expected failures 43 # of untested testcases7 # of unsupported tests 59 Nice, thanks. So. I am going to conclude that, more or less, utrace-ptrace passes these tests. Jan, if you see something particular which needs more attention or should be fixed, please let me know. I'll try to investigate then. Oleg.
Re: utrace-ptrace gdb testsuite tesults
On Fri, 27 Nov 2009 15:34:05 +0100, Oleg Nesterov wrote: Jan, if you see something particular which needs more attention or should be fixed, please let me know. I'll try to investigate then. I am still not finished with the verifications yesterday but so far no kernel behavior change has been proven and I doubt it will be. Going to reply today. The ppc kernel should be checked but I do not have built two non-utrace/utrace matching kernel rpms for it. Regards, Jan
Re: utrace-ptrace gdb testsuite tesults
In general everything where is a word thread has unstable results and nonstop tests are also a bit unstable. So where exactly is the problem in these cases? Are the tests overly timing-sensitive where there is no actual behavior bug? Or is gdb overly timing-sensitive where there is no actual kernel bug? Or is it just unknown, and might be a kernel bug after all (even an undiagnosed one in vanilla kernels)? There are IMO/hopefully very few cases tested by the gdb testsuite and still not covered by the ptrace-testsuite, I even do not much expect we will see again a new utrace regression caught by the gdb testsuite uncaught by the ptrace-testsuite. That's certainly good to hear. If you are pretty confident about that, then I am quite happy to consider nonregression on all of ptrace-tests the sole gating test for kernel changes. We just don't want to wind up having other upstream reviewers notice a regression using gdb that we didn't notice before we submitted a kernel change. Please point at some built or easily buildable kernel .rpm first. http://kojipkgs.fedoraproject.org/scratch/roland/task_1825649/ Thanks, Roland
Re: utrace-ptrace gdb testsuite tesults
On Wed, 25 Nov 2009 22:17:15 +0100, Roland McGrath wrote: In general everything where is a word thread has unstable results and nonstop tests are also a bit unstable. So where exactly is the problem in these cases? Are the tests overly timing-sensitive where there is no actual behavior bug? Or is gdb overly timing-sensitive where there is no actual kernel bug? Or is it just unknown, and might be a kernel bug after all (even an undiagnosed one in vanilla kernels)? gdb.server/server-run.exp: gdbserver contains data overflow/corruption, occasionally it crashes, occasionally passes. gdb.mi/mi-nonstop-exit.exp: Some race in GDB non-stop code. gdb.threads/attach-stopped.exp: Race in the testcase (I think so). etc. But in most cases I do not know, gdb.log is commonly not enough to find the problem and when it is not reproducible on the 2nd..nth run... But I+upstream already caught many races but still a lot of them remains. There are IMO/hopefully very few cases tested by the gdb testsuite and still not covered by the ptrace-testsuite, I even do not much expect we will see again a new utrace regression caught by the gdb testsuite uncaught by the ptrace-testsuite. That's certainly good to hear. If you are pretty confident about that, then I am quite happy to consider nonregression on all of ptrace-tests the sole gating test for kernel changes. We just don't want to wind up having other upstream reviewers notice a regression using gdb that we didn't notice before we submitted a kernel change. I did not verify the GDB codebase for all the ptrace calls in any way. If it is a kernel patch submit after long development period it is probably still worth checking it against GDB. Please point at some built or easily buildable kernel .rpm first. http://kojipkgs.fedoraproject.org/scratch/roland/task_1825649/ OK, taken for reverification. Regards, Jan
