Re: utrace-ptrace gdb testsuite tesults
Followed the differences found by Qian and verified none of them (did not verify the ppc suspicious one) has any regression in GDB testsuite. I did not reproduce the original possible regression seen on ppc64 RHEL6 systems. The kernel was build directly from roland's git tree with and without CONFIG_UTRACE. $ diff -u noutrace/gdb-64.sum utrace/gdb-64.sum --- noutrace/gdb-64.sum 2009-11-30 16:29:16.793769391 +0800 +++ utrace/gdb-64.sum 2009-11-30 16:28:06.249892712 +0800 @@ -1,4 +1,4 @@ -Test Run By root on Mon Nov 30 02:27:53 2009 +Test Run By root on Mon Nov 30 02:27:59 2009 Native configuration is powerpc64-unknown-linux-gnu === gdb tests === @@ -4046,7 +4046,7 @@ PASS: gdb.base/foll-fork.exp: default parent follow, no catchpoints PASS: gdb.base/foll-fork.exp: set follow parent PASS: gdb.base/foll-fork.exp: explicit show parent follow, no catchpoints -PASS: gdb.base/foll-fork.exp: explicit parent follow, no catchpoints +FAIL: gdb.base/foll-fork.exp: (timeout) explicit parent follow, no catchpoints PASS: gdb.base/foll-fork.exp: set follow child PASS: gdb.base/foll-fork.exp: explicit show child follow, no catchpoints PASS: gdb.base/foll-fork.exp: explicit child follow, no catchpoints @@ -14764,7 +14764,7 @@ PASS: gdb.threads/attach-stopped.exp: threaded: set file, before attach1 to stopped process (re-read) PASS: gdb.threads/attach-stopped.exp: threaded: attach1 to stopped, after setting file PASS: gdb.threads/attach-stopped.exp: threaded: attach1 to stopped bt -FAIL: gdb.threads/attach-stopped.exp: threaded: attach1, exit leaves process stopped +PASS: gdb.threads/attach-stopped.exp: threaded: attach1, exit leaves process stopped PASS: gdb.threads/attach-stopped.exp: threaded: attach2 to stopped, after setting file PASS: gdb.threads/attach-stopped.exp: threaded: attach2 to stopped bt PASS: gdb.threads/attach-stopped.exp: continue (threaded: attach2 continue) I'll try to schedule it on a few other ppc64 systems to see if make any difference. CAI Qian
Re: utrace-ptrace gdb testsuite tesults
On Wed, 25 Nov 2009 23:30:37 +0100, Jan Kratochvil wrote: Please point at some built or easily buildable kernel .rpm first. http://kojipkgs.fedoraproject.org/scratch/roland/task_1825649/ OK, taken for reverification. Followed the differences found by Qian and verified none of them (did not verify the ppc suspicious one) has any regression in GDB testsuite. Regards, Jan
Re: utrace-ptrace gdb testsuite tesults
On Sun, 29 Nov 2009 23:39:59 +0100, Jan Kratochvil wrote: Followed the differences found by Qian and verified none of them (did not verify the ppc suspicious one) has any regression in GDB testsuite. Forgot the log FYI. Regards, Jan -result-2.6.31.5-127.fc12.x86_64/gdb +result-2.6.32-0.53.rc8.496.fc13.x86_64/gdb *attach*stop* generally unchecked, it should be covered by ptrace-testsuite and the GDB testcases are currently racy. -FAIL: gdb.base/follow-child.exp: break +PASS: gdb.base/follow-child.exp: break = unstable testcase, RH-specific, dropped as redundant to other testcases /root/jkratoch/redhat/gdb-7.0-3.fc12.src/gdb-7.0-m64/gdb/testsuite -PASS: gdb.threads/attachstop-mt.exp: attach4 stop by interrupt -PASS: gdb.threads/attachstop-mt.exp: attach4, exit leaves process sleeping +FAIL: gdb.threads/attachstop-mt.exp: attach4 stop by interrupt (timeout) +FAIL: gdb.threads/attachstop-mt.exp: attach4, exit leaves process sleeping = racy, ignored -PASS: gdb.base/foll-fork.exp: default parent follow, no catchpoints +FAIL: gdb.base/foll-fork.exp: (timeout) default parent follow, no catchpoints = racy, fixed the testcase upstream /root/jkratoch/redhat/gdb-7.0-3.fc12.src/gdb-7.0-m64/gdb/testsuite.unix.-m32 -FAIL: gdb.threads/attach-stopped.exp: threaded: attach1, exit leaves process stopped +PASS: gdb.threads/attach-stopped.exp: threaded: attach1, exit leaves process stopped = racy, ignored -FAIL: gdb.base/interrupt.exp: continue +PASS: gdb.base/interrupt.exp: continue FAIL: gdb.base/interrupt.exp: echo data (timeout) ERROR: Undefined command . UNRESOLVED: gdb.base/interrupt.exp: Send Control-C, second time FAIL: gdb.base/interrupt.exp: signal SIGINT (the program is no longer running) -FAIL: gdb.base/interrupt.exp: echo more data (timeout) -FAIL: gdb.base/interrupt.exp: send end of file +PASS: gdb.base/interrupt.exp: echo more data +FAIL: gdb.base/interrupt.exp: send end of file (eof) = both kernels behave the same - correctly, updated erestart* tests set, for x86_64-x86_64-i386 (kernel-debugger-inferior) GDB needs a fix: http://sourceware.org/ml/gdb-patches/2009-11/msg00592.html -FAIL: gdb.server/ext-run.exp: get process list +PASS: gdb.server/ext-run.exp: get process list = upstream gdbserver data corruption -FAIL: gdb.java/jnpe.exp: next over NPE +PASS: gdb.java/jnpe.exp: next over NPE = fixed the testcase in archer /root/jkratoch/redhat/gdb-7.0-3.fc12.src/gdb-7.0-m32/gdb/testsuite.unix.-m32 -ERROR: Couldn't send info inferior 16 to GDB. -UNRESOLVED: gdb.base/multi-forks.exp: Did kill 16 +PASS: gdb.base/multi-forks.exp: Run to exit 11 = always ignored by me, IMO racy -PASS: gdb.threads/attachstop-mt.exp: attach4 stop by interrupt -PASS: gdb.threads/attachstop-mt.exp: attach4, exit leaves process sleeping +FAIL: gdb.threads/attachstop-mt.exp: attach4 stop by interrupt (timeout) +FAIL: gdb.threads/attachstop-mt.exp: attach4, exit leaves process sleeping = racy, ignored -FAIL: gdb.cp/constructortest.exp: running to main in runto PASS: gdb.cp/constructortest.exp: breaking on A::A -FAIL: gdb.cp/constructortest.exp: continue to breakpoint: First line A -FAIL: gdb.cp/constructortest.exp: Verify in in-charge A::A -FAIL: gdb.cp/constructortest.exp: continue to breakpoint: First line A -FAIL: gdb.cp/constructortest.exp: Verify in not-in-charge A::A +PASS: gdb.cp/constructortest.exp: continue to breakpoint: First line A +PASS: gdb.cp/constructortest.exp: Verify in in-charge A::A +PASS: gdb.cp/constructortest.exp: continue to breakpoint: First line A +PASS: gdb.cp/constructortest.exp: Verify in not-in-charge A::A -FAIL: gdb.pie/break.exp: run until function breakpoint -FAIL: gdb.pie/break.exp: run until breakpoint set at a line number -FAIL: gdb.pie/break.exp: run until file:function(6) breakpoint -FAIL: gdb.pie/break.exp: run until file:function(5) breakpoint (the program is no longer running) -FAIL: gdb.pie/break.exp: run until file:function(4) breakpoint (the program is no longer running) -FAIL: gdb.pie/break.exp: run until file:function(3) breakpoint (the program is no longer running) -FAIL: gdb.pie/break.exp: run until file:function(2) breakpoint (the program is no longer running) -FAIL: gdb.pie/break.exp: run until file:function(1) breakpoint (the program is no longer running) -FAIL: gdb.pie/break.exp: run until quoted breakpoint (the program is no longer running) -FAIL: gdb.pie/break.exp: run until file:linenum breakpoint (the program is no longer running) -FAIL: gdb.pie/break.exp: breakpoint offset +1 -FAIL: gdb.pie/break.exp: step onto breakpoint (the program is no longer running) +PASS: gdb.pie/break.exp: run until function breakpoint +PASS: gdb.pie/break.exp: run until breakpoint set at a line number +PASS: gdb.pie/break.exp: run until file:function(6) breakpoint +PASS: gdb.pie/break.exp: run until file:function(5) breakpoint +PASS: gdb.pie/break.exp: run until file:function(4) breakpoint +PASS:
Re: utrace-ptrace gdb testsuite tesults
On Wed, Nov 25, 2009 at 01:17:15PM -0800, Roland McGrath wrote: That's certainly good to hear. If you are pretty confident about that, then I am quite happy to consider nonregression on all of ptrace-tests the sole gating test for kernel changes. We just don't want to wind up having other upstream reviewers notice a regression using gdb that we didn't notice before we submitted a kernel change. I've just done 'make check' twice on unpatched kernel, and found that the results are not stable: --- gdb.sum 2009-11-27 09:54:14.0 +0100 +++ gdb.sum22009-11-27 10:51:42.0 +0100 @@ -1,4 +1,4 @@ -Test Run By root on Thu Nov 26 18:52:09 2009 +Test Run By root on Fri Nov 27 09:54:33 2009 Native configuration is i686-pc-linux-gnu === gdb tests === @@ -3537,12 +3537,12 @@ PASS: gdb.base/foll-fork.exp: unpatch ch PASS: gdb.base/foll-fork.exp: unpatch child, catch fork PASS: gdb.base/foll-fork.exp: unpatch child, breakpoint at exit call PASS: gdb.base/foll-fork.exp: unpatch child, set follow child -FAIL: gdb.base/foll-fork.exp: unpatch child, unpatched parent breakpoints from child (timeout) +PASS: gdb.base/foll-fork.exp: unpatch child, unpatched parent breakpoints from child PASS: gdb.base/foll-fork.exp: explicit parent follow, set tcatch fork PASS: gdb.base/foll-fork.exp: explicit parent follow, tcatch fork PASS: gdb.base/foll-fork.exp: set follow parent PASS: gdb.base/foll-fork.exp: set follow parent, tbreak -PASS: gdb.base/foll-fork.exp: set follow parent, hit tbreak +FAIL: gdb.base/foll-fork.exp: (timeout) set follow parent, hit tbreak PASS: gdb.base/foll-fork.exp: set follow parent, cleanup Running ./gdb.base/foll-vfork.exp ... PASS: gdb.base/foll-vfork.exp: set verbose @@ -12499,7 +12499,7 @@ PASS: gdb.mi/mi-nsmoribund.exp: thread s PASS: gdb.mi/mi-nsmoribund.exp: resume all, thread specific breakpoint PASS: gdb.mi/mi-nsmoribund.exp: hit thread specific breakpoint PASS: gdb.mi/mi-nsmoribund.exp: thread state: all running except the breakpoint thread -PASS: gdb.mi/mi-nsmoribund.exp: resume all, program exited normally +FAIL: gdb.mi/mi-nsmoribund.exp: unexpected stop Running ./gdb.mi/mi-nsthrexec.exp ... PASS: gdb.mi/mi-nsthrexec.exp: successfully compiled posix threads test case PASS: gdb.mi/mi-nsthrexec.exp: breakpoint at main @@ -14507,7 +14507,7 @@ PASS: gdb.threads/watchthreads2.exp: bre PASS: gdb.threads/watchthreads2.exp: all threads started PASS: gdb.threads/watchthreads2.exp: watch x PASS: gdb.threads/watchthreads2.exp: set var test_ready = 1 -KFAIL: gdb.threads/watchthreads2.exp: gdb can drop watchpoints in multithreaded app (PRMS: gdb/10116) +PASS: gdb.threads/watchthreads2.exp: all threads incremented x Running ./gdb.threads/watchthreads.exp ... PASS: gdb.threads/watchthreads.exp: successfully compiled posix threads test case PASS: gdb.threads/watchthreads.exp: watch args[0] @@ -14672,7 +14672,7 @@ UNSUPPORTED: gdb.xml/tdesc-xinclude.exp: === gdb Summary === # of expected passes 13854 -# of unexpected failures 75 +# of unexpected failures 76 # of expected failures 43 # of untested testcases7 # of unsupported tests 59 -- Veaceslav
Re: utrace-ptrace gdb testsuite tesults
On Fri, 27 Nov 2009 15:11:09 +0100, Veaceslav Falico wrote: -FAIL: gdb.base/foll-fork.exp: unpatch child, unpatched parent breakpoints from child (timeout) +PASS: gdb.base/foll-fork.exp: unpatch child, unpatched parent breakpoints from child -PASS: gdb.base/foll-fork.exp: set follow parent, hit tbreak +FAIL: gdb.base/foll-fork.exp: (timeout) set follow parent, hit tbreak To be ignored, fixed upstream: http://sourceware.org/ml/gdb-patches/2009-11/msg00573.html -PASS: gdb.mi/mi-nsmoribund.exp: resume all, program exited normally +FAIL: gdb.mi/mi-nsmoribund.exp: unexpected stop -KFAIL: gdb.threads/watchthreads2.exp: gdb can drop watchpoints in multithreaded app (PRMS: gdb/10116) +PASS: gdb.threads/watchthreads2.exp: all threads incremented x These are known to be unstable but there some known watch and non-stop problems so it may not even be a testcase-side bug. Therefore this test shows no changes/regressions. Regards, Jan
Re: utrace-ptrace gdb testsuite tesults
On 11/27, Veaceslav Falico wrote: On Wed, Nov 25, 2009 at 01:17:15PM -0800, Roland McGrath wrote: That's certainly good to hear. If you are pretty confident about that, then I am quite happy to consider nonregression on all of ptrace-tests the sole gating test for kernel changes. We just don't want to wind up having other upstream reviewers notice a regression using gdb that we didn't notice before we submitted a kernel change. I've just done 'make check' twice on unpatched kernel, and found that the results are not stable: --- gdb.sum 2009-11-27 09:54:14.0 +0100 +++ gdb.sum22009-11-27 10:51:42.0 +0100 @@ -1,4 +1,4 @@ -Test Run By root on Thu Nov 26 18:52:09 2009 +Test Run By root on Fri Nov 27 09:54:33 2009 Native configuration is i686-pc-linux-gnu === gdb tests === @@ -3537,12 +3537,12 @@ PASS: gdb.base/foll-fork.exp: unpatch ch PASS: gdb.base/foll-fork.exp: unpatch child, catch fork PASS: gdb.base/foll-fork.exp: unpatch child, breakpoint at exit call PASS: gdb.base/foll-fork.exp: unpatch child, set follow child -FAIL: gdb.base/foll-fork.exp: unpatch child, unpatched parent breakpoints from child (timeout) +PASS: gdb.base/foll-fork.exp: unpatch child, unpatched parent breakpoints from child PASS: gdb.base/foll-fork.exp: explicit parent follow, set tcatch fork PASS: gdb.base/foll-fork.exp: explicit parent follow, tcatch fork PASS: gdb.base/foll-fork.exp: set follow parent PASS: gdb.base/foll-fork.exp: set follow parent, tbreak -PASS: gdb.base/foll-fork.exp: set follow parent, hit tbreak +FAIL: gdb.base/foll-fork.exp: (timeout) set follow parent, hit tbreak PASS: gdb.base/foll-fork.exp: set follow parent, cleanup Running ./gdb.base/foll-vfork.exp ... PASS: gdb.base/foll-vfork.exp: set verbose @@ -12499,7 +12499,7 @@ PASS: gdb.mi/mi-nsmoribund.exp: thread s PASS: gdb.mi/mi-nsmoribund.exp: resume all, thread specific breakpoint PASS: gdb.mi/mi-nsmoribund.exp: hit thread specific breakpoint PASS: gdb.mi/mi-nsmoribund.exp: thread state: all running except the breakpoint thread -PASS: gdb.mi/mi-nsmoribund.exp: resume all, program exited normally +FAIL: gdb.mi/mi-nsmoribund.exp: unexpected stop Running ./gdb.mi/mi-nsthrexec.exp ... PASS: gdb.mi/mi-nsthrexec.exp: successfully compiled posix threads test case PASS: gdb.mi/mi-nsthrexec.exp: breakpoint at main @@ -14507,7 +14507,7 @@ PASS: gdb.threads/watchthreads2.exp: bre PASS: gdb.threads/watchthreads2.exp: all threads started PASS: gdb.threads/watchthreads2.exp: watch x PASS: gdb.threads/watchthreads2.exp: set var test_ready = 1 -KFAIL: gdb.threads/watchthreads2.exp: gdb can drop watchpoints in multithreaded app (PRMS: gdb/10116) +PASS: gdb.threads/watchthreads2.exp: all threads incremented x Running ./gdb.threads/watchthreads.exp ... PASS: gdb.threads/watchthreads.exp: successfully compiled posix threads test case PASS: gdb.threads/watchthreads.exp: watch args[0] @@ -14672,7 +14672,7 @@ UNSUPPORTED: gdb.xml/tdesc-xinclude.exp: === gdb Summary === # of expected passes 13854 -# of unexpected failures 75 +# of unexpected failures 76 # of expected failures 43 # of untested testcases7 # of unsupported tests 59 Nice, thanks. So. I am going to conclude that, more or less, utrace-ptrace passes these tests. Jan, if you see something particular which needs more attention or should be fixed, please let me know. I'll try to investigate then. Oleg.
Re: utrace-ptrace gdb testsuite tesults
On Fri, 27 Nov 2009 15:34:05 +0100, Oleg Nesterov wrote: Jan, if you see something particular which needs more attention or should be fixed, please let me know. I'll try to investigate then. I am still not finished with the verifications yesterday but so far no kernel behavior change has been proven and I doubt it will be. Going to reply today. The ppc kernel should be checked but I do not have built two non-utrace/utrace matching kernel rpms for it. Regards, Jan
utrace-ptrace gdb testsuite tesults
First of all, thanks Ananth and Cai for help! Jan, I need your help ;) looking at different reports I can't understand how to interpret them. To the point, I do not understand if the overall results are good or bad. The first question, are these tests supposed to be stable? For example, Those are the test results on i686 F12 hosts with and without CONFIG_UTRACE. Interesting thing is that the results on quite different on two Intel hosts. Yes! I'd say the results are just reversed. We have some PASS-FAIL changes on ProLiant DL360 G4p, and on HP Workstation XW4200 machine the _same_ tests show FAIL-PASS change. I can't imagine how utrace-ptrace can explain this difference if results are stable. I spent several hours trying to figure out how can I fix the failures, but since I never used gdb this is very much nontrivial to me. Because I just don't understand whats going on and what any particular test actually does. So. Given that the number of test is huge, and (I guess) we can't hope utrace-ptrace can pass 100% of tests, I am asking you to tell me which failures are important. Then I'll try to fix them (most probably I will ask a lot of stupid questions before I will be able to do this ;). IOW. Only you and Roland can say whether utrace-ptrace is ready for use from gdb-testsuite's pov. Please tell me which failures should be fixed to conclude that utrace-ptrace works. Oleg.
Re: utrace-ptrace gdb testsuite tesults
In general everything where is a word thread has unstable results and nonstop tests are also a bit unstable. So where exactly is the problem in these cases? Are the tests overly timing-sensitive where there is no actual behavior bug? Or is gdb overly timing-sensitive where there is no actual kernel bug? Or is it just unknown, and might be a kernel bug after all (even an undiagnosed one in vanilla kernels)? There are IMO/hopefully very few cases tested by the gdb testsuite and still not covered by the ptrace-testsuite, I even do not much expect we will see again a new utrace regression caught by the gdb testsuite uncaught by the ptrace-testsuite. That's certainly good to hear. If you are pretty confident about that, then I am quite happy to consider nonregression on all of ptrace-tests the sole gating test for kernel changes. We just don't want to wind up having other upstream reviewers notice a regression using gdb that we didn't notice before we submitted a kernel change. Please point at some built or easily buildable kernel .rpm first. http://kojipkgs.fedoraproject.org/scratch/roland/task_1825649/ Thanks, Roland
Re: utrace-ptrace gdb testsuite tesults
On Wed, 25 Nov 2009 22:17:15 +0100, Roland McGrath wrote: In general everything where is a word thread has unstable results and nonstop tests are also a bit unstable. So where exactly is the problem in these cases? Are the tests overly timing-sensitive where there is no actual behavior bug? Or is gdb overly timing-sensitive where there is no actual kernel bug? Or is it just unknown, and might be a kernel bug after all (even an undiagnosed one in vanilla kernels)? gdb.server/server-run.exp: gdbserver contains data overflow/corruption, occasionally it crashes, occasionally passes. gdb.mi/mi-nonstop-exit.exp: Some race in GDB non-stop code. gdb.threads/attach-stopped.exp: Race in the testcase (I think so). etc. But in most cases I do not know, gdb.log is commonly not enough to find the problem and when it is not reproducible on the 2nd..nth run... But I+upstream already caught many races but still a lot of them remains. There are IMO/hopefully very few cases tested by the gdb testsuite and still not covered by the ptrace-testsuite, I even do not much expect we will see again a new utrace regression caught by the gdb testsuite uncaught by the ptrace-testsuite. That's certainly good to hear. If you are pretty confident about that, then I am quite happy to consider nonregression on all of ptrace-tests the sole gating test for kernel changes. We just don't want to wind up having other upstream reviewers notice a regression using gdb that we didn't notice before we submitted a kernel change. I did not verify the GDB codebase for all the ptrace calls in any way. If it is a kernel patch submit after long development period it is probably still worth checking it against GDB. Please point at some built or easily buildable kernel .rpm first. http://kojipkgs.fedoraproject.org/scratch/roland/task_1825649/ OK, taken for reverification. Regards, Jan
