On Jun 11, 2017 4:13 AM, "Cillian O'Donnell" <cpodonne...@gmail.com> wrote:
> Does pc386 look similar? I haven't been able to get a decent set of results for PC386. For RSB-Qemu all the tests time out. Using command qemu-system-i386 -m 128 -boot b -hda path_to/rtems-boot.img -no-reboot -monitor null -serial stdio -nographic -append "--console=com1;boot;" ---------------- RSB-QEMU ----------------- -------------- SAMPLES -------------- Passed: 0 Failed: 0 User Input: 0 Expected Fail: 0 Indeterminate: 0 Benchmark: 0 Timeout: 13 Invalid: 0 Total: 13 Average test time: 0:01:09.359520 Testing time : 0:15:01.673760 ------------------------------------------------------ For Couverture-Qemu I ran the whole test suite which took 7 hours and it decided a lot of them passed, although they probably should have timed out. For hello.exe they both take 3 minutes, RSB-Qemu times out and Couverture-qemu passes it. ------------------------------------- COUVERTURE-QEMU ------------------------------------- -------------- SAMPLES -------------- Passed: 9 Failed: 0 User Input: 1 Expected Fail: 0 Indeterminate: 0 Benchmark: 0 Timeout: 2 Invalid: 1 Total: 13 Average test time: 0:00:14.133522 Testing time : 0:03:03.735787 ------------ FULL ------------ Passed: 485 Failed: 0 User Input: 4 Expected Fail: 0 Indeterminate: 0 Benchmark: 3 Timeout: 74 Invalid: 0 Total: 566 Average test time: 0:00:44.922802 Testing time : 7:03:46.305933 ------------------------------------------------------ > Are there other qemu bsps? There are a few other rtems BSPS that seem to have a corresponding machine that is supported by both qemu varaints. I had been hoping to test all of these. But again all tests time out for RSB-Qemu and Couverture-Qemu. I can't seem to find any qemu options to prevent this. Should I keep testing and see can I get all of these working? ARM: lm3s6965evb LM32: lm32-evr, milkymist MIPS: malta POWERPC: prep (qemuprep) You might should hand check hello on all the bsps. I recall that some do not have a way for the bsp to "reset" the board and exit back to Linux. The tests run but since qemu doesn't exit at RTEMS shutdown, you have to timeout. The sim-scripts covered this up watching the log file for "END OF TEST" and killing the simulator. But if it turns out that not exiting is the issue, that's a separate problem and you should file a ticket a few move along. On 10 June 2017 at 19:35, Joel Sherrill <j...@rtems.org> wrote: > I would expect them to be the same. Couverture is a mild fork by AdaCore > focused on embedded targets and coverage. They periodically debase and > carefully apply patches. They are interested in those two targets. :) > > Does pc386 look similar? Are there other qemu bsps? > > I guess this means it is almost time to look at switching the rsb > > On Jun 10, 2017 1:25 PM, "Cillian O'Donnell" <cpodonne...@gmail.com> wrote: > > Hi, > > I'm testing Couverture-Qemu against the RSB-Qemu to check if it is > working correctly. So far I have good results for ARM bsps > xilinx_zynq_a9_qemu and realview_pbx_a9_qemu and Sparc Leon3. They > show identical results for the samples and almost identical results > for the full testsuite. They just need a slight change in qemu options > to agree (results at the end). > > **************************** > COMMON BSPS > **************************** > > ARM: > xilinx-zynq-a9 > realview-pbx-a9 > lm3s6965evb > > i386: > pc (default PC386) > > LM32: > lm32-evr (default) > milkymist > > MIPS: > malta (default) > > POWERPC: > prep > > SPARC: > leon3_generic > > > I created bsp_qemu.mc files for any missing bsps from RTEMS Tester. > However the other bsps mostly timeout, this is true for Qemu and > Couverture-qemu so they are reacting the same way. Exceptions are > PC386 runs on Couverture-qemu but not on regular Qemu. I haven't been > able to find the right qemu options to get rid of the timeouts and the > --report-mode=all feature of rtems-test doesn't turn up any other info > as to why except for powerpc/qemuprep. > > So are the following results enough to verify Couverture-qemu or > should I keep going trying to get the others to work too before doing > the RSB support? > > *********************** > Leon 3 > *********************** > > qemu-system-sparc -no-reboot -monitor null -serial stdio -nographic > -net none -M leon3_generic -kernel... > > -------------------------- > > COUVERTURE-QEMU > > -------------------------- > > --------------- > SAMPLES > --------------- > > > Passed: 11 > Failed: 0 > User Input: 1 > Expected Fail: 0 > Indeterminate: 0 > Benchmark: 0 > Timeout: 0 > Invalid: 1 > Total: 13 > Average test time: 0:00:03.679456 > Testing time : 0:00:47.832933 > > ------------- > FULL > ------------- > > Passed: 543 > Failed: 3 > User Input: 4 > Expected Fail: 0 > Indeterminate: 0 > Benchmark: 3 > Timeout: 9 > Invalid: 3 > Total: 565 > Average test time: 0:00:01.693401 > Testing time : 0:15:56.771872 > > --------------------------- > > RSB-QEMU > > --------------------------- > > --------------- > SAMPLES > --------------- > > > Passed: 11 > Failed: 0 > User Input: 1 > Expected Fail: 0 > Indeterminate: 0 > Benchmark: 0 > Timeout: 0 > Invalid: 1 > Total: 13 > Average test time: 0:00:03.679446 > Testing time : 0:00:47.832800 > > ------------- > FULL > ------------- > > Passed: 544 > Failed: 2 > User Input: 4 > Expected Fail: 0 > Indeterminate: 0 > Benchmark: 3 > Timeout: 9 > Invalid: 3 > Total: 565 > Average test time: 0:00:01.739428 > Testing time : 0:16:22.777222 > > ***************************** > > xilinx_zynq_a9_qemu > > ***************************** > > qemu-system-arm -no-reboot -monitor null -serial stdio -nographic -M > xilinx-zynq-a9 -kernel... > > ---------------------------- > > COUVERTURE-QEMU > > ---------------------------- > > --------------- > SAMPLES > --------------- > > > Passed: 11 > Failed: 0 > User Input: 1 > Expected Fail: 0 > Indeterminate: 0 > Benchmark: 0 > Timeout: 0 > Invalid: 1 > Total: 13 > Average test time: 0:00:00.559658 > Testing time : 0:00:07.275561 > > ------------- > FULL > ------------- > > Passed: 551 > Failed: 5 > User Input: 4 > Expected Fail: 0 > Indeterminate: 0 > Benchmark: 3 > Timeout: 2 > Invalid: 1 > Total: 566 > Average test time: 0:00:00.652736 > Testing time : 0:06:09.448995 > > > ------------------ > > RSB-QEMU > > ------------------ > > qemu-system-arm -no-reboot -serial null -serial mon:stdio -nographic > -M xilinx-zynq-a9 -kernel... > > --------------- > SAMPLES > --------------- > > Passed: 11 > Failed: 0 > User Input: 1 > Expected Fail: 0 > Indeterminate: 0 > Benchmark: 0 > Timeout: 0 > Invalid: 1 > Total: 13 > Average test time: 0:00:00.560756 > Testing time : 0:00:07.289829 > > ------------- > FULL > ------------- > > Passed: 545 > Failed: 4 > User Input: 4 > Expected Fail: 0 > Indeterminate: 0 > Benchmark: 3 > Timeout: 9 > Invalid: 1 > Total: 566 > Average test time: 0:00:01.168632 > Testing time : 0:11:01.445923 > > > ********************************* > > realview_pbx_a9_qemu > > ********************************* > -------------------------- > > RSB-QEMU > > -------------------------- > > qemu-system-arm -no-reboot -monitor null -serial stdio -nographic -M > realview-pbx-a9 -kernel... > > --------------- > SAMPLES > --------------- > > Passed: 11 > Failed: 0 > User Input: 1 > Expected Fail: 0 > Indeterminate: 0 > Benchmark: 0 > Timeout: 0 > Invalid: 1 > Total: 13 > Average test time: 0:00:01.656712 > Testing time : 0:00:21.537256 > > -------------- > FULL > -------------- > > Passed: 547 > Failed: 3 > User Input: 4 > Expected Fail: 0 > Indeterminate: 0 > Benchmark: 3 > Timeout: 8 > Invalid: 1 > Total: 566 > Average test time: 0:00:01.175699 > Testing time : 0:11:05.445813 > > ------------------------ > > COUVERTURE-QEMU > > ------------------------ > > qemu-system-arm -no-reboot -monitor null -serial stdio -nographic -M > realview-pbx-a9 -kernel... > > --------------- > SAMPLES > --------------- > > > Passed: 11 > Failed: 0 > User Input: 1 > Expected Fail: 0 > Indeterminate: 0 > Benchmark: 0 > Timeout: 0 > Invalid: 1 > Total: 13 > Average test time: 0:00:00.617387 > Testing time : 0:00:08.026043 > > -------------- > FULL > -------------- > > Passed: 553 > Failed: 4 > User Input: 4 > Expected Fail: 0 > Indeterminate: 0 > Benchmark: 3 > Timeout: 1 > Invalid: 1 > Total: 566 > Average test time: 0:00:00.671176 > Testing time : 0:06:19.886106 > >
_______________________________________________ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel