Re: [riot-devel] Which tests are expected to succeed for my board
Hi, > I'll look at LOW_MEMORY_BOARDS, but if other SAMD21 don't need it, then > my board > shouldn't need it either. All other SAMD21 boards with stdio over USB are in this list so I think yours should be there as well :) > I've seen strange problems with > puts/printf, where > part of the output is missing (and thus I get test timeouts). Same here, I think there's an issue stdio over USB. If the string to be printed starts with a space or a tab, it's not printed over USB. Alex - Le 1 Juin 20, à 10:58, Kees Bakker k...@ijzerbout.nl a écrit : > Thanks Alex, > > The tests that need sudo, I've skipped. For now I don't care about > these. I was able to > rerun quite a few tests by skipping the ones that caused the the board > to hang. I also > skip all the tests that need sudo. For now I don't care about these. > > I'll look at LOW_MEMORY_BOARDS, but if other SAMD21 don't need it, then > my board > shouldn't need it either. > > I also need to look at my execution environment. I don't docker (yet), > so maybe I'm > using incompatible gcc and/or newlib. I've seen strange problems with > puts/printf, where > part of the output is missing (and thus I get test timeouts). > -- Kees > > On 31-05-2020 21:40, Alexandre Abadie wrote: >> Hi Kees, >> >> Indeed that's a lot of failures! >> >> Regarding the failures during flash, the flashing over USB workflow is rather >> fragile: if something went wrong (a crash) when testing an application, the >> board might no recover and cannot be flashed automatically anymore after. >> There's also the issue with connecting the terminal application to stdio that >> can timeout sometimes. >> >> The main issues are with the "failures during test" list. I tried some of >> them >> on arduino-mkr1000 and also got failures. So there are bugs. >> >> Note that you can run each of them individually using "make BOARD=> board> >> -C tests/ flash test". This is what is called by the >> "compile_and_test_board.py" script. >> >> Regarding "tests/bench_xtimer", you can fix the build issue by adding your >> board >> to the LOW_MEMORY_BOARDS list [1]. For the other build failures, I guess some >> Python packages are missing. Maybe add BUILD_IN_DOCKER=1 to the test command. >> >> Alex >> >> [1] >> https://github.com/RIOT-OS/RIOT/blob/master/tests/bench_xtimer/Makefile#L7 >> >> - Le 31 Mai 20, à 21:06, Kees Bakker k...@ijzerbout.nl a écrit : >> >>> Let me explain a little bit more why I am struggling right now. >>> >>> First thing you should know is that I'm creating the support for >>> a new SODAQ board (Sara SFF). >>> >>> Second, I haven't run the automated tests before, so I don't >>> know what to expect. >>> >>> The complete run gives me the result below. No need to comment >>> on it. I just wanted to let you know what I have to chew on. I will >>> probably get back with more questions. >>> >>> ERROR:sodaq-sara-sff:Tests failed: 110 >>> Failures during compilation: >>> - [tests/bench_xtimer](tests/bench_xtimer/compilation.failed) >>> - [tests/pkg_nanopb](tests/pkg_nanopb/compilation.failed) >>> - [tests/suit_manifest](tests/suit_manifest/compilation.failed) >>> >>> Failures during test: >>> - [examples/micropython](examples/micropython/test.failed) >>> - [tests/bench_runtime_coreapis](tests/bench_runtime_coreapis/test.failed) >>> - [tests/bench_sizeof_coretypes](tests/bench_sizeof_coretypes/test.failed) >>> - [tests/bloom_bytes](tests/bloom_bytes/test.failed) >>> - [tests/cond_order](tests/cond_order/test.failed) >>> - >>> [tests/cpp11_condition_variable](tests/cpp11_condition_variable/test.failed) >>> - [tests/cpp11_mutex](tests/cpp11_mutex/test.failed) >>> - [tests/cpp11_thread](tests/cpp11_thread/test.failed) >>> - [tests/gnrc_ipv6_nib_dns](tests/gnrc_ipv6_nib_dns/test.failed) >>> - [tests/mutex_order](tests/mutex_order/test.failed) >>> - [tests/periph_gpio](tests/periph_gpio/test.failed) >>> - [tests/periph_rtt](tests/periph_rtt/test.failed) >>> - [tests/periph_timer](tests/periph_timer/test.failed) >>> - [tests/periph_timer_periodic](tests/periph_timer_periodic/test.failed) >>> - >>> [tests/periph_timer_short_relative_set](tests/periph_timer_short_relative_set/test.failed) >>> - [tests/periph_uart_nonblocking](tests/periph_uart_nonblocking/test.failed) >>> - [tests/periph_wdt](tests/periph_wdt/test.failed) >>> - [tests/pkg_c25519](tests/pkg_c25519/test.failed) >>> - [tests/pkg_cifra](tests/pkg_cifra/test.failed) >>> - [tests/pkg_cn-cbor](tests/pkg_cn-cbor/test.failed) >>> - [tests/pkg_hacl](tests/pkg_hacl/test.failed) >>> - [tests/pkg_heatshrink](tests/pkg_heatshrink/test.failed) >>> - [tests/pkg_libb2](tests/pkg_libb2/test.failed) >>> - [tests/pkg_libbase58](tests/pkg_libbase58/test.failed) >>> - [tests/pkg_libcose](tests/pkg_libcose/test.failed) >>> - [tests/pkg_libhydrogen](tests/pkg_libhydrogen/test.failed) >>> - [tests/pkg_littlefs](tests/pkg_littlefs/test.failed) >>> - [tests/pkg_littlefs2](tests/pkg_littlefs2/test.failed) >>> -
Re: [riot-devel] Which tests are expected to succeed for my board
Hi Kees, Am 31.05.20 um 20:49 schrieb Kees Bakker: > On 31-05-2020 12:33, Martine Sophie Lenders wrote: >> Hi, >> >> Am 31.05.20 um 12:25 schrieb Kaspar Schleiser: >>> "... FAILED (due to $reason)", and maybe not change return code to >>> something that's an error. >> While I like this idea I foresee a usage problem since some tests in >> that script fail due to missing root permissions (or missing >> scapy_unroot). If we say 'FAILED (due to missing root permissions)' I >> virtually expect any sane developer to re-run the script then with >> `sudo`, leading to people wondering why they are not able to compile >> applications anymore, just using `make all`, as the `bin` directory was >> created with root permission. Sure this can be easily fixed, I just say >> "mind the consequences" ;-). >> > I don't agree with the "sane" part in that sentence. ;-) I forgot that sarcasm does not translate well to text ;-) Best regards, Martine ___ devel mailing list devel@riot-os.org https://lists.riot-os.org/mailman/listinfo/devel
Re: [riot-devel] Which tests are expected to succeed for my board
Thanks Alex, The tests that need sudo, I've skipped. For now I don't care about these. I was able to rerun quite a few tests by skipping the ones that caused the the board to hang. I also skip all the tests that need sudo. For now I don't care about these. I'll look at LOW_MEMORY_BOARDS, but if other SAMD21 don't need it, then my board shouldn't need it either. I also need to look at my execution environment. I don't docker (yet), so maybe I'm using incompatible gcc and/or newlib. I've seen strange problems with puts/printf, where part of the output is missing (and thus I get test timeouts). -- Kees On 31-05-2020 21:40, Alexandre Abadie wrote: > Hi Kees, > > Indeed that's a lot of failures! > > Regarding the failures during flash, the flashing over USB workflow is rather > fragile: if something went wrong (a crash) when testing an application, the > board might no recover and cannot be flashed automatically anymore after. > There's also the issue with connecting the terminal application to stdio that > can timeout sometimes. > > The main issues are with the "failures during test" list. I tried some of > them on arduino-mkr1000 and also got failures. So there are bugs. > > Note that you can run each of them individually using "make BOARD= board> -C tests/ flash test". This is what is called by the > "compile_and_test_board.py" script. > > Regarding "tests/bench_xtimer", you can fix the build issue by adding your > board to the LOW_MEMORY_BOARDS list [1]. For the other build failures, I > guess some Python packages are missing. Maybe add BUILD_IN_DOCKER=1 to the > test command. > > Alex > > [1] https://github.com/RIOT-OS/RIOT/blob/master/tests/bench_xtimer/Makefile#L7 > > - Le 31 Mai 20, à 21:06, Kees Bakker k...@ijzerbout.nl a écrit : > >> Let me explain a little bit more why I am struggling right now. >> >> First thing you should know is that I'm creating the support for >> a new SODAQ board (Sara SFF). >> >> Second, I haven't run the automated tests before, so I don't >> know what to expect. >> >> The complete run gives me the result below. No need to comment >> on it. I just wanted to let you know what I have to chew on. I will >> probably get back with more questions. >> >> ERROR:sodaq-sara-sff:Tests failed: 110 >> Failures during compilation: >> - [tests/bench_xtimer](tests/bench_xtimer/compilation.failed) >> - [tests/pkg_nanopb](tests/pkg_nanopb/compilation.failed) >> - [tests/suit_manifest](tests/suit_manifest/compilation.failed) >> >> Failures during test: >> - [examples/micropython](examples/micropython/test.failed) >> - [tests/bench_runtime_coreapis](tests/bench_runtime_coreapis/test.failed) >> - [tests/bench_sizeof_coretypes](tests/bench_sizeof_coretypes/test.failed) >> - [tests/bloom_bytes](tests/bloom_bytes/test.failed) >> - [tests/cond_order](tests/cond_order/test.failed) >> - >> [tests/cpp11_condition_variable](tests/cpp11_condition_variable/test.failed) >> - [tests/cpp11_mutex](tests/cpp11_mutex/test.failed) >> - [tests/cpp11_thread](tests/cpp11_thread/test.failed) >> - [tests/gnrc_ipv6_nib_dns](tests/gnrc_ipv6_nib_dns/test.failed) >> - [tests/mutex_order](tests/mutex_order/test.failed) >> - [tests/periph_gpio](tests/periph_gpio/test.failed) >> - [tests/periph_rtt](tests/periph_rtt/test.failed) >> - [tests/periph_timer](tests/periph_timer/test.failed) >> - [tests/periph_timer_periodic](tests/periph_timer_periodic/test.failed) >> - >> [tests/periph_timer_short_relative_set](tests/periph_timer_short_relative_set/test.failed) >> - [tests/periph_uart_nonblocking](tests/periph_uart_nonblocking/test.failed) >> - [tests/periph_wdt](tests/periph_wdt/test.failed) >> - [tests/pkg_c25519](tests/pkg_c25519/test.failed) >> - [tests/pkg_cifra](tests/pkg_cifra/test.failed) >> - [tests/pkg_cn-cbor](tests/pkg_cn-cbor/test.failed) >> - [tests/pkg_hacl](tests/pkg_hacl/test.failed) >> - [tests/pkg_heatshrink](tests/pkg_heatshrink/test.failed) >> - [tests/pkg_libb2](tests/pkg_libb2/test.failed) >> - [tests/pkg_libbase58](tests/pkg_libbase58/test.failed) >> - [tests/pkg_libcose](tests/pkg_libcose/test.failed) >> - [tests/pkg_libhydrogen](tests/pkg_libhydrogen/test.failed) >> - [tests/pkg_littlefs](tests/pkg_littlefs/test.failed) >> - [tests/pkg_littlefs2](tests/pkg_littlefs2/test.failed) >> - [tests/pkg_monocypher](tests/pkg_monocypher/test.failed) >> - [tests/pkg_nanocbor](tests/pkg_nanocbor/test.failed) >> - [tests/pkg_qcbor](tests/pkg_qcbor/test.failed) >> - [tests/pkg_qdsa](tests/pkg_qdsa/test.failed) >> - [tests/pkg_relic](tests/pkg_relic/test.failed) >> - [tests/pkg_spiffs](tests/pkg_spiffs/test.failed) >> - [tests/pkg_tinycbor](tests/pkg_tinycbor/test.failed) >> - [tests/pkg_tweetnacl](tests/pkg_tweetnacl/test.failed) >> - [tests/pkg_yxml](tests/pkg_yxml/test.failed) >> - [tests/posix_semaphore](tests/posix_semaphore/test.failed) >> - [tests/progress_bar](tests/progress_bar/test.failed) >> - [tests/pthread_tls](tests/pthread_tls/test.failed) >> -
Re: [riot-devel] Which tests are expected to succeed for my board
Hi Kees, Indeed that's a lot of failures! Regarding the failures during flash, the flashing over USB workflow is rather fragile: if something went wrong (a crash) when testing an application, the board might no recover and cannot be flashed automatically anymore after. There's also the issue with connecting the terminal application to stdio that can timeout sometimes. The main issues are with the "failures during test" list. I tried some of them on arduino-mkr1000 and also got failures. So there are bugs. Note that you can run each of them individually using "make BOARD= -C tests/ flash test". This is what is called by the "compile_and_test_board.py" script. Regarding "tests/bench_xtimer", you can fix the build issue by adding your board to the LOW_MEMORY_BOARDS list [1]. For the other build failures, I guess some Python packages are missing. Maybe add BUILD_IN_DOCKER=1 to the test command. Alex [1] https://github.com/RIOT-OS/RIOT/blob/master/tests/bench_xtimer/Makefile#L7 - Le 31 Mai 20, à 21:06, Kees Bakker k...@ijzerbout.nl a écrit : > Let me explain a little bit more why I am struggling right now. > > First thing you should know is that I'm creating the support for > a new SODAQ board (Sara SFF). > > Second, I haven't run the automated tests before, so I don't > know what to expect. > > The complete run gives me the result below. No need to comment > on it. I just wanted to let you know what I have to chew on. I will > probably get back with more questions. > > ERROR:sodaq-sara-sff:Tests failed: 110 > Failures during compilation: > - [tests/bench_xtimer](tests/bench_xtimer/compilation.failed) > - [tests/pkg_nanopb](tests/pkg_nanopb/compilation.failed) > - [tests/suit_manifest](tests/suit_manifest/compilation.failed) > > Failures during test: > - [examples/micropython](examples/micropython/test.failed) > - [tests/bench_runtime_coreapis](tests/bench_runtime_coreapis/test.failed) > - [tests/bench_sizeof_coretypes](tests/bench_sizeof_coretypes/test.failed) > - [tests/bloom_bytes](tests/bloom_bytes/test.failed) > - [tests/cond_order](tests/cond_order/test.failed) > - > [tests/cpp11_condition_variable](tests/cpp11_condition_variable/test.failed) > - [tests/cpp11_mutex](tests/cpp11_mutex/test.failed) > - [tests/cpp11_thread](tests/cpp11_thread/test.failed) > - [tests/gnrc_ipv6_nib_dns](tests/gnrc_ipv6_nib_dns/test.failed) > - [tests/mutex_order](tests/mutex_order/test.failed) > - [tests/periph_gpio](tests/periph_gpio/test.failed) > - [tests/periph_rtt](tests/periph_rtt/test.failed) > - [tests/periph_timer](tests/periph_timer/test.failed) > - [tests/periph_timer_periodic](tests/periph_timer_periodic/test.failed) > - > [tests/periph_timer_short_relative_set](tests/periph_timer_short_relative_set/test.failed) > - [tests/periph_uart_nonblocking](tests/periph_uart_nonblocking/test.failed) > - [tests/periph_wdt](tests/periph_wdt/test.failed) > - [tests/pkg_c25519](tests/pkg_c25519/test.failed) > - [tests/pkg_cifra](tests/pkg_cifra/test.failed) > - [tests/pkg_cn-cbor](tests/pkg_cn-cbor/test.failed) > - [tests/pkg_hacl](tests/pkg_hacl/test.failed) > - [tests/pkg_heatshrink](tests/pkg_heatshrink/test.failed) > - [tests/pkg_libb2](tests/pkg_libb2/test.failed) > - [tests/pkg_libbase58](tests/pkg_libbase58/test.failed) > - [tests/pkg_libcose](tests/pkg_libcose/test.failed) > - [tests/pkg_libhydrogen](tests/pkg_libhydrogen/test.failed) > - [tests/pkg_littlefs](tests/pkg_littlefs/test.failed) > - [tests/pkg_littlefs2](tests/pkg_littlefs2/test.failed) > - [tests/pkg_monocypher](tests/pkg_monocypher/test.failed) > - [tests/pkg_nanocbor](tests/pkg_nanocbor/test.failed) > - [tests/pkg_qcbor](tests/pkg_qcbor/test.failed) > - [tests/pkg_qdsa](tests/pkg_qdsa/test.failed) > - [tests/pkg_relic](tests/pkg_relic/test.failed) > - [tests/pkg_spiffs](tests/pkg_spiffs/test.failed) > - [tests/pkg_tinycbor](tests/pkg_tinycbor/test.failed) > - [tests/pkg_tweetnacl](tests/pkg_tweetnacl/test.failed) > - [tests/pkg_yxml](tests/pkg_yxml/test.failed) > - [tests/posix_semaphore](tests/posix_semaphore/test.failed) > - [tests/progress_bar](tests/progress_bar/test.failed) > - [tests/pthread_tls](tests/pthread_tls/test.failed) > - [tests/riotboot_hdr](tests/riotboot_hdr/test.failed) > - [tests/rmutex](tests/rmutex/test.failed) > - [tests/rmutex_cpp](tests/rmutex_cpp/test.failed) > - [tests/shell](tests/shell/test.failed) > - [tests/sys_arduino](tests/sys_arduino/test.failed) > - [tests/sys_crypto](tests/sys_crypto/test.failed) > - [tests/thread_msg_seq](tests/thread_msg_seq/test.failed) > - [tests/usbus](tests/usbus/test.failed) > > Failures during test.flash: > - [tests/cpp_ctors](tests/cpp_ctors/test.flash.failed) > - [tests/cpp_exclude](tests/cpp_exclude/test.flash.failed) > - [tests/cpp_ext](tests/cpp_ext/test.flash.failed) > - [tests/driver_apds99xx](tests/driver_apds99xx/test.flash.failed) > - [tests/driver_apds99xx_full](tests/driver_apds99xx_full/test.flash.failed) > -
Re: [riot-devel] Which tests are expected to succeed for my board
Let me explain a little bit more why I am struggling right now. First thing you should know is that I'm creating the support for a new SODAQ board (Sara SFF). Second, I haven't run the automated tests before, so I don't know what to expect. The complete run gives me the result below. No need to comment on it. I just wanted to let you know what I have to chew on. I will probably get back with more questions. ERROR:sodaq-sara-sff:Tests failed: 110 Failures during compilation: - [tests/bench_xtimer](tests/bench_xtimer/compilation.failed) - [tests/pkg_nanopb](tests/pkg_nanopb/compilation.failed) - [tests/suit_manifest](tests/suit_manifest/compilation.failed) Failures during test: - [examples/micropython](examples/micropython/test.failed) - [tests/bench_runtime_coreapis](tests/bench_runtime_coreapis/test.failed) - [tests/bench_sizeof_coretypes](tests/bench_sizeof_coretypes/test.failed) - [tests/bloom_bytes](tests/bloom_bytes/test.failed) - [tests/cond_order](tests/cond_order/test.failed) - [tests/cpp11_condition_variable](tests/cpp11_condition_variable/test.failed) - [tests/cpp11_mutex](tests/cpp11_mutex/test.failed) - [tests/cpp11_thread](tests/cpp11_thread/test.failed) - [tests/gnrc_ipv6_nib_dns](tests/gnrc_ipv6_nib_dns/test.failed) - [tests/mutex_order](tests/mutex_order/test.failed) - [tests/periph_gpio](tests/periph_gpio/test.failed) - [tests/periph_rtt](tests/periph_rtt/test.failed) - [tests/periph_timer](tests/periph_timer/test.failed) - [tests/periph_timer_periodic](tests/periph_timer_periodic/test.failed) - [tests/periph_timer_short_relative_set](tests/periph_timer_short_relative_set/test.failed) - [tests/periph_uart_nonblocking](tests/periph_uart_nonblocking/test.failed) - [tests/periph_wdt](tests/periph_wdt/test.failed) - [tests/pkg_c25519](tests/pkg_c25519/test.failed) - [tests/pkg_cifra](tests/pkg_cifra/test.failed) - [tests/pkg_cn-cbor](tests/pkg_cn-cbor/test.failed) - [tests/pkg_hacl](tests/pkg_hacl/test.failed) - [tests/pkg_heatshrink](tests/pkg_heatshrink/test.failed) - [tests/pkg_libb2](tests/pkg_libb2/test.failed) - [tests/pkg_libbase58](tests/pkg_libbase58/test.failed) - [tests/pkg_libcose](tests/pkg_libcose/test.failed) - [tests/pkg_libhydrogen](tests/pkg_libhydrogen/test.failed) - [tests/pkg_littlefs](tests/pkg_littlefs/test.failed) - [tests/pkg_littlefs2](tests/pkg_littlefs2/test.failed) - [tests/pkg_monocypher](tests/pkg_monocypher/test.failed) - [tests/pkg_nanocbor](tests/pkg_nanocbor/test.failed) - [tests/pkg_qcbor](tests/pkg_qcbor/test.failed) - [tests/pkg_qdsa](tests/pkg_qdsa/test.failed) - [tests/pkg_relic](tests/pkg_relic/test.failed) - [tests/pkg_spiffs](tests/pkg_spiffs/test.failed) - [tests/pkg_tinycbor](tests/pkg_tinycbor/test.failed) - [tests/pkg_tweetnacl](tests/pkg_tweetnacl/test.failed) - [tests/pkg_yxml](tests/pkg_yxml/test.failed) - [tests/posix_semaphore](tests/posix_semaphore/test.failed) - [tests/progress_bar](tests/progress_bar/test.failed) - [tests/pthread_tls](tests/pthread_tls/test.failed) - [tests/riotboot_hdr](tests/riotboot_hdr/test.failed) - [tests/rmutex](tests/rmutex/test.failed) - [tests/rmutex_cpp](tests/rmutex_cpp/test.failed) - [tests/shell](tests/shell/test.failed) - [tests/sys_arduino](tests/sys_arduino/test.failed) - [tests/sys_crypto](tests/sys_crypto/test.failed) - [tests/thread_msg_seq](tests/thread_msg_seq/test.failed) - [tests/usbus](tests/usbus/test.failed) Failures during test.flash: - [tests/cpp_ctors](tests/cpp_ctors/test.flash.failed) - [tests/cpp_exclude](tests/cpp_exclude/test.flash.failed) - [tests/cpp_ext](tests/cpp_ext/test.flash.failed) - [tests/driver_apds99xx](tests/driver_apds99xx/test.flash.failed) - [tests/driver_apds99xx_full](tests/driver_apds99xx_full/test.flash.failed) - [tests/driver_bme680](tests/driver_bme680/test.flash.failed) - [tests/driver_ds1307](tests/driver_ds1307/test.flash.failed) - [tests/driver_grove_ledbar](tests/driver_grove_ledbar/test.flash.failed) - [tests/driver_hd44780](tests/driver_hd44780/test.flash.failed) - [tests/driver_my9221](tests/driver_my9221/test.flash.failed) - [tests/embunit](tests/embunit/test.flash.failed) - [tests/emcute](tests/emcute/test.flash.failed) - [tests/float](tests/float/test.flash.failed) - [tests/fmt_print](tests/fmt_print/test.flash.failed) - [tests/gnrc_dhcpv6_client](tests/gnrc_dhcpv6_client/test.flash.failed) - [tests/gnrc_dhcpv6_client_6lbr](tests/gnrc_dhcpv6_client_6lbr/test.flash.failed) - [tests/gnrc_ipv6_ext](tests/gnrc_ipv6_ext/test.flash.failed) - [tests/gnrc_ipv6_ext_frag](tests/gnrc_ipv6_ext_frag/test.flash.failed) - [tests/gnrc_ipv6_ext_opt](tests/gnrc_ipv6_ext_opt/test.flash.failed) - [tests/gnrc_ipv6_fwd_w_sub](tests/gnrc_ipv6_fwd_w_sub/test.flash.failed) - [tests/gnrc_ipv6_nib](tests/gnrc_ipv6_nib/test.flash.failed) - [tests/gnrc_ipv6_nib_6ln](tests/gnrc_ipv6_nib_6ln/test.flash.failed) - [tests/gnrc_ndp](tests/gnrc_ndp/test.flash.failed) - [tests/gnrc_netif](tests/gnrc_netif/test.flash.failed) - [tests/gnrc_rpl_p2p](tests/gnrc_rpl_p2p/test.flash.failed) -
Re: [riot-devel] Which tests are expected to succeed for my board
On 31-05-2020 12:33, Martine Sophie Lenders wrote: > Hi, > > Am 31.05.20 um 12:25 schrieb Kaspar Schleiser: >> "... FAILED (due to $reason)", and maybe not change return code to >> something that's an error. > While I like this idea I foresee a usage problem since some tests in > that script fail due to missing root permissions (or missing > scapy_unroot). If we say 'FAILED (due to missing root permissions)' I > virtually expect any sane developer to re-run the script then with > `sudo`, leading to people wondering why they are not able to compile > applications anymore, just using `make all`, as the `bin` directory was > created with root permission. Sure this can be easily fixed, I just say > "mind the consequences" ;-). > I don't agree with the "sane" part in that sentence. ;-) -- Kees ___ devel mailing list devel@riot-os.org https://lists.riot-os.org/mailman/listinfo/devel
Re: [riot-devel] Which tests are expected to succeed for my board
Hi, Am 31.05.20 um 12:25 schrieb Kaspar Schleiser: > "... FAILED (due to $reason)", and maybe not change return code to > something that's an error. While I like this idea I foresee a usage problem since some tests in that script fail due to missing root permissions (or missing scapy_unroot). If we say 'FAILED (due to missing root permissions)' I virtually expect any sane developer to re-run the script then with `sudo`, leading to people wondering why they are not able to compile applications anymore, just using `make all`, as the `bin` directory was created with root permission. Sure this can be easily fixed, I just say "mind the consequences" ;-). Best regards, Martine > > Kaspar > ___ > devel mailing list > devel@riot-os.org > https://lists.riot-os.org/mailman/listinfo/devel ___ devel mailing list devel@riot-os.org https://lists.riot-os.org/mailman/listinfo/devel
Re: [riot-devel] Which tests are expected to succeed for my board
Hi, On 5/30/20 10:14 PM, Alexandre Abadie wrote: > You can just put all of them in the same issue. It will be easier to track. > That is what is done in [1]. I think we should add this information to compile_and_test.py. Some way to expect failure for specific tests or board/test combinations, and report accordingly. "... FAILED (due to $reason)", and maybe not change return code to something that's an error. Kaspar ___ devel mailing list devel@riot-os.org https://lists.riot-os.org/mailman/listinfo/devel
Re: [riot-devel] Which tests are expected to succeed for my board
Well, not easier for me, I'm afraid. It will delay adding my board even longer (which is already several months in progress). And it's just a hobby for me. Otherwise it begins to look like work :-) Nevertheless, I'll see what I can do. On 30-05-2020 22:14, Alexandre Abadie wrote: > Hi, > > You can just put all of them in the same issue. It will be easier to track. > That is what is done in [1]. > > Alex > > [1] https://github.com/RIOT-OS/RIOT/issues/12651 > > - Le 30 Mai 20, à 21:42, Kees Bakker k...@ijzerbout.nl a écrit : > >> OK, first one created. I'm afraid there are many more to come. >> >> On 30-05-2020 21:27, Alexandre Abadie wrote: >>> Hi Kees, >>> >>> To my knowledge, there is no wiki page for this kind of thing. >>> The simplest is probably to open an issue and list there the failures with >>> their >>> output. This way we can easily track the on going work to fix them. >>> >>> See [1] as an example. >>> >>> Alex >>> >>> [1] https://github.com/RIOT-OS/RIOT/issues/12651 >>> >>> - Le 29 Mai 20, à 23:14, Kees Bakker k...@ijzerbout.nl a écrit : >>> Hi, Now that I'm happily running automated tests for my SAMD21 board(s) I am wondering which tests should succeed. There are several that fail, but I don't know if that "normal" or not. Some examples of test that fail xtimer_periodic_wakeup: hangs at the end, last couple of printf don't come out tests/thread_flood: [ERROR] expected 30, created 29 Is there maybe a wiki with the status of the tests (per board?). -- Kees ___ devel mailing list devel@riot-os.org https://lists.riot-os.org/mailman/listinfo/devel >>> ___ >>> devel mailing list >>> devel@riot-os.org >>> https://lists.riot-os.org/mailman/listinfo/devel >> ___ >> devel mailing list >> devel@riot-os.org >> https://lists.riot-os.org/mailman/listinfo/devel > ___ > devel mailing list > devel@riot-os.org > https://lists.riot-os.org/mailman/listinfo/devel ___ devel mailing list devel@riot-os.org https://lists.riot-os.org/mailman/listinfo/devel
Re: [riot-devel] Which tests are expected to succeed for my board
Hi, You can just put all of them in the same issue. It will be easier to track. That is what is done in [1]. Alex [1] https://github.com/RIOT-OS/RIOT/issues/12651 - Le 30 Mai 20, à 21:42, Kees Bakker k...@ijzerbout.nl a écrit : > OK, first one created. I'm afraid there are many more to come. > > On 30-05-2020 21:27, Alexandre Abadie wrote: >> Hi Kees, >> >> To my knowledge, there is no wiki page for this kind of thing. >> The simplest is probably to open an issue and list there the failures with >> their >> output. This way we can easily track the on going work to fix them. >> >> See [1] as an example. >> >> Alex >> >> [1] https://github.com/RIOT-OS/RIOT/issues/12651 >> >> - Le 29 Mai 20, à 23:14, Kees Bakker k...@ijzerbout.nl a écrit : >> >>> Hi, >>> >>> Now that I'm happily running automated tests for my SAMD21 board(s) I am >>> wondering which tests should succeed. There are several that fail, but I >>> don't know if that "normal" or not. >>> >>> Some examples of test that fail >>> >>> xtimer_periodic_wakeup: hangs at the end, last couple of printf don't >>> come out >>> tests/thread_flood: [ERROR] expected 30, created 29 >>> >>> Is there maybe a wiki with the status of the tests (per board?). >>> -- >>> Kees >>> ___ >>> devel mailing list >>> devel@riot-os.org >>> https://lists.riot-os.org/mailman/listinfo/devel >> ___ >> devel mailing list >> devel@riot-os.org >> https://lists.riot-os.org/mailman/listinfo/devel > > ___ > devel mailing list > devel@riot-os.org > https://lists.riot-os.org/mailman/listinfo/devel ___ devel mailing list devel@riot-os.org https://lists.riot-os.org/mailman/listinfo/devel
Re: [riot-devel] Which tests are expected to succeed for my board
The reason I ask all these questions is that I'm adding support for another SODAQ board. And I was trying to do it the right way and run some tests. Maybe that was a mistake :-) Anyway, I've created the first issue. On 30-05-2020 21:30, Marian Buschsieweke wrote: > Hi, > > every failing test indicates a bug. (From time to time the bug is found in the > test rather than in the code to test.) Feel free to open an issue for the > failing tests. I personally favor a tracking issue that lists each failing > test > with a checkbox. It is motivating to tick them of one by one; and with the > tiring task of bug hunting some motivation is always welcome :-) > > Kind regards, > Marian > > On Fri, 29 May 2020 23:14:19 +0200 > Kees Bakker wrote: > >> Hi, >> >> Now that I'm happily running automated tests for my SAMD21 board(s) I am >> wondering which tests should succeed. There are several that fail, but I >> don't know if that "normal" or not. >> >> Some examples of test that fail >> >> xtimer_periodic_wakeup: hangs at the end, last couple of printf don't >> come out >> tests/thread_flood: [ERROR] expected 30, created 29 >> >> Is there maybe a wiki with the status of the tests (per board?). > > ___ > devel mailing list > devel@riot-os.org > https://lists.riot-os.org/mailman/listinfo/devel ___ devel mailing list devel@riot-os.org https://lists.riot-os.org/mailman/listinfo/devel
Re: [riot-devel] Which tests are expected to succeed for my board
OK, first one created. I'm afraid there are many more to come. On 30-05-2020 21:27, Alexandre Abadie wrote: > Hi Kees, > > To my knowledge, there is no wiki page for this kind of thing. > The simplest is probably to open an issue and list there the failures with > their output. This way we can easily track the on going work to fix them. > > See [1] as an example. > > Alex > > [1] https://github.com/RIOT-OS/RIOT/issues/12651 > > - Le 29 Mai 20, à 23:14, Kees Bakker k...@ijzerbout.nl a écrit : > >> Hi, >> >> Now that I'm happily running automated tests for my SAMD21 board(s) I am >> wondering which tests should succeed. There are several that fail, but I >> don't know if that "normal" or not. >> >> Some examples of test that fail >> >> xtimer_periodic_wakeup: hangs at the end, last couple of printf don't >> come out >> tests/thread_flood: [ERROR] expected 30, created 29 >> >> Is there maybe a wiki with the status of the tests (per board?). >> -- >> Kees >> ___ >> devel mailing list >> devel@riot-os.org >> https://lists.riot-os.org/mailman/listinfo/devel > ___ > devel mailing list > devel@riot-os.org > https://lists.riot-os.org/mailman/listinfo/devel ___ devel mailing list devel@riot-os.org https://lists.riot-os.org/mailman/listinfo/devel
Re: [riot-devel] Which tests are expected to succeed for my board
Hi, every failing test indicates a bug. (From time to time the bug is found in the test rather than in the code to test.) Feel free to open an issue for the failing tests. I personally favor a tracking issue that lists each failing test with a checkbox. It is motivating to tick them of one by one; and with the tiring task of bug hunting some motivation is always welcome :-) Kind regards, Marian On Fri, 29 May 2020 23:14:19 +0200 Kees Bakker wrote: > Hi, > > Now that I'm happily running automated tests for my SAMD21 board(s) I am > wondering which tests should succeed. There are several that fail, but I > don't know if that "normal" or not. > > Some examples of test that fail > > xtimer_periodic_wakeup: hangs at the end, last couple of printf don't > come out > tests/thread_flood: [ERROR] expected 30, created 29 > > Is there maybe a wiki with the status of the tests (per board?). pgpjm62I2o1MM.pgp Description: OpenPGP digital signature ___ devel mailing list devel@riot-os.org https://lists.riot-os.org/mailman/listinfo/devel
Re: [riot-devel] Which tests are expected to succeed for my board
Hi Kees, To my knowledge, there is no wiki page for this kind of thing. The simplest is probably to open an issue and list there the failures with their output. This way we can easily track the on going work to fix them. See [1] as an example. Alex [1] https://github.com/RIOT-OS/RIOT/issues/12651 - Le 29 Mai 20, à 23:14, Kees Bakker k...@ijzerbout.nl a écrit : > Hi, > > Now that I'm happily running automated tests for my SAMD21 board(s) I am > wondering which tests should succeed. There are several that fail, but I > don't know if that "normal" or not. > > Some examples of test that fail > > xtimer_periodic_wakeup: hangs at the end, last couple of printf don't > come out > tests/thread_flood: [ERROR] expected 30, created 29 > > Is there maybe a wiki with the status of the tests (per board?). > -- > Kees > ___ > devel mailing list > devel@riot-os.org > https://lists.riot-os.org/mailman/listinfo/devel ___ devel mailing list devel@riot-os.org https://lists.riot-os.org/mailman/listinfo/devel
[riot-devel] Which tests are expected to succeed for my board
Hi, Now that I'm happily running automated tests for my SAMD21 board(s) I am wondering which tests should succeed. There are several that fail, but I don't know if that "normal" or not. Some examples of test that fail xtimer_periodic_wakeup: hangs at the end, last couple of printf don't come out tests/thread_flood: [ERROR] expected 30, created 29 Is there maybe a wiki with the status of the tests (per board?). -- Kees ___ devel mailing list devel@riot-os.org https://lists.riot-os.org/mailman/listinfo/devel
