Hi Kees, Indeed that's a lot of failures!
Regarding the failures during flash, the flashing over USB workflow is rather fragile: if something went wrong (a crash) when testing an application, the board might no recover and cannot be flashed automatically anymore after. There's also the issue with connecting the terminal application to stdio that can timeout sometimes. The main issues are with the "failures during test" list. I tried some of them on arduino-mkr1000 and also got failures. So there are bugs. Note that you can run each of them individually using "make BOARD=<your board> -C tests/<test application> flash test". This is what is called by the "compile_and_test_board.py" script. Regarding "tests/bench_xtimer", you can fix the build issue by adding your board to the LOW_MEMORY_BOARDS list [1]. For the other build failures, I guess some Python packages are missing. Maybe add BUILD_IN_DOCKER=1 to the test command. Alex [1] https://github.com/RIOT-OS/RIOT/blob/master/tests/bench_xtimer/Makefile#L7 ----- Le 31 Mai 20, à 21:06, Kees Bakker k...@ijzerbout.nl a écrit : > Let me explain a little bit more why I am struggling right now. > > First thing you should know is that I'm creating the support for > a new SODAQ board (Sara SFF). > > Second, I haven't run the automated tests before, so I don't > know what to expect. > > The complete run gives me the result below. No need to comment > on it. I just wanted to let you know what I have to chew on. I will > probably get back with more questions. > > ERROR:sodaq-sara-sff:Tests failed: 110 > Failures during compilation: > - [tests/bench_xtimer](tests/bench_xtimer/compilation.failed) > - [tests/pkg_nanopb](tests/pkg_nanopb/compilation.failed) > - [tests/suit_manifest](tests/suit_manifest/compilation.failed) > > Failures during test: > - [examples/micropython](examples/micropython/test.failed) > - [tests/bench_runtime_coreapis](tests/bench_runtime_coreapis/test.failed) > - [tests/bench_sizeof_coretypes](tests/bench_sizeof_coretypes/test.failed) > - [tests/bloom_bytes](tests/bloom_bytes/test.failed) > - [tests/cond_order](tests/cond_order/test.failed) > - > [tests/cpp11_condition_variable](tests/cpp11_condition_variable/test.failed) > - [tests/cpp11_mutex](tests/cpp11_mutex/test.failed) > - [tests/cpp11_thread](tests/cpp11_thread/test.failed) > - [tests/gnrc_ipv6_nib_dns](tests/gnrc_ipv6_nib_dns/test.failed) > - [tests/mutex_order](tests/mutex_order/test.failed) > - [tests/periph_gpio](tests/periph_gpio/test.failed) > - [tests/periph_rtt](tests/periph_rtt/test.failed) > - [tests/periph_timer](tests/periph_timer/test.failed) > - [tests/periph_timer_periodic](tests/periph_timer_periodic/test.failed) > - > [tests/periph_timer_short_relative_set](tests/periph_timer_short_relative_set/test.failed) > - [tests/periph_uart_nonblocking](tests/periph_uart_nonblocking/test.failed) > - [tests/periph_wdt](tests/periph_wdt/test.failed) > - [tests/pkg_c25519](tests/pkg_c25519/test.failed) > - [tests/pkg_cifra](tests/pkg_cifra/test.failed) > - [tests/pkg_cn-cbor](tests/pkg_cn-cbor/test.failed) > - [tests/pkg_hacl](tests/pkg_hacl/test.failed) > - [tests/pkg_heatshrink](tests/pkg_heatshrink/test.failed) > - [tests/pkg_libb2](tests/pkg_libb2/test.failed) > - [tests/pkg_libbase58](tests/pkg_libbase58/test.failed) > - [tests/pkg_libcose](tests/pkg_libcose/test.failed) > - [tests/pkg_libhydrogen](tests/pkg_libhydrogen/test.failed) > - [tests/pkg_littlefs](tests/pkg_littlefs/test.failed) > - [tests/pkg_littlefs2](tests/pkg_littlefs2/test.failed) > - [tests/pkg_monocypher](tests/pkg_monocypher/test.failed) > - [tests/pkg_nanocbor](tests/pkg_nanocbor/test.failed) > - [tests/pkg_qcbor](tests/pkg_qcbor/test.failed) > - [tests/pkg_qdsa](tests/pkg_qdsa/test.failed) > - [tests/pkg_relic](tests/pkg_relic/test.failed) > - [tests/pkg_spiffs](tests/pkg_spiffs/test.failed) > - [tests/pkg_tinycbor](tests/pkg_tinycbor/test.failed) > - [tests/pkg_tweetnacl](tests/pkg_tweetnacl/test.failed) > - [tests/pkg_yxml](tests/pkg_yxml/test.failed) > - [tests/posix_semaphore](tests/posix_semaphore/test.failed) > - [tests/progress_bar](tests/progress_bar/test.failed) > - [tests/pthread_tls](tests/pthread_tls/test.failed) > - [tests/riotboot_hdr](tests/riotboot_hdr/test.failed) > - [tests/rmutex](tests/rmutex/test.failed) > - [tests/rmutex_cpp](tests/rmutex_cpp/test.failed) > - [tests/shell](tests/shell/test.failed) > - [tests/sys_arduino](tests/sys_arduino/test.failed) > - [tests/sys_crypto](tests/sys_crypto/test.failed) > - [tests/thread_msg_seq](tests/thread_msg_seq/test.failed) > - [tests/usbus](tests/usbus/test.failed) > > Failures during test.flash: > - [tests/cpp_ctors](tests/cpp_ctors/test.flash.failed) > - [tests/cpp_exclude](tests/cpp_exclude/test.flash.failed) > - [tests/cpp_ext](tests/cpp_ext/test.flash.failed) > - [tests/driver_apds99xx](tests/driver_apds99xx/test.flash.failed) > - [tests/driver_apds99xx_full](tests/driver_apds99xx_full/test.flash.failed) > - [tests/driver_bme680](tests/driver_bme680/test.flash.failed) > - [tests/driver_ds1307](tests/driver_ds1307/test.flash.failed) > - [tests/driver_grove_ledbar](tests/driver_grove_ledbar/test.flash.failed) > - [tests/driver_hd44780](tests/driver_hd44780/test.flash.failed) > - [tests/driver_my9221](tests/driver_my9221/test.flash.failed) > - [tests/embunit](tests/embunit/test.flash.failed) > - [tests/emcute](tests/emcute/test.flash.failed) > - [tests/float](tests/float/test.flash.failed) > - [tests/fmt_print](tests/fmt_print/test.flash.failed) > - [tests/gnrc_dhcpv6_client](tests/gnrc_dhcpv6_client/test.flash.failed) > - > [tests/gnrc_dhcpv6_client_6lbr](tests/gnrc_dhcpv6_client_6lbr/test.flash.failed) > - [tests/gnrc_ipv6_ext](tests/gnrc_ipv6_ext/test.flash.failed) > - [tests/gnrc_ipv6_ext_frag](tests/gnrc_ipv6_ext_frag/test.flash.failed) > - [tests/gnrc_ipv6_ext_opt](tests/gnrc_ipv6_ext_opt/test.flash.failed) > - [tests/gnrc_ipv6_fwd_w_sub](tests/gnrc_ipv6_fwd_w_sub/test.flash.failed) > - [tests/gnrc_ipv6_nib](tests/gnrc_ipv6_nib/test.flash.failed) > - [tests/gnrc_ipv6_nib_6ln](tests/gnrc_ipv6_nib_6ln/test.flash.failed) > - [tests/gnrc_ndp](tests/gnrc_ndp/test.flash.failed) > - [tests/gnrc_netif](tests/gnrc_netif/test.flash.failed) > - [tests/gnrc_rpl_p2p](tests/gnrc_rpl_p2p/test.flash.failed) > - [tests/gnrc_rpl_srh](tests/gnrc_rpl_srh/test.flash.failed) > - [tests/gnrc_sixlowpan](tests/gnrc_sixlowpan/test.flash.failed) > - [tests/gnrc_sixlowpan_frag](tests/gnrc_sixlowpan_frag/test.flash.failed) > - > [tests/gnrc_sixlowpan_iphc_w_vrb](tests/gnrc_sixlowpan_iphc_w_vrb/test.flash.failed) > - > [tests/gnrc_sock_async_event](tests/gnrc_sock_async_event/test.flash.failed) > - [tests/gnrc_sock_dns](tests/gnrc_sock_dns/test.flash.failed) > - [tests/gnrc_sock_ip](tests/gnrc_sock_ip/test.flash.failed) > - [tests/gnrc_sock_neterr](tests/gnrc_sock_neterr/test.flash.failed) > - [tests/gnrc_sock_udp](tests/gnrc_sock_udp/test.flash.failed) > - [tests/gnrc_tcp](tests/gnrc_tcp/test.flash.failed) > - [tests/heap_cmd](tests/heap_cmd/test.flash.failed) > - [tests/irq](tests/irq/test.flash.failed) > - [tests/irq_disable_restore](tests/irq_disable_restore/test.flash.failed) > - [tests/isr_yield_higher](tests/isr_yield_higher/test.flash.failed) > - [tests/kconfig](tests/kconfig/test.flash.failed) > - [tests/l2util](tests/l2util/test.flash.failed) > - [tests/libc_newlib](tests/libc_newlib/test.flash.failed) > - [tests/log_color](tests/log_color/test.flash.failed) > - [tests/log_printfnoformat](tests/log_printfnoformat/test.flash.failed) > - [tests/lwip_sock_ip](tests/lwip_sock_ip/test.flash.failed) > - [tests/lwip_sock_tcp](tests/lwip_sock_tcp/test.flash.failed) > - [tests/lwip_sock_udp](tests/lwip_sock_udp/test.flash.failed) > - [tests/malloc](tests/malloc/test.flash.failed) > - [tests/memarray](tests/memarray/test.flash.failed) > - [tests/msg_avail](tests/msg_avail/test.flash.failed) > - [tests/msg_send_receive](tests/msg_send_receive/test.flash.failed) > - [tests/msg_try_receive](tests/msg_try_receive/test.flash.failed) > - [tests/mtd_flashpage](tests/mtd_flashpage/test.flash.failed) > - [tests/mtd_mapper](tests/mtd_mapper/test.flash.failed) > - [tests/pkg_tensorflow-lite](tests/pkg_tensorflow-lite/test.flash.failed) > - [tests/stdin](tests/stdin/test.flash.failed) > - [tests/vfs_plus_stdio](tests/vfs_plus_stdio/test.flash.failed) > - > [tests/xtimer_mutex_lock_timeout](tests/xtimer_mutex_lock_timeout/test.flash.failed) > - > [tests/xtimer_periodic_wakeup](tests/xtimer_periodic_wakeup/test.flash.failed) > > > On 30-05-2020 22:19, Kees Bakker wrote: >> Well, not easier for me, I'm afraid. It will delay adding my board >> even longer (which is already several months in progress). >> And it's just a hobby for me. Otherwise it begins to look like >> work :-) >> >> Nevertheless, I'll see what I can do. >> >> On 30-05-2020 22:14, Alexandre Abadie wrote: >>> Hi, >>> >>> You can just put all of them in the same issue. It will be easier to track. >>> That >>> is what is done in [1]. >>> >>> Alex >>> >>> [1] https://github.com/RIOT-OS/RIOT/issues/12651 >>> >>> ----- Le 30 Mai 20, à 21:42, Kees Bakker k...@ijzerbout.nl a écrit : >>> >>>> OK, first one created. I'm afraid there are many more to come. >>>> >>>> On 30-05-2020 21:27, Alexandre Abadie wrote: >>>>> Hi Kees, >>>>> >>>>> To my knowledge, there is no wiki page for this kind of thing. >>>>> The simplest is probably to open an issue and list there the failures >>>>> with their >>>>> output. This way we can easily track the on going work to fix them. >>>>> >>>>> See [1] as an example. >>>>> >>>>> Alex >>>>> >>>>> [1] https://github.com/RIOT-OS/RIOT/issues/12651 >>>>> >>>>> ----- Le 29 Mai 20, à 23:14, Kees Bakker k...@ijzerbout.nl a écrit : >>>>> >>>>>> Hi, >>>>>> >>>>>> Now that I'm happily running automated tests for my SAMD21 board(s) I am >>>>>> wondering which tests should succeed. There are several that fail, but I >>>>>> don't know if that "normal" or not. >>>>>> >>>>>> Some examples of test that fail >>>>>> >>>>>> xtimer_periodic_wakeup: hangs at the end, last couple of printf don't >>>>>> come out >>>>>> tests/thread_flood: [ERROR] expected 30, created 29 >>>>>> >>>>>> Is there maybe a wiki with the status of the tests (per board?). >>>>>> -- >>>>>> Kees >>>>>> _______________________________________________ >>>>>> devel mailing list >>>>>> devel@riot-os.org >>>>>> https://lists.riot-os.org/mailman/listinfo/devel >>>>> _______________________________________________ >>>>> devel mailing list >>>>> devel@riot-os.org >>>>> https://lists.riot-os.org/mailman/listinfo/devel >>>> _______________________________________________ >>>> devel mailing list >>>> devel@riot-os.org >>>> https://lists.riot-os.org/mailman/listinfo/devel >>> _______________________________________________ >>> devel mailing list >>> devel@riot-os.org >>> https://lists.riot-os.org/mailman/listinfo/devel >> _______________________________________________ >> devel mailing list >> devel@riot-os.org >> https://lists.riot-os.org/mailman/listinfo/devel > > _______________________________________________ > devel mailing list > devel@riot-os.org > https://lists.riot-os.org/mailman/listinfo/devel _______________________________________________ devel mailing list devel@riot-os.org https://lists.riot-os.org/mailman/listinfo/devel
