Re: [riot-devel] Which tests are expected to succeed for my board

2020-06-01 Thread Alexandre Abadie
Hi, > I'll look at LOW_MEMORY_BOARDS, but if other SAMD21 don't need it, then > my board > shouldn't need it either. All other SAMD21 boards with stdio over USB are in this list so I think yours should be there as well :) > I've seen strange problems with > puts/printf, where > part of the

Re: [riot-devel] Which tests are expected to succeed for my board

2020-06-01 Thread Martine Sophie Lenders
Hi Kees, Am 31.05.20 um 20:49 schrieb Kees Bakker: > On 31-05-2020 12:33, Martine Sophie Lenders wrote: >> Hi, >> >> Am 31.05.20 um 12:25 schrieb Kaspar Schleiser: >>> "... FAILED (due to $reason)", and maybe not change return code to >>> something that's an error. >> While I like this idea I

Re: [riot-devel] Which tests are expected to succeed for my board

2020-06-01 Thread Kees Bakker
Thanks Alex, The tests that need sudo, I've skipped. For now I don't care about these. I was able to rerun quite a few tests by skipping the ones that caused the the board to hang. I also skip all the tests that need sudo. For now I don't care about these. I'll look at LOW_MEMORY_BOARDS, but if

Re: [riot-devel] Which tests are expected to succeed for my board

2020-05-31 Thread Alexandre Abadie
Hi Kees, Indeed that's a lot of failures! Regarding the failures during flash, the flashing over USB workflow is rather fragile: if something went wrong (a crash) when testing an application, the board might no recover and cannot be flashed automatically anymore after. There's also the issue

Re: [riot-devel] Which tests are expected to succeed for my board

2020-05-31 Thread Kees Bakker
Let me explain a little bit more why I am struggling right now. First thing you should know is that I'm creating the support for a new SODAQ board (Sara SFF). Second, I haven't run the automated tests before, so I don't know what to expect. The complete run gives me the result below. No need to

Re: [riot-devel] Which tests are expected to succeed for my board

2020-05-31 Thread Kees Bakker
On 31-05-2020 12:33, Martine Sophie Lenders wrote: > Hi, > > Am 31.05.20 um 12:25 schrieb Kaspar Schleiser: >> "... FAILED (due to $reason)", and maybe not change return code to >> something that's an error. > While I like this idea I foresee a usage problem since some tests in > that script fail

Re: [riot-devel] Which tests are expected to succeed for my board

2020-05-31 Thread Martine Sophie Lenders
Hi, Am 31.05.20 um 12:25 schrieb Kaspar Schleiser: > "... FAILED (due to $reason)", and maybe not change return code to > something that's an error. While I like this idea I foresee a usage problem since some tests in that script fail due to missing root permissions (or missing scapy_unroot). If

Re: [riot-devel] Which tests are expected to succeed for my board

2020-05-31 Thread Kaspar Schleiser
Hi, On 5/30/20 10:14 PM, Alexandre Abadie wrote: > You can just put all of them in the same issue. It will be easier to track. > That is what is done in [1]. I think we should add this information to compile_and_test.py. Some way to expect failure for specific tests or board/test combinations,

Re: [riot-devel] Which tests are expected to succeed for my board

2020-05-30 Thread Kees Bakker
Well, not easier for me, I'm afraid. It will delay adding my board even longer (which is already several months in progress). And it's just a hobby for me. Otherwise it begins to look like work :-) Nevertheless, I'll see what I can do. On 30-05-2020 22:14, Alexandre Abadie wrote: > Hi, > > You

Re: [riot-devel] Which tests are expected to succeed for my board

2020-05-30 Thread Alexandre Abadie
Hi, You can just put all of them in the same issue. It will be easier to track. That is what is done in [1]. Alex [1] https://github.com/RIOT-OS/RIOT/issues/12651 - Le 30 Mai 20, à 21:42, Kees Bakker k...@ijzerbout.nl a écrit : > OK, first one created. I'm afraid there are many more to

Re: [riot-devel] Which tests are expected to succeed for my board

2020-05-30 Thread Kees Bakker
The reason I ask all these questions is that I'm adding support for another SODAQ board. And I was trying to do it the right way and run some tests. Maybe that was a mistake :-) Anyway, I've created the first issue. On 30-05-2020 21:30, Marian Buschsieweke wrote: > Hi, > > every failing test

Re: [riot-devel] Which tests are expected to succeed for my board

2020-05-30 Thread Kees Bakker
OK, first one created. I'm afraid there are many more to come. On 30-05-2020 21:27, Alexandre Abadie wrote: > Hi Kees, > > To my knowledge, there is no wiki page for this kind of thing. > The simplest is probably to open an issue and list there the failures with > their output. This way we can

Re: [riot-devel] Which tests are expected to succeed for my board

2020-05-30 Thread Marian Buschsieweke
Hi, every failing test indicates a bug. (From time to time the bug is found in the test rather than in the code to test.) Feel free to open an issue for the failing tests. I personally favor a tracking issue that lists each failing test with a checkbox. It is motivating to tick them of one by

Re: [riot-devel] Which tests are expected to succeed for my board

2020-05-30 Thread Alexandre Abadie
Hi Kees, To my knowledge, there is no wiki page for this kind of thing. The simplest is probably to open an issue and list there the failures with their output. This way we can easily track the on going work to fix them. See [1] as an example. Alex [1]

[riot-devel] Which tests are expected to succeed for my board

2020-05-29 Thread Kees Bakker
Hi, Now that I'm happily running automated tests for my SAMD21 board(s) I am wondering which tests should succeed. There are several that fail, but I don't know if that "normal" or not. Some examples of test that fail xtimer_periodic_wakeup: hangs at the end, last couple of printf don't come