On Fri, Oct 2, 2020 at 9:37 PM Rob Landley <r...@landley.net> wrote: > On 10/2/20 4:13 PM, enh via Toybox wrote: > > bonus question: why didn't the toybox CI catch this? looks like the tar > tests > > aren't being run there for some reason, despite the `make > > defconfig`? https://github.com/landley/toybox/runs/1196859091 > > > > ah, it's the "missing execute permission on the .test file" issue again. > isn't > > that a bug for everything not in pending? > > Yes and no. I was using +x to indicate the test itself is essentially in > pending, and a lot of TESTS need work even when the command itself has been > promoted. For example, tar.tests has a big "if false" block at the end > with a > bunch of TODO comments on tests I should add, but haven't yet. > > That said, at this point I need to do a full pass over all the tests > again, it > looks like I've commited other people's patches that changed permissions > for > tests that aren't remotely complete. (Maybe someday somebody will want to > sponsor me putting time into the test suite, until then it's a todo item > for > somewhere post-1.0. Right now I'm trying to get a release out before I fly > to > Japan on the 14th, at which point I probably ignore this project entirely > for a > few months because I expect to be busy with paying work.) > > Anyway, the tests that aren't +x even though their commands aren't in > pending are: > > for i in $(ls -l tests/*.test | grep '[-] ' | sed 's@ > .*tests/@@;s/[.]test//'); > do echo toys/*/$i.c; done | grep -v pending > > And I guess I can just chmod +x all of the ones that pass their tests and > go > "test_kill has two tests, and that's it for the foreseeable future I > guess, oh > well, I no longer have an obvious way to track what needs work"... >
you can always add a TODO in the .test files for the ones you're not happy with? the current setup seems like the cure is worse than the disease: "these tests aren't perfect ... so let's not run any of them" :-) this gets caught eventually because i run all the tests on Android (and have a "if actual tool not a symlink to toybox, report but ignore test failures" hack). but it does mean that it's easier to miss things when developing on the host (which is how i usually work for anything that's not Android-specific), and it means that the github continuous build isn't running these tests and letting us know after the fact that we've missed something. i'm happy to send you a "+x and add TODO" patch, but i'm assuming you have something more specific to add about _what_ you think is missing from the tests currently? (otherwise it would just be the default assumption of "of course we could do more testing", even for something like chromium: https://analysis.chromium.org/p/chromium/coverage !) > Rob >
_______________________________________________ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net