I don't always read the entire log -- if a build is SUCCESSFUL, I assume it's successful.
I hit this while using luceneutil/repeatLuceneTest.py -- it happily ran a couple of hundred iterations until I suspected something's wrong because the iterations finished too quickly. Indeed, if I run 'ant test -Dtestcase=TestFoo* -Dtests.iters=1000', nothing is run and then it's easy to note that in the console because I'm missing the 1000 OK lines. Maybe it should be fixed in luceneutil, I'm not sure. To fix it there would mean that we'll need to parse the test log to search for "0 suites" or "0 tests", which is both fragile and only possible if you choose to log the output to a file (if it's logged to the console there's nothing you can do I believe). So I thought if the runner would fail the build, it will help more easily detect such errors. Why rely on someone to note within the hundreds of characters the framework spits at you that no tests were actually ran? What's the harm of failing the build in that case? Shai On Mon, Oct 14, 2013 at 4:00 PM, Dawid Weiss <[email protected]> wrote: > > I don't think this should be an error. If you make a typo you will see no > tests were executed. I don't see thr benefit of failing the build. > > In any case, this would belong to the runner. > > D. > On Oct 14, 2013 2:17 PM, "Shai Erera" <[email protected]> wrote: > >> Hi >> >> If you run something like "ant test -Dtestcase=TestFoo" where there's no >> TestFoo.java, or "ant test -Dtestcase=TestBar -Dtests.method=testFoo" where >> there's TestBar.java but no testFoo() method, the build currently passes as >> SUCCESSFUL. Though, the report says "0 suits" or "1 suits, 0 tests". >> >> I wonder if it should be a build failure? The problem I have is that you >> can e.g. make a very stupid typo (usually around plurals as in >> testSomethingBadOnIOException vs testSomethingBadOnIOExceptions) and get a >> false SUCCESSFUL notification. >> >> If we want to fix/change it, where does it belong - build scripts, >> LuceneTestCase or randomizedrunner? >> >> Shai >> >
