I think this is a general request applying for any test you write on
any project:
- When you write a new test, please make sure you run the test in a loop
(Idea has a nice support for that, Edit Test Profile, and make it
run it 300 times, until fails.. .etc).
I have been dealing with intermittent failures and most of the time
the issue was there from day one on the test.
- Do not use Sleep. Unless you really mean to test things like
expire... and even then, try to make it fast... every second counts
when you have 1000s tests...
- Use Wait clauses instead of sleep.
- Do not use Assert.assertNull(consumer.receive(long-timeout)); if you
are doing Assert.assertNull, use consumer.receiveNoWait());
- Limit test output.
You know that System.out hack you added on the test...
Things like:
System.out.println("Received message')
System.out.println("passed through here"); // this one is mine
Please dont do this. Try to limit log output please
In artemis all tests should be using log.debug() as the CI profile
will disable log.debug(); and only keep log.info().
I have done some major work on the testsuite, even though the
testsuite is taking 2 hours and 15 (with a variance of 5 minutes +/-),
it's always passing. (with the caveat of a few retries along the run).
So, I want to keep it that, way, so please be diligent about how you
write your tests. Avoid flaky tests by running them in a loop before
you send the PR.
And this is a general request, that I'm pretty sure it's valid for all
the projects.
Thank you!
--
Clebert Suconic
