Robbie Gemmell created QPIDJMS-75:
-------------------------------------
Summary: make the test peer report more useful failure causes and
update related tests to be more forgiving of slowdowns on shared CI systems
Key: QPIDJMS-75
URL: https://issues.apache.org/jira/browse/QPIDJMS-75
Project: Qpid JMS
Issue Type: Task
Affects Versions: 0.3.0, 0.2.0, 0.1.0
Reporter: Robbie Gemmell
Assignee: Robbie Gemmell
Fix For: 0.4.0
In adddition to running unit tests and some brokered interop/integration tests,
we also created a 'test peer' to allow performing some brokerless integration
testing of the client against a peer we could use to match/validate the AMQP
frames emitted by the client and construct AMQP frames to send to the client.
Thus far, whenever a particular matcher fails (e.g checking a particular frame
field has an expected value, and it did not), the resulting assertion error in
the test peer thread was recorded and then peer simply stops processing,
wihtout undertaking any action that would have occurred had the value been as
expected. As a result, if the client is awaiting a response from the peer
(which in most cases it is) then it is never sent, and the test idles until it
is timed out by JUnit (or ceased by some other action), which has also resulted
in use of artificially low timesouts to bound run times against 'expected'
failure cases, additionally making the tests unforgiving of sporadic slowdown
on shared CI machines. Although the assertion failure is recorded, the test
peer typically is not shut down in those cases as the test timeout itself
becomes the reported cause of failure, leading to inspection of the test log
output being necessary to identify anything about what actually happened.
To improve things in terms of the reported test failure/error cause and also
the time taken for the test to fail/error in many cases, assertion failures
continue to be recorded but the subsequent actions actually still performed.
Where the test is able to continue to completion the first assertion can then
be thrown when closing down the test peer, meaning the test is much more likely
to fail fast and the assertion failure actually becoming the reported
cause/error by JUnit on the console, thus improving reporting and simplifying
analysis. By avoiding need to use of the test timeout to bound run time in the
case of these 'expected failures', the applied test timeouts can also be
increased which enables the tests to be more forgiving of sporadic slowdowns
while being run on shared CI machines.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
