Improvement thanks to Simon, I’ve now moved the 2 new annotations to the UITest
annotation:
@Documented
@Retention(RUNTIME)
@Target({ TYPE, METHOD, ANNOTATION_TYPE })
@ExtendWith(XWikiDockerExtension.class)
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
public @interface UITest
Example usage:
@UITest
public class MenuIT
{
@Test
@Order(1)
…
Simpler, thanks Simon!
-Vincent
> On 12 Feb 2019, at 15:04, Vincent Massol <[email protected]> wrote:
>
> Actually to be even more rigorous, we should use:
>
> @UITest
> @TestMethodOrder(OrderAnnotation.class)
> @TestInstance(Lifecycle.PER_CLASS)
> public class MenuIT
>
> See
> https://junit.org/junit5/docs/current/user-guide/#writing-tests-test-instance-lifecycle
> for more details.
>
> Thanks
> -Vincent
>
>> On 12 Feb 2019, at 14:44, Vincent Massol <[email protected]> wrote:
>>
>> Hi,
>>
>> Now that JUnit 5.4.0 is out we have test ordering. I propose the following
>> best practices for TC-based docker tests:
>>
>> @UITest
>> @TestMethodOrder(OrderAnnotation.class)
>> public class MenuIT
>> {
>> @Test
>> @Order(1)
>> public void verifyMenuInApplicationsIndex(TestUtils setup)
>> {
>> ...
>> }
>>
>> @Test
>> @Order(2)
>> public void
>> verifyMenuCreationInLeftPanelWithCurrentWikiVisibility(TestUtils setup)
>> {
>> ...
>> }
>>
>> @Test
>> @Order(3)
>> public void verifyMenuIsAvailableInAdministration(TestUtils setup) throws
>> Exception
>> {
>> …
>> }
>>
>> Instead of the current:
>>
>> @Test
>> public void verifyMenu(TestUtils setup) throws Exception
>> {
>> verifyMenuInApplicationsIndex(setup);
>> verifyMenuCreationInLeftPanelWithCurrentWikiVisibility(setup);
>> verifyMenuIsAvailableInAdministration(setup);
>> }
>>
>>
>> Pros:
>> * Easier to run each test
>> * Easier to debug and view recorded video for a specific failing test
>> * More in sync with JUnit’s practices
>> * It’s still a scenario and thus doesn’t incur penalty of extra test setup
>>
>> Cons:
>> * Starts and stop the VNC docker container for each test. It takes between 1
>> and 2s to start the VNC container and the stop (i.e. video recording save)
>> should be the same as before.
>>
>> So I think it’s worth it. Out of 3mn, 3-6 more seconds for 3 tests is not
>> too much (between 2% and 3% more).
>>
>> WDYT?
>>
>> Thanks
>> -Vincent
>>
>>
>>
>>
>>
>>
>>
>>
>
