[ https://issues.apache.org/jira/browse/SCB-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
liubao resolved SCB-2718. ------------------------- Resolution: Fixed > Fix flaky test failure in TestOperationGenerator#apiOperationThenResponse > -------------------------------------------------------------------------- > > Key: SCB-2718 > URL: https://issues.apache.org/jira/browse/SCB-2718 > Project: Apache ServiceComb > Issue Type: Bug > Components: Java-Chassis > Reporter: Mohsen Dehghankar > Priority: Major > Fix For: java-chassis-2.9.0 > > > h3. What is the purpose of this PR > - This PR fixes the flaky test > `org.apache.servicecomb.swagger.generator.core.TestOperationGenerator.apiOperationThenResponse` > - The mentioned test may fail or pass without changes made to the source code > when it is run in different JVMs due to the non-deterministic order of the > returned array from `getDeclaredAnnotations`. Because there is no assumption > on the order of the returned array from this method in [Java > documentation|https://docs.oracle.com/javase/8/docs/api/java/lang/reflect/AccessibleObject.html#getDeclaredAnnotations--]. > h3. Why the test fails > - When processing the annotations of a method in > `AbstractOperationGenerator#scanMethodAnnotations()`, the result of > `method.getAnnotations()` is used which uses > `AccessibleObject.getDeclaredAnnotations()` which does not guarantee a > deterministic result order based on Java specifications. > - This results in processing annotations in arbitrary order, so the expected > assertions in both `TestOperationGenerator#apiOperationThenResponse` and > `TestOperationGenerator#responseThenApiOperation` might fail due to this > change in orders of annotations. > h3. Reproduce the test failure > - Run the test with 'NonDex' maven plugin. The command to recreate the flaky > test failure is `mvn edu.illinois:nondex-maven-plugin:2.1-SNAPSHOT:nondex -pl > swagger/swagger-generator/generator-core > -Dtest=TestOperationGenerator#apiOperationThenResponse` > - Fixing the flaky test now may prevent flaky test failures in future Java > versions. > h3. Expected Result > - The tests should run successfully when run with 'NonDex' maven with the > same command for recreating the flaky test. > h3. Actual Result > - We get the following error: > ``` > [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.258 > s <<< FAILURE! - in > org.apache.servicecomb.swagger.generator.core.TestOperationGenerator > [ERROR] > org.apache.servicecomb.swagger.generator.core.TestOperationGenerator.apiOperationThenResponse > Time elapsed: 0.25 s <<< FAILURE! > org.opentest4j.AssertionFailedError: expected: <null> but was: > <io.swagger.models.properties.StringProperty@8f5d1e11> > at > org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151) > at > org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132) > at org.junit.jupiter.api.AssertNull.failNotNull(AssertNull.java:50) > at org.junit.jupiter.api.AssertNull.assertNull(AssertNull.java:35) > at org.junit.jupiter.api.AssertNull.assertNull(AssertNull.java:30) > at org.junit.jupiter.api.Assertions.assertNull(Assertions.java:276) > at > org.apache.servicecomb.swagger.generator.core.TestOperationGenerator.apiOperationThenResponse(TestOperationGenerator.java:114) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base/java.lang.reflect.Method.invoke(Method.java:566) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) > at > org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) > at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) > at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) > at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) > at org.junit.runners.ParentRunner.run(ParentRunner.java:413) > at org.junit.runner.JUnitCore.run(JUnitCore.java:137) > at org.junit.runner.JUnitCore.run(JUnitCore.java:115) > at > org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) > at > org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) > at > org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) > at > org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:147) > at > org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:127) > at > org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:90) > at > org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:55) > at > org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:102) > at > org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:54) > at > org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114) > at > org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86) > at > org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86) > at > org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:55) > at > org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:223) > at > org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:175) > at > org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:139) > at > org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:456) > at > org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:169) > at > org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:595) > at > org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:581) > ``` > h3. Fix > - First, we should sort the `method.getAnnotations()` result > deterministically. > - Secondly, there should be no different in the `assertEqual` expressions > between these two tests: `apiOperationThenResponse` and > `responseThenApiOperation` -- This message was sent by Atlassian Jira (v8.20.10#820010)