[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16443687#comment-16443687 ] ASF GitHub Bot commented on FLINK-8955: --- Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/5780 > Port ClassLoaderITCase to flip6 > --- > > Key: FLINK-8955 > URL: https://issues.apache.org/jira/browse/FLINK-8955 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.5.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16442596#comment-16442596 ] ASF GitHub Bot commented on FLINK-8955: --- Github user zentol commented on the issue: https://github.com/apache/flink/pull/5780 merging > Port ClassLoaderITCase to flip6 > --- > > Key: FLINK-8955 > URL: https://issues.apache.org/jira/browse/FLINK-8955 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.5.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16441190#comment-16441190 ] ASF GitHub Bot commented on FLINK-8955: --- Github user zentol commented on the issue: https://github.com/apache/flink/pull/5780 checkstyle 😡 > Port ClassLoaderITCase to flip6 > --- > > Key: FLINK-8955 > URL: https://issues.apache.org/jira/browse/FLINK-8955 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.5.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16441019#comment-16441019 ] ASF GitHub Bot commented on FLINK-8955: --- Github user aljoscha commented on the issue: https://github.com/apache/flink/pull/5780 still failures because of this? > Port ClassLoaderITCase to flip6 > --- > > Key: FLINK-8955 > URL: https://issues.apache.org/jira/browse/FLINK-8955 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.5.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16440768#comment-16440768 ] ASF GitHub Bot commented on FLINK-8955: --- Github user zentol commented on the issue: https://github.com/apache/flink/pull/5780 I may have found the case, we have to set `MANAGED_MEMORY_SIZE` for this test. > Port ClassLoaderITCase to flip6 > --- > > Key: FLINK-8955 > URL: https://issues.apache.org/jira/browse/FLINK-8955 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.5.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16440691#comment-16440691 ] ASF GitHub Bot commented on FLINK-8955: --- Github user aljoscha commented on the issue: https://github.com/apache/flink/pull/5780 Failing reproducibly, and only on this branch? > Port ClassLoaderITCase to flip6 > --- > > Key: FLINK-8955 > URL: https://issues.apache.org/jira/browse/FLINK-8955 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.5.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16439920#comment-16439920 ] ASF GitHub Bot commented on FLINK-8955: --- Github user zentol commented on the issue: https://github.com/apache/flink/pull/5780 travis is still failing with mysterious error messages.. ``` 15:58:07.061 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test (integration-tests) on project flink-tests_2.11: ExecutionException: java.lang.RuntimeException: The forked VM terminated without properly saying goodbye. VM crash or System.exit called? 15:58:07.061 [ERROR] Command was /bin/sh -c cd /home/travis/build/apache/flink/flink-tests/target && /usr/lib/jvm/java-8-oracle/jre/bin/java -Xms256m -Xmx2048m -Dmvn.forkNumber=1 -XX:+UseG1GC -jar /home/travis/build/apache/flink/flink-tests/target/surefire/surefirebooter5397929458643043612.jar /home/travis/build/apache/flink/flink-tests/target/surefire/surefire4818404066082555175tmp /home/travis/build/apache/flink/flink-tests/target/surefire/surefire_1027420331622776884606tmp ``` > Port ClassLoaderITCase to flip6 > --- > > Key: FLINK-8955 > URL: https://issues.apache.org/jira/browse/FLINK-8955 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.5.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16439536#comment-16439536 ] ASF GitHub Bot commented on FLINK-8955: --- Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/5780#discussion_r181759432 --- Diff: flink-tests/src/test/java/org/apache/flink/test/classloading/ClassLoaderITCase.java --- @@ -202,15 +204,27 @@ public void testCheckpointedStreamingClassloaderJobWithCustomClassLoader() throw Collections.singleton(new Path(STREAMING_CHECKPOINTED_PROG_JAR_FILE)), Collections.emptyList()); - // Program should terminate with a 'SuccessException': - // we can not access the SuccessException here when executing the tests with maven, because its not available in the jar. - expectedException.expectCause( - Matchers.hasProperty("cause", - hasProperty("class", - hasProperty("canonicalName", equalTo( - "org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException"); - - streamingCheckpointedProg.invokeInteractiveModeForExecution(); + try { --- End diff -- my guess is that the exception stack has changed. The SuccessException is no longer the direct cause of the top-level exception. ``` Expected: exception with cause hasProperty("cause", hasProperty("class", hasProperty("canonicalName", "org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException"))) but: cause property 'cause' property 'class' null Stacktrace was: org.apache.flink.client.program.ProgramInvocationException: The main method caused an error. at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:545) at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:420) at org.apache.flink.test.classloading.ClassLoaderITCase.testCheckpointedStreamingClassloaderJobWithCustomClassLoader(ClassLoaderITCase.java:212) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:239) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at org.junit.rules.RunRules.evaluate(RunRules.java:20) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.junit.
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16439488#comment-16439488 ] ASF GitHub Bot commented on FLINK-8955: --- Github user aljoscha commented on a diff in the pull request: https://github.com/apache/flink/pull/5780#discussion_r181746661 --- Diff: flink-tests/src/test/java/org/apache/flink/test/classloading/ClassLoaderITCase.java --- @@ -202,15 +204,27 @@ public void testCheckpointedStreamingClassloaderJobWithCustomClassLoader() throw Collections.singleton(new Path(STREAMING_CHECKPOINTED_PROG_JAR_FILE)), Collections.emptyList()); - // Program should terminate with a 'SuccessException': - // we can not access the SuccessException here when executing the tests with maven, because its not available in the jar. - expectedException.expectCause( - Matchers.hasProperty("cause", - hasProperty("class", - hasProperty("canonicalName", equalTo( - "org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException"); - - streamingCheckpointedProg.invokeInteractiveModeForExecution(); + try { --- End diff -- Why doesn't the old code work here anymore? > Port ClassLoaderITCase to flip6 > --- > > Key: FLINK-8955 > URL: https://issues.apache.org/jira/browse/FLINK-8955 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.5.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16438702#comment-16438702 ] ASF GitHub Bot commented on FLINK-8955: --- Github user zentol commented on the issue: https://github.com/apache/flink/pull/5780 I've cut down the PR to only not make any changes outside the tests, fixed the checkstyle violations and the failing test. > Port ClassLoaderITCase to flip6 > --- > > Key: FLINK-8955 > URL: https://issues.apache.org/jira/browse/FLINK-8955 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.5.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16438700#comment-16438700 ] ASF GitHub Bot commented on FLINK-8955: --- Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/5780#discussion_r181582943 --- Diff: flink-tests/src/test/java/org/apache/flink/test/classloading/ClassLoaderITCase.java --- @@ -196,21 +188,23 @@ public void testCheckpointedStreamingClassloaderJobWithCustomClassLoader() throw // the test also ensures that user specific exceptions are serializable between JobManager <--> JobClient. PackagedProgram streamingCheckpointedProg = new PackagedProgram(new File(STREAMING_CHECKPOINTED_PROG_JAR_FILE)); - TestStreamEnvironment.setAsContext( - testCluster, + TestEnvironment.setAsContext( + MINI_CLUSTER_RESOURCE, parallelism, Collections.singleton(new Path(STREAMING_CHECKPOINTED_PROG_JAR_FILE)), Collections.emptyList()); - // Program should terminate with a 'SuccessException': - // we can not access the SuccessException here when executing the tests with maven, because its not available in the jar. - expectedException.expectCause( - Matchers.hasProperty("cause", - hasProperty("class", - hasProperty("canonicalName", equalTo( - "org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException"); - - streamingCheckpointedProg.invokeInteractiveModeForExecution(); + try { + streamingCheckpointedProg.invokeInteractiveModeForExecution(); + } catch (Exception e) { + // Program should terminate with a 'SuccessException': + // we can not access the SuccessException here when executing the tests with maven, because its not available in the jar. + Optional expectedCause = ExceptionUtils.findThrowable(e, + candidate -> candidate.getClass().getCanonicalName().equals("org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException")); --- End diff -- The test failed since the `TestEnvironment` was set as the context, and not the `TestStreamEnvironment` as intended. > Port ClassLoaderITCase to flip6 > --- > > Key: FLINK-8955 > URL: https://issues.apache.org/jira/browse/FLINK-8955 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.5.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16438492#comment-16438492 ] ASF GitHub Bot commented on FLINK-8955: --- Github user zentol commented on the issue: https://github.com/apache/flink/pull/5780 I would be fine with changing the PR to work against a `MiniCluster` (to drop the JobExecutor changes) and a `TemporaryFolder` that we manage ourselves, i,e. not as a `@Rule` (to drop the icky configuration changes). That's the quickest approach for now. > Port ClassLoaderITCase to flip6 > --- > > Key: FLINK-8955 > URL: https://issues.apache.org/jira/browse/FLINK-8955 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.5.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16438129#comment-16438129 ] ASF GitHub Bot commented on FLINK-8955: --- Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/5780#discussion_r181534220 --- Diff: flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/test/util/MiniClusterResource.java --- @@ -49,15 +54,17 @@ * Starts a Flink mini cluster as a resource and registers the respective * ExecutionEnvironment and StreamExecutionEnvironment. */ -public class MiniClusterResource extends ExternalResource { +public class MiniClusterResource extends ExternalResource implements JobExecutor { --- End diff -- otherwise we have to revert to using a `MiniCluster` directly, but then we lose the ´@Rule` benefits. > Port ClassLoaderITCase to flip6 > --- > > Key: FLINK-8955 > URL: https://issues.apache.org/jira/browse/FLINK-8955 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.5.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16438128#comment-16438128 ] ASF GitHub Bot commented on FLINK-8955: --- Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/5780#discussion_r181534097 --- Diff: flink-tests/src/test/java/org/apache/flink/test/classloading/ClassLoaderITCase.java --- @@ -196,21 +188,23 @@ public void testCheckpointedStreamingClassloaderJobWithCustomClassLoader() throw // the test also ensures that user specific exceptions are serializable between JobManager <--> JobClient. PackagedProgram streamingCheckpointedProg = new PackagedProgram(new File(STREAMING_CHECKPOINTED_PROG_JAR_FILE)); - TestStreamEnvironment.setAsContext( - testCluster, + TestEnvironment.setAsContext( + MINI_CLUSTER_RESOURCE, parallelism, Collections.singleton(new Path(STREAMING_CHECKPOINTED_PROG_JAR_FILE)), Collections.emptyList()); - // Program should terminate with a 'SuccessException': - // we can not access the SuccessException here when executing the tests with maven, because its not available in the jar. - expectedException.expectCause( - Matchers.hasProperty("cause", - hasProperty("class", - hasProperty("canonicalName", equalTo( - "org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException"); - - streamingCheckpointedProg.invokeInteractiveModeForExecution(); + try { + streamingCheckpointedProg.invokeInteractiveModeForExecution(); + } catch (Exception e) { + // Program should terminate with a 'SuccessException': + // we can not access the SuccessException here when executing the tests with maven, because its not available in the jar. + Optional expectedCause = ExceptionUtils.findThrowable(e, + candidate -> candidate.getClass().getCanonicalName().equals("org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException")); --- End diff -- Note that this test is a bit funky at the moment; it should not be possible to access the exception at all since it shouldn't be deserializable. I had one version where the predicate version of `findThrowble` actually failed since it couldn't deserialize it, but this issue for some reason disappeared after a rebase. > Port ClassLoaderITCase to flip6 > --- > > Key: FLINK-8955 > URL: https://issues.apache.org/jira/browse/FLINK-8955 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.5.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16438125#comment-16438125 ] ASF GitHub Bot commented on FLINK-8955: --- Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/5780#discussion_r181533944 --- Diff: flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/test/util/MiniClusterResource.java --- @@ -117,6 +138,7 @@ public TestEnvironment getTestEnvironment() { @Override public void before() throws Exception { + miniClusterResourceConfiguration = Preconditions.checkNotNull(miniClusterResourceConfigurationSupplier.get()); --- End diff -- That's not "easy", it doesn't even solve the issue. The contained `TemporaryFolder` would only be available after the construction of the `MiniClusterResource`, the folder is however necessary to create the configuration of the `MiniClusterResource`. Naturally we would like both of these to be `Rules`, but that's not possible unless the configuration is generated lazily, as i propose in the PR. Due to the `RuleChain` at the time that `MiniClusterResource#before` is called we can already access the `TemporaryFolder`. The alternative is for the `TemporaryFolder` to be setup in a `@BeforeClass` method which is then used in the for the configuration. That would totally works, but it is really unfortunate as the setting of temporary paths is rather common and required for virtually all tests that use file-based state backends. I agree that my proposal is quite hacky, but the question we have to ask is whether we prefer more complexity in the test vs the `MiniClusterResource`, where i would prefer the latter. > Port ClassLoaderITCase to flip6 > --- > > Key: FLINK-8955 > URL: https://issues.apache.org/jira/browse/FLINK-8955 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.5.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16438119#comment-16438119 ] ASF GitHub Bot commented on FLINK-8955: --- Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/5780#discussion_r181533175 --- Diff: flink-tests/src/test/java/org/apache/flink/test/classloading/ClassLoaderITCase.java --- @@ -196,21 +188,23 @@ public void testCheckpointedStreamingClassloaderJobWithCustomClassLoader() throw // the test also ensures that user specific exceptions are serializable between JobManager <--> JobClient. PackagedProgram streamingCheckpointedProg = new PackagedProgram(new File(STREAMING_CHECKPOINTED_PROG_JAR_FILE)); - TestStreamEnvironment.setAsContext( - testCluster, + TestEnvironment.setAsContext( + MINI_CLUSTER_RESOURCE, parallelism, Collections.singleton(new Path(STREAMING_CHECKPOINTED_PROG_JAR_FILE)), Collections.emptyList()); - // Program should terminate with a 'SuccessException': - // we can not access the SuccessException here when executing the tests with maven, because its not available in the jar. - expectedException.expectCause( - Matchers.hasProperty("cause", - hasProperty("class", - hasProperty("canonicalName", equalTo( - "org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException"); - - streamingCheckpointedProg.invokeInteractiveModeForExecution(); + try { + streamingCheckpointedProg.invokeInteractiveModeForExecution(); + } catch (Exception e) { + // Program should terminate with a 'SuccessException': + // we can not access the SuccessException here when executing the tests with maven, because its not available in the jar. + Optional expectedCause = ExceptionUtils.findThrowable(e, + candidate -> candidate.getClass().getCanonicalName().equals("org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException")); --- End diff -- Technically that shouldn't be possible. The exception class is part of the user-jar which is not on the classpath of the test. > Port ClassLoaderITCase to flip6 > --- > > Key: FLINK-8955 > URL: https://issues.apache.org/jira/browse/FLINK-8955 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.5.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16438117#comment-16438117 ] ASF GitHub Bot commented on FLINK-8955: --- Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/5780#discussion_r181532978 --- Diff: flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/test/util/MiniClusterResource.java --- @@ -49,15 +54,17 @@ * Starts a Flink mini cluster as a resource and registers the respective * ExecutionEnvironment and StreamExecutionEnvironment. */ -public class MiniClusterResource extends ExternalResource { +public class MiniClusterResource extends ExternalResource implements JobExecutor { --- End diff -- so that we can pass the MiniClusterResource to `TestEnvironment.setAsContext()` > Port ClassLoaderITCase to flip6 > --- > > Key: FLINK-8955 > URL: https://issues.apache.org/jira/browse/FLINK-8955 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.5.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16437682#comment-16437682 ] ASF GitHub Bot commented on FLINK-8955: --- Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5780#discussion_r181467478 --- Diff: flink-tests/src/test/java/org/apache/flink/test/classloading/ClassLoaderITCase.java --- @@ -196,21 +188,23 @@ public void testCheckpointedStreamingClassloaderJobWithCustomClassLoader() throw // the test also ensures that user specific exceptions are serializable between JobManager <--> JobClient. PackagedProgram streamingCheckpointedProg = new PackagedProgram(new File(STREAMING_CHECKPOINTED_PROG_JAR_FILE)); - TestStreamEnvironment.setAsContext( - testCluster, + TestEnvironment.setAsContext( + MINI_CLUSTER_RESOURCE, parallelism, Collections.singleton(new Path(STREAMING_CHECKPOINTED_PROG_JAR_FILE)), Collections.emptyList()); - // Program should terminate with a 'SuccessException': - // we can not access the SuccessException here when executing the tests with maven, because its not available in the jar. - expectedException.expectCause( - Matchers.hasProperty("cause", - hasProperty("class", - hasProperty("canonicalName", equalTo( - "org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException"); - - streamingCheckpointedProg.invokeInteractiveModeForExecution(); + try { + streamingCheckpointedProg.invokeInteractiveModeForExecution(); + } catch (Exception e) { + // Program should terminate with a 'SuccessException': + // we can not access the SuccessException here when executing the tests with maven, because its not available in the jar. + Optional expectedCause = ExceptionUtils.findThrowable(e, + candidate -> candidate.getClass().getCanonicalName().equals("org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException")); --- End diff -- Can't we use `findThrowable(Throwable throwable, Class searchType)`? > Port ClassLoaderITCase to flip6 > --- > > Key: FLINK-8955 > URL: https://issues.apache.org/jira/browse/FLINK-8955 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.5.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16437679#comment-16437679 ] ASF GitHub Bot commented on FLINK-8955: --- Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5780#discussion_r181466728 --- Diff: flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/test/util/MiniClusterResource.java --- @@ -117,6 +138,7 @@ public TestEnvironment getTestEnvironment() { @Override public void before() throws Exception { + miniClusterResourceConfiguration = Preconditions.checkNotNull(miniClusterResourceConfigurationSupplier.get()); --- End diff -- I think we are mixing concerns by letting the `MiniClusterResource` change its behaviour across multiple tests. I think we should not pass in a configuration supplier which can lazily instantiate a configuration. Instead if you need a different `MiniClusterResource` per test, then the test should instantiate the respective `MiniClusterResource` with the proper configuration. > Port ClassLoaderITCase to flip6 > --- > > Key: FLINK-8955 > URL: https://issues.apache.org/jira/browse/FLINK-8955 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.5.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16437680#comment-16437680 ] ASF GitHub Bot commented on FLINK-8955: --- Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5780#discussion_r181465966 --- Diff: flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/test/util/MiniClusterResource.java --- @@ -225,6 +247,11 @@ private void startMiniCluster() throws Exception { } } + @Override + public JobExecutionResult executeJobBlocking(JobGraph jobGraph) throws JobExecutionException, InterruptedException { --- End diff -- I think we should use the `MiniClusterClient` for job submission. The `JobExecutor` interface is only an internal interface to bridge between the two different mini clusters. > Port ClassLoaderITCase to flip6 > --- > > Key: FLINK-8955 > URL: https://issues.apache.org/jira/browse/FLINK-8955 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.5.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16437681#comment-16437681 ] ASF GitHub Bot commented on FLINK-8955: --- Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5780#discussion_r181467950 --- Diff: flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/test/util/MiniClusterResource.java --- @@ -117,6 +138,7 @@ public TestEnvironment getTestEnvironment() { @Override public void before() throws Exception { + miniClusterResourceConfiguration = Preconditions.checkNotNull(miniClusterResourceConfigurationSupplier.get()); --- End diff -- The same applies to rule chains. An easy solution would be to create a `MiniClusterWithTemporaryFolderResource` which encapsulates this logic. > Port ClassLoaderITCase to flip6 > --- > > Key: FLINK-8955 > URL: https://issues.apache.org/jira/browse/FLINK-8955 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.5.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16437683#comment-16437683 ] ASF GitHub Bot commented on FLINK-8955: --- Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5780#discussion_r181468061 --- Diff: flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/test/util/MiniClusterResource.java --- @@ -49,15 +54,17 @@ * Starts a Flink mini cluster as a resource and registers the respective * ExecutionEnvironment and StreamExecutionEnvironment. */ -public class MiniClusterResource extends ExternalResource { +public class MiniClusterResource extends ExternalResource implements JobExecutor { --- End diff -- Why do we need to implement `JobExecutor`? > Port ClassLoaderITCase to flip6 > --- > > Key: FLINK-8955 > URL: https://issues.apache.org/jira/browse/FLINK-8955 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.5.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16437667#comment-16437667 ] ASF GitHub Bot commented on FLINK-8955: --- Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/5780 The `ClassLoaderITCase` fails on Travis. There are also checkstyle violations: ``` 10:04:01.988 [ERROR] src/test/java/org/apache/flink/test/classloading/ClassLoaderITCase.java:[62,15] (imports) UnusedImports: Unused import: org.junit.Assert.assertEquals. 10:04:01.988 [ERROR] src/test/java/org/apache/flink/test/classloading/ClassLoaderITCase.java:[65,15] (imports) UnusedImports: Unused import: org.junit.Assert.fail. ``` > Port ClassLoaderITCase to flip6 > --- > > Key: FLINK-8955 > URL: https://issues.apache.org/jira/browse/FLINK-8955 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.5.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)