[jira] [Commented] (FLINK-19983) ShuffleCompressionITCase.testDataCompressionForSortMergeBlockingShuffle unstable
[ https://issues.apache.org/jira/browse/FLINK-19983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17232792#comment-17232792 ] Arvid Heise commented on FLINK-19983: - Merged into master as e4f525e7c5270e33972cbbc9f6360435d0ff87ae. > ShuffleCompressionITCase.testDataCompressionForSortMergeBlockingShuffle > unstable > > > Key: FLINK-19983 > URL: https://issues.apache.org/jira/browse/FLINK-19983 > Project: Flink > Issue Type: Bug > Components: Runtime / Network >Affects Versions: 1.12.0 >Reporter: Robert Metzger >Assignee: Yingjie Cao >Priority: Critical > Labels: pull-request-available, test-stability > Fix For: 1.12.0 > > > https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=8997=logs=5c8e7682-d68f-54d1-16a2-a09310218a49=f508e270-48d6-5f1e-3138-42a17e0714f0 > {code} > 2020-11-04T14:32:19.7227316Z [ERROR] Tests run: 4, Failures: 1, Errors: 0, > Skipped: 0, Time elapsed: 16.882 s <<< FAILURE! - in > org.apache.flink.test.runtime.ShuffleCompressionITCase > 2020-11-04T14:32:19.7228708Z [ERROR] > testDataCompressionForSortMergeBlockingShuffle[useBroadcastPartitioner = > true](org.apache.flink.test.runtime.ShuffleCompressionITCase) Time elapsed: > 5.058 s <<< FAILURE! > 2020-11-04T14:32:19.7230032Z java.lang.AssertionError: > org.apache.flink.runtime.JobException: Recovery is suppressed by > NoRestartBackoffTimeStrategy > 2020-11-04T14:32:19.7230580Z at > org.apache.flink.test.runtime.JobGraphRunningUtil.execute(JobGraphRunningUtil.java:58) > 2020-11-04T14:32:19.7231173Z at > org.apache.flink.test.runtime.ShuffleCompressionITCase.testDataCompressionForSortMergeBlockingShuffle(ShuffleCompressionITCase.java:98) > 2020-11-04T14:32:19.7232076Z at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > 2020-11-04T14:32:19.7232624Z at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > 2020-11-04T14:32:19.7233242Z at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > 2020-11-04T14:32:19.7233741Z at > java.lang.reflect.Method.invoke(Method.java:498) > 2020-11-04T14:32:19.7234353Z at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) > 2020-11-04T14:32:19.7235141Z at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > 2020-11-04T14:32:19.7238521Z at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) > 2020-11-04T14:32:19.7239371Z at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > 2020-11-04T14:32:19.7240010Z at > org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) > 2020-11-04T14:32:19.7240688Z at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) > 2020-11-04T14:32:19.7241396Z at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) > 2020-11-04T14:32:19.7242019Z at > org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) > 2020-11-04T14:32:19.7242623Z at > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) > 2020-11-04T14:32:19.7243379Z at > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) > 2020-11-04T14:32:19.7244051Z at > org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) > 2020-11-04T14:32:19.7244631Z at > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) > 2020-11-04T14:32:19.7245313Z at > org.junit.runners.ParentRunner.run(ParentRunner.java:363) > 2020-11-04T14:32:19.7245844Z at > org.junit.runners.Suite.runChild(Suite.java:128) > 2020-11-04T14:32:19.7246341Z at > org.junit.runners.Suite.runChild(Suite.java:27) > 2020-11-04T14:32:19.7246868Z at > org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) > 2020-11-04T14:32:19.7247616Z at > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) > 2020-11-04T14:32:19.7248223Z at > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) > 2020-11-04T14:32:19.7248826Z at > org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) > 2020-11-04T14:32:19.7249393Z at > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) > 2020-11-04T14:32:19.7249963Z at > org.junit.runners.ParentRunner.run(ParentRunner.java:363) > 2020-11-04T14:32:19.7250586Z at > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) > 2020-11-04T14:32:19.7251277Z at > org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273) > 2020-11-04T14:32:19.7252024Z at > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238) > 2020-11-04T14:32:19.7252839Z at >
[jira] [Commented] (FLINK-19983) ShuffleCompressionITCase.testDataCompressionForSortMergeBlockingShuffle unstable
[ https://issues.apache.org/jira/browse/FLINK-19983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228333#comment-17228333 ] Yingjie Cao commented on FLINK-19983: - After some investigation, I find that the state check is not true, we just need to remove it. The caller, including CreditBasedSequenceNumberingViewReader and LocalInputChannel can handle the case correctly. BoundedBlockingSubpartitionReader does the same thing. The PR is available for review now. > ShuffleCompressionITCase.testDataCompressionForSortMergeBlockingShuffle > unstable > > > Key: FLINK-19983 > URL: https://issues.apache.org/jira/browse/FLINK-19983 > Project: Flink > Issue Type: Bug > Components: Runtime / Network >Affects Versions: 1.12.0 >Reporter: Robert Metzger >Assignee: Yingjie Cao >Priority: Critical > Labels: pull-request-available, test-stability > > https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=8997=logs=5c8e7682-d68f-54d1-16a2-a09310218a49=f508e270-48d6-5f1e-3138-42a17e0714f0 > {code} > 2020-11-04T14:32:19.7227316Z [ERROR] Tests run: 4, Failures: 1, Errors: 0, > Skipped: 0, Time elapsed: 16.882 s <<< FAILURE! - in > org.apache.flink.test.runtime.ShuffleCompressionITCase > 2020-11-04T14:32:19.7228708Z [ERROR] > testDataCompressionForSortMergeBlockingShuffle[useBroadcastPartitioner = > true](org.apache.flink.test.runtime.ShuffleCompressionITCase) Time elapsed: > 5.058 s <<< FAILURE! > 2020-11-04T14:32:19.7230032Z java.lang.AssertionError: > org.apache.flink.runtime.JobException: Recovery is suppressed by > NoRestartBackoffTimeStrategy > 2020-11-04T14:32:19.7230580Z at > org.apache.flink.test.runtime.JobGraphRunningUtil.execute(JobGraphRunningUtil.java:58) > 2020-11-04T14:32:19.7231173Z at > org.apache.flink.test.runtime.ShuffleCompressionITCase.testDataCompressionForSortMergeBlockingShuffle(ShuffleCompressionITCase.java:98) > 2020-11-04T14:32:19.7232076Z at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > 2020-11-04T14:32:19.7232624Z at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > 2020-11-04T14:32:19.7233242Z at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > 2020-11-04T14:32:19.7233741Z at > java.lang.reflect.Method.invoke(Method.java:498) > 2020-11-04T14:32:19.7234353Z at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) > 2020-11-04T14:32:19.7235141Z at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > 2020-11-04T14:32:19.7238521Z at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) > 2020-11-04T14:32:19.7239371Z at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > 2020-11-04T14:32:19.7240010Z at > org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) > 2020-11-04T14:32:19.7240688Z at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) > 2020-11-04T14:32:19.7241396Z at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) > 2020-11-04T14:32:19.7242019Z at > org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) > 2020-11-04T14:32:19.7242623Z at > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) > 2020-11-04T14:32:19.7243379Z at > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) > 2020-11-04T14:32:19.7244051Z at > org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) > 2020-11-04T14:32:19.7244631Z at > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) > 2020-11-04T14:32:19.7245313Z at > org.junit.runners.ParentRunner.run(ParentRunner.java:363) > 2020-11-04T14:32:19.7245844Z at > org.junit.runners.Suite.runChild(Suite.java:128) > 2020-11-04T14:32:19.7246341Z at > org.junit.runners.Suite.runChild(Suite.java:27) > 2020-11-04T14:32:19.7246868Z at > org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) > 2020-11-04T14:32:19.7247616Z at > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) > 2020-11-04T14:32:19.7248223Z at > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) > 2020-11-04T14:32:19.7248826Z at > org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) > 2020-11-04T14:32:19.7249393Z at > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) > 2020-11-04T14:32:19.7249963Z at > org.junit.runners.ParentRunner.run(ParentRunner.java:363) > 2020-11-04T14:32:19.7250586Z at > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) > 2020-11-04T14:32:19.7251277Z at >
[jira] [Commented] (FLINK-19983) ShuffleCompressionITCase.testDataCompressionForSortMergeBlockingShuffle unstable
[ https://issues.apache.org/jira/browse/FLINK-19983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17227363#comment-17227363 ] Robert Metzger commented on FLINK-19983: Thanks a lot. I assigned you. > ShuffleCompressionITCase.testDataCompressionForSortMergeBlockingShuffle > unstable > > > Key: FLINK-19983 > URL: https://issues.apache.org/jira/browse/FLINK-19983 > Project: Flink > Issue Type: Bug > Components: Runtime / Network >Affects Versions: 1.12.0 >Reporter: Robert Metzger >Assignee: Yingjie Cao >Priority: Critical > Labels: test-stability > > https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=8997=logs=5c8e7682-d68f-54d1-16a2-a09310218a49=f508e270-48d6-5f1e-3138-42a17e0714f0 > {code} > 2020-11-04T14:32:19.7227316Z [ERROR] Tests run: 4, Failures: 1, Errors: 0, > Skipped: 0, Time elapsed: 16.882 s <<< FAILURE! - in > org.apache.flink.test.runtime.ShuffleCompressionITCase > 2020-11-04T14:32:19.7228708Z [ERROR] > testDataCompressionForSortMergeBlockingShuffle[useBroadcastPartitioner = > true](org.apache.flink.test.runtime.ShuffleCompressionITCase) Time elapsed: > 5.058 s <<< FAILURE! > 2020-11-04T14:32:19.7230032Z java.lang.AssertionError: > org.apache.flink.runtime.JobException: Recovery is suppressed by > NoRestartBackoffTimeStrategy > 2020-11-04T14:32:19.7230580Z at > org.apache.flink.test.runtime.JobGraphRunningUtil.execute(JobGraphRunningUtil.java:58) > 2020-11-04T14:32:19.7231173Z at > org.apache.flink.test.runtime.ShuffleCompressionITCase.testDataCompressionForSortMergeBlockingShuffle(ShuffleCompressionITCase.java:98) > 2020-11-04T14:32:19.7232076Z at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > 2020-11-04T14:32:19.7232624Z at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > 2020-11-04T14:32:19.7233242Z at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > 2020-11-04T14:32:19.7233741Z at > java.lang.reflect.Method.invoke(Method.java:498) > 2020-11-04T14:32:19.7234353Z at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) > 2020-11-04T14:32:19.7235141Z at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > 2020-11-04T14:32:19.7238521Z at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) > 2020-11-04T14:32:19.7239371Z at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > 2020-11-04T14:32:19.7240010Z at > org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) > 2020-11-04T14:32:19.7240688Z at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) > 2020-11-04T14:32:19.7241396Z at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) > 2020-11-04T14:32:19.7242019Z at > org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) > 2020-11-04T14:32:19.7242623Z at > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) > 2020-11-04T14:32:19.7243379Z at > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) > 2020-11-04T14:32:19.7244051Z at > org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) > 2020-11-04T14:32:19.7244631Z at > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) > 2020-11-04T14:32:19.7245313Z at > org.junit.runners.ParentRunner.run(ParentRunner.java:363) > 2020-11-04T14:32:19.7245844Z at > org.junit.runners.Suite.runChild(Suite.java:128) > 2020-11-04T14:32:19.7246341Z at > org.junit.runners.Suite.runChild(Suite.java:27) > 2020-11-04T14:32:19.7246868Z at > org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) > 2020-11-04T14:32:19.7247616Z at > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) > 2020-11-04T14:32:19.7248223Z at > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) > 2020-11-04T14:32:19.7248826Z at > org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) > 2020-11-04T14:32:19.7249393Z at > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) > 2020-11-04T14:32:19.7249963Z at > org.junit.runners.ParentRunner.run(ParentRunner.java:363) > 2020-11-04T14:32:19.7250586Z at > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) > 2020-11-04T14:32:19.7251277Z at > org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273) > 2020-11-04T14:32:19.7252024Z at > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238) > 2020-11-04T14:32:19.7252839Z at > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159) >
[jira] [Commented] (FLINK-19983) ShuffleCompressionITCase.testDataCompressionForSortMergeBlockingShuffle unstable
[ https://issues.apache.org/jira/browse/FLINK-19983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17227242#comment-17227242 ] Yingjie Cao commented on FLINK-19983: - I will take a look at this issue. > ShuffleCompressionITCase.testDataCompressionForSortMergeBlockingShuffle > unstable > > > Key: FLINK-19983 > URL: https://issues.apache.org/jira/browse/FLINK-19983 > Project: Flink > Issue Type: Bug > Components: Runtime / Network >Affects Versions: 1.12.0 >Reporter: Robert Metzger >Priority: Critical > Labels: test-stability > > https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=8997=logs=5c8e7682-d68f-54d1-16a2-a09310218a49=f508e270-48d6-5f1e-3138-42a17e0714f0 > {code} > 2020-11-04T14:32:19.7227316Z [ERROR] Tests run: 4, Failures: 1, Errors: 0, > Skipped: 0, Time elapsed: 16.882 s <<< FAILURE! - in > org.apache.flink.test.runtime.ShuffleCompressionITCase > 2020-11-04T14:32:19.7228708Z [ERROR] > testDataCompressionForSortMergeBlockingShuffle[useBroadcastPartitioner = > true](org.apache.flink.test.runtime.ShuffleCompressionITCase) Time elapsed: > 5.058 s <<< FAILURE! > 2020-11-04T14:32:19.7230032Z java.lang.AssertionError: > org.apache.flink.runtime.JobException: Recovery is suppressed by > NoRestartBackoffTimeStrategy > 2020-11-04T14:32:19.7230580Z at > org.apache.flink.test.runtime.JobGraphRunningUtil.execute(JobGraphRunningUtil.java:58) > 2020-11-04T14:32:19.7231173Z at > org.apache.flink.test.runtime.ShuffleCompressionITCase.testDataCompressionForSortMergeBlockingShuffle(ShuffleCompressionITCase.java:98) > 2020-11-04T14:32:19.7232076Z at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > 2020-11-04T14:32:19.7232624Z at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > 2020-11-04T14:32:19.7233242Z at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > 2020-11-04T14:32:19.7233741Z at > java.lang.reflect.Method.invoke(Method.java:498) > 2020-11-04T14:32:19.7234353Z at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) > 2020-11-04T14:32:19.7235141Z at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > 2020-11-04T14:32:19.7238521Z at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) > 2020-11-04T14:32:19.7239371Z at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > 2020-11-04T14:32:19.7240010Z at > org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) > 2020-11-04T14:32:19.7240688Z at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) > 2020-11-04T14:32:19.7241396Z at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) > 2020-11-04T14:32:19.7242019Z at > org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) > 2020-11-04T14:32:19.7242623Z at > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) > 2020-11-04T14:32:19.7243379Z at > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) > 2020-11-04T14:32:19.7244051Z at > org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) > 2020-11-04T14:32:19.7244631Z at > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) > 2020-11-04T14:32:19.7245313Z at > org.junit.runners.ParentRunner.run(ParentRunner.java:363) > 2020-11-04T14:32:19.7245844Z at > org.junit.runners.Suite.runChild(Suite.java:128) > 2020-11-04T14:32:19.7246341Z at > org.junit.runners.Suite.runChild(Suite.java:27) > 2020-11-04T14:32:19.7246868Z at > org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) > 2020-11-04T14:32:19.7247616Z at > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) > 2020-11-04T14:32:19.7248223Z at > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) > 2020-11-04T14:32:19.7248826Z at > org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) > 2020-11-04T14:32:19.7249393Z at > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) > 2020-11-04T14:32:19.7249963Z at > org.junit.runners.ParentRunner.run(ParentRunner.java:363) > 2020-11-04T14:32:19.7250586Z at > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) > 2020-11-04T14:32:19.7251277Z at > org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273) > 2020-11-04T14:32:19.7252024Z at > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238) > 2020-11-04T14:32:19.7252839Z at > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159) > 2020-11-04T14:32:19.7253584Z at >