[jira] [Commented] (FLINK-16550) HadoopS3* tests fail with NullPointerException exceptions

2020-03-12 Thread Robert Metzger (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-16550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058038#comment-17058038
 ] 

Robert Metzger commented on FLINK-16550:


Done in 15a149ebc4042c5f623c88c1afddb826bc4d09e7

> HadoopS3* tests fail with NullPointerException exceptions
> -
>
> Key: FLINK-16550
> URL: https://issues.apache.org/jira/browse/FLINK-16550
> Project: Flink
>  Issue Type: Bug
>  Components: FileSystems
>Affects Versions: 1.11.0
>Reporter: Robert Metzger
>Assignee: Arvid Heise
>Priority: Blocker
>  Labels: pull-request-available
> Fix For: 1.11.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Logs: 
> https://travis-ci.org/github/apache/flink/jobs/660975486?utm_medium=notification
> All subsequent builds failed as well. It is likely that this commit / 
> FLINK-16014 introduced the issue, as these tests depend on S3 credentials to 
> be available.
> {code}
> 09:38:48.022 [INFO] ---
> 09:38:48.025 [INFO]  T E S T S
> 09:38:48.026 [INFO] ---
> 09:38:48.657 [INFO] Running 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase
> 09:38:48.669 [INFO] Running 
> org.apache.flink.fs.s3hadoop.HadoopS3FileSystemITCase
> 09:38:54.541 [ERROR] Tests run: 3, Failures: 0, Errors: 3, Skipped: 0, Time 
> elapsed: 5.88 s <<< FAILURE! - in 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase
> 09:38:54.542 [ERROR] 
> testResumeAfterCommit(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase)
>   Time elapsed: 3.592 s  <<< ERROR!
> java.lang.Exception: Unexpected exception, expected but 
> was
>   at 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase.testResumeAfterCommit(HadoopS3RecoverableWriterExceptionITCase.java:162)
> 09:38:54.542 [ERROR] 
> testResumeWithWrongOffset(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase)
>   Time elapsed: 0.24 s  <<< ERROR!
> java.lang.Exception: Unexpected exception, expected but 
> was
>   at 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase.testResumeWithWrongOffset(HadoopS3RecoverableWriterExceptionITCase.java:182)
> 09:38:54.542 [ERROR] 
> testExceptionWritingAfterCloseForCommit(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase)
>   Time elapsed: 0.448 s  <<< ERROR!
> java.lang.Exception: Unexpected exception, expected but 
> was
>   at 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase.testExceptionWritingAfterCloseForCommit(HadoopS3RecoverableWriterExceptionITCase.java:144)
> 09:38:55.173 [INFO] Running 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase
> 09:38:58.737 [ERROR] Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time 
> elapsed: 10.066 s <<< FAILURE! - in 
> org.apache.flink.fs.s3hadoop.HadoopS3FileSystemITCase
> 09:38:58.737 [ERROR] 
> testDirectoryListing(org.apache.flink.fs.s3hadoop.HadoopS3FileSystemITCase)  
> Time elapsed: 3.448 s  <<< ERROR!
> java.io.FileNotFoundException: No such file or directory: 
> s3://[secure]/temp/tests-f37db36e-c116-4c58-a16b-8ca241baae4b/testdir
> 09:38:59.447 [INFO] Running 
> org.apache.flink.fs.s3hadoop.HadoopS3FileSystemBehaviorITCase
> 09:39:01.791 [ERROR] Tests run: 13, Failures: 0, Errors: 13, Skipped: 0, Time 
> elapsed: 6.611 s <<< FAILURE! - in 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase
> 09:39:01.797 [ERROR] 
> testCloseWithNoData(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase)
>   Time elapsed: 2.394 s  <<< ERROR!
> java.lang.NullPointerException
>   at 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testCloseWithNoData(HadoopS3RecoverableWriterITCase.java:186)
> 09:39:01.798 [ERROR] 
> testCommitAfterPersist(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase)
>   Time elapsed: 0.191 s  <<< ERROR!
> java.lang.NullPointerException
>   at 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testCommitAfterPersist(HadoopS3RecoverableWriterITCase.java:208)
> 09:39:01.799 [ERROR] 
> testRecoverWithEmptyState(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase)
>   Time elapsed: 0.235 s  <<< ERROR!
> java.lang.NullPointerException
>   at 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testResumeAfterMultiplePersist(HadoopS3RecoverableWriterITCase.java:384)
>   at 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testResumeAfterMultiplePersistWithSmallData(HadoopS3RecoverableWriterITCase.java:352)
>   at 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testRecoverWithEmptyState(HadoopS3RecoverableWriterITCase.java:302)
> 

[jira] [Commented] (FLINK-16550) HadoopS3* tests fail with NullPointerException exceptions

2020-03-12 Thread Arvid Heise (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-16550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058036#comment-17058036
 ] 

Arvid Heise commented on FLINK-16550:
-

Should also be backported to 1.10.

> HadoopS3* tests fail with NullPointerException exceptions
> -
>
> Key: FLINK-16550
> URL: https://issues.apache.org/jira/browse/FLINK-16550
> Project: Flink
>  Issue Type: Bug
>  Components: FileSystems
>Affects Versions: 1.11.0
>Reporter: Robert Metzger
>Assignee: Arvid Heise
>Priority: Blocker
>  Labels: pull-request-available
> Fix For: 1.11.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Logs: 
> https://travis-ci.org/github/apache/flink/jobs/660975486?utm_medium=notification
> All subsequent builds failed as well. It is likely that this commit / 
> FLINK-16014 introduced the issue, as these tests depend on S3 credentials to 
> be available.
> {code}
> 09:38:48.022 [INFO] ---
> 09:38:48.025 [INFO]  T E S T S
> 09:38:48.026 [INFO] ---
> 09:38:48.657 [INFO] Running 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase
> 09:38:48.669 [INFO] Running 
> org.apache.flink.fs.s3hadoop.HadoopS3FileSystemITCase
> 09:38:54.541 [ERROR] Tests run: 3, Failures: 0, Errors: 3, Skipped: 0, Time 
> elapsed: 5.88 s <<< FAILURE! - in 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase
> 09:38:54.542 [ERROR] 
> testResumeAfterCommit(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase)
>   Time elapsed: 3.592 s  <<< ERROR!
> java.lang.Exception: Unexpected exception, expected but 
> was
>   at 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase.testResumeAfterCommit(HadoopS3RecoverableWriterExceptionITCase.java:162)
> 09:38:54.542 [ERROR] 
> testResumeWithWrongOffset(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase)
>   Time elapsed: 0.24 s  <<< ERROR!
> java.lang.Exception: Unexpected exception, expected but 
> was
>   at 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase.testResumeWithWrongOffset(HadoopS3RecoverableWriterExceptionITCase.java:182)
> 09:38:54.542 [ERROR] 
> testExceptionWritingAfterCloseForCommit(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase)
>   Time elapsed: 0.448 s  <<< ERROR!
> java.lang.Exception: Unexpected exception, expected but 
> was
>   at 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase.testExceptionWritingAfterCloseForCommit(HadoopS3RecoverableWriterExceptionITCase.java:144)
> 09:38:55.173 [INFO] Running 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase
> 09:38:58.737 [ERROR] Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time 
> elapsed: 10.066 s <<< FAILURE! - in 
> org.apache.flink.fs.s3hadoop.HadoopS3FileSystemITCase
> 09:38:58.737 [ERROR] 
> testDirectoryListing(org.apache.flink.fs.s3hadoop.HadoopS3FileSystemITCase)  
> Time elapsed: 3.448 s  <<< ERROR!
> java.io.FileNotFoundException: No such file or directory: 
> s3://[secure]/temp/tests-f37db36e-c116-4c58-a16b-8ca241baae4b/testdir
> 09:38:59.447 [INFO] Running 
> org.apache.flink.fs.s3hadoop.HadoopS3FileSystemBehaviorITCase
> 09:39:01.791 [ERROR] Tests run: 13, Failures: 0, Errors: 13, Skipped: 0, Time 
> elapsed: 6.611 s <<< FAILURE! - in 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase
> 09:39:01.797 [ERROR] 
> testCloseWithNoData(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase)
>   Time elapsed: 2.394 s  <<< ERROR!
> java.lang.NullPointerException
>   at 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testCloseWithNoData(HadoopS3RecoverableWriterITCase.java:186)
> 09:39:01.798 [ERROR] 
> testCommitAfterPersist(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase)
>   Time elapsed: 0.191 s  <<< ERROR!
> java.lang.NullPointerException
>   at 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testCommitAfterPersist(HadoopS3RecoverableWriterITCase.java:208)
> 09:39:01.799 [ERROR] 
> testRecoverWithEmptyState(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase)
>   Time elapsed: 0.235 s  <<< ERROR!
> java.lang.NullPointerException
>   at 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testResumeAfterMultiplePersist(HadoopS3RecoverableWriterITCase.java:384)
>   at 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testResumeAfterMultiplePersistWithSmallData(HadoopS3RecoverableWriterITCase.java:352)
>   at 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testRecoverWithEmptyState(HadoopS3RecoverableWriterITCase.java:302)
> 09:39:01.799 [ERROR] 
> 

[jira] [Commented] (FLINK-16550) HadoopS3* tests fail with NullPointerException exceptions

2020-03-11 Thread Robert Metzger (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-16550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057451#comment-17057451
 ] 

Robert Metzger commented on FLINK-16550:


Thanks for providing a patch Arvid. I assigned you to the ticket.

> HadoopS3* tests fail with NullPointerException exceptions
> -
>
> Key: FLINK-16550
> URL: https://issues.apache.org/jira/browse/FLINK-16550
> Project: Flink
>  Issue Type: Bug
>  Components: FileSystems
>Affects Versions: 1.11.0
>Reporter: Robert Metzger
>Assignee: Arvid Heise
>Priority: Blocker
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Logs: 
> https://travis-ci.org/github/apache/flink/jobs/660975486?utm_medium=notification
> All subsequent builds failed as well. It is likely that this commit / 
> FLINK-16014 introduced the issue, as these tests depend on S3 credentials to 
> be available.
> {code}
> 09:38:48.022 [INFO] ---
> 09:38:48.025 [INFO]  T E S T S
> 09:38:48.026 [INFO] ---
> 09:38:48.657 [INFO] Running 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase
> 09:38:48.669 [INFO] Running 
> org.apache.flink.fs.s3hadoop.HadoopS3FileSystemITCase
> 09:38:54.541 [ERROR] Tests run: 3, Failures: 0, Errors: 3, Skipped: 0, Time 
> elapsed: 5.88 s <<< FAILURE! - in 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase
> 09:38:54.542 [ERROR] 
> testResumeAfterCommit(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase)
>   Time elapsed: 3.592 s  <<< ERROR!
> java.lang.Exception: Unexpected exception, expected but 
> was
>   at 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase.testResumeAfterCommit(HadoopS3RecoverableWriterExceptionITCase.java:162)
> 09:38:54.542 [ERROR] 
> testResumeWithWrongOffset(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase)
>   Time elapsed: 0.24 s  <<< ERROR!
> java.lang.Exception: Unexpected exception, expected but 
> was
>   at 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase.testResumeWithWrongOffset(HadoopS3RecoverableWriterExceptionITCase.java:182)
> 09:38:54.542 [ERROR] 
> testExceptionWritingAfterCloseForCommit(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase)
>   Time elapsed: 0.448 s  <<< ERROR!
> java.lang.Exception: Unexpected exception, expected but 
> was
>   at 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase.testExceptionWritingAfterCloseForCommit(HadoopS3RecoverableWriterExceptionITCase.java:144)
> 09:38:55.173 [INFO] Running 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase
> 09:38:58.737 [ERROR] Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time 
> elapsed: 10.066 s <<< FAILURE! - in 
> org.apache.flink.fs.s3hadoop.HadoopS3FileSystemITCase
> 09:38:58.737 [ERROR] 
> testDirectoryListing(org.apache.flink.fs.s3hadoop.HadoopS3FileSystemITCase)  
> Time elapsed: 3.448 s  <<< ERROR!
> java.io.FileNotFoundException: No such file or directory: 
> s3://[secure]/temp/tests-f37db36e-c116-4c58-a16b-8ca241baae4b/testdir
> 09:38:59.447 [INFO] Running 
> org.apache.flink.fs.s3hadoop.HadoopS3FileSystemBehaviorITCase
> 09:39:01.791 [ERROR] Tests run: 13, Failures: 0, Errors: 13, Skipped: 0, Time 
> elapsed: 6.611 s <<< FAILURE! - in 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase
> 09:39:01.797 [ERROR] 
> testCloseWithNoData(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase)
>   Time elapsed: 2.394 s  <<< ERROR!
> java.lang.NullPointerException
>   at 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testCloseWithNoData(HadoopS3RecoverableWriterITCase.java:186)
> 09:39:01.798 [ERROR] 
> testCommitAfterPersist(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase)
>   Time elapsed: 0.191 s  <<< ERROR!
> java.lang.NullPointerException
>   at 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testCommitAfterPersist(HadoopS3RecoverableWriterITCase.java:208)
> 09:39:01.799 [ERROR] 
> testRecoverWithEmptyState(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase)
>   Time elapsed: 0.235 s  <<< ERROR!
> java.lang.NullPointerException
>   at 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testResumeAfterMultiplePersist(HadoopS3RecoverableWriterITCase.java:384)
>   at 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testResumeAfterMultiplePersistWithSmallData(HadoopS3RecoverableWriterITCase.java:352)
>   at 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testRecoverWithEmptyState(HadoopS3RecoverableWriterITCase.java:302)
> 09:39:01.799 [ERROR] 

[jira] [Commented] (FLINK-16550) HadoopS3* tests fail with NullPointerException exceptions

2020-03-11 Thread Nico Kruber (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-16550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057449#comment-17057449
 ] 

Nico Kruber commented on FLINK-16550:
-

This actually also showed up in an end-to-end Flink cluster setup where I 
couldn't download my savepoint from s3 anymore and this showed up on job 
submission:

{code}
 The program finished with the following exception:

org.apache.flink.client.program.ProgramInvocationException: The main method 
caused an error: java.util.concurrent.ExecutionException: 
org.apache.flink.runtime.client.JobSubmissionException: Failed to submit 
JobGraph.
at 
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:335)
at 
org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205)
at 
org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:138)
at 
org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:664)
at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:213)
at 
org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:895)
at 
org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:968)
at 
org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:968)
Caused by: java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
org.apache.flink.runtime.client.JobSubmissionException: Failed to submit 
JobGraph.
at org.apache.flink.util.ExceptionUtils.rethrow(ExceptionUtils.java:199)
at 
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.executeAsync(StreamExecutionEnvironment.java:1741)
at 
org.apache.flink.streaming.api.environment.StreamContextEnvironment.executeAsync(StreamContextEnvironment.java:94)
at 
org.apache.flink.streaming.api.environment.StreamContextEnvironment.execute(StreamContextEnvironment.java:63)
at 
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1620)
at 
org.apache.flink.streaming.examples.windowing.TopSpeedWindowing.main(TopSpeedWindowing.java:96)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:321)
... 8 more
Caused by: java.util.concurrent.ExecutionException: 
org.apache.flink.runtime.client.JobSubmissionException: Failed to submit 
JobGraph.
at 
java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
at 
java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
at 
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.executeAsync(StreamExecutionEnvironment.java:1736)
... 17 more
Caused by: org.apache.flink.runtime.client.JobSubmissionException: Failed to 
submit JobGraph.
at 
org.apache.flink.client.program.rest.RestClusterClient.lambda$submitJob$7(RestClusterClient.java:359)
at 
java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:884)
at 
java.util.concurrent.CompletableFuture$UniExceptionally.tryFire(CompletableFuture.java:866)
at 
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
at 
java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990)
at 
org.apache.flink.runtime.concurrent.FutureUtils.lambda$retryOperationWithDelay$8(FutureUtils.java:274)
at 
java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
at 
java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
at 
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
at 
java.util.concurrent.CompletableFuture.postFire(CompletableFuture.java:575)
at 
java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:943)
at 
java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:456)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.flink.runtime.rest.util.RestClientException: [Internal 
server error., (JobManagerRunnerImpl.java:152)
at 

[jira] [Commented] (FLINK-16550) HadoopS3* tests fail with NullPointerException exceptions

2020-03-11 Thread Chesnay Schepler (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-16550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057370#comment-17057370
 ] 

Chesnay Schepler commented on FLINK-16550:
--

ping [~AHeise]

> HadoopS3* tests fail with NullPointerException exceptions
> -
>
> Key: FLINK-16550
> URL: https://issues.apache.org/jira/browse/FLINK-16550
> Project: Flink
>  Issue Type: Bug
>  Components: FileSystems
>Affects Versions: 1.11.0
>Reporter: Robert Metzger
>Priority: Blocker
>
> Logs: 
> https://travis-ci.org/github/apache/flink/jobs/660975486?utm_medium=notification
> All subsequent builds failed as well. It is likely that this commit / 
> FLINK-16014 introduced the issue, as these tests depend on S3 credentials to 
> be available.
> {code}
> 09:38:48.022 [INFO] ---
> 09:38:48.025 [INFO]  T E S T S
> 09:38:48.026 [INFO] ---
> 09:38:48.657 [INFO] Running 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase
> 09:38:48.669 [INFO] Running 
> org.apache.flink.fs.s3hadoop.HadoopS3FileSystemITCase
> 09:38:54.541 [ERROR] Tests run: 3, Failures: 0, Errors: 3, Skipped: 0, Time 
> elapsed: 5.88 s <<< FAILURE! - in 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase
> 09:38:54.542 [ERROR] 
> testResumeAfterCommit(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase)
>   Time elapsed: 3.592 s  <<< ERROR!
> java.lang.Exception: Unexpected exception, expected but 
> was
>   at 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase.testResumeAfterCommit(HadoopS3RecoverableWriterExceptionITCase.java:162)
> 09:38:54.542 [ERROR] 
> testResumeWithWrongOffset(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase)
>   Time elapsed: 0.24 s  <<< ERROR!
> java.lang.Exception: Unexpected exception, expected but 
> was
>   at 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase.testResumeWithWrongOffset(HadoopS3RecoverableWriterExceptionITCase.java:182)
> 09:38:54.542 [ERROR] 
> testExceptionWritingAfterCloseForCommit(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase)
>   Time elapsed: 0.448 s  <<< ERROR!
> java.lang.Exception: Unexpected exception, expected but 
> was
>   at 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase.testExceptionWritingAfterCloseForCommit(HadoopS3RecoverableWriterExceptionITCase.java:144)
> 09:38:55.173 [INFO] Running 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase
> 09:38:58.737 [ERROR] Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time 
> elapsed: 10.066 s <<< FAILURE! - in 
> org.apache.flink.fs.s3hadoop.HadoopS3FileSystemITCase
> 09:38:58.737 [ERROR] 
> testDirectoryListing(org.apache.flink.fs.s3hadoop.HadoopS3FileSystemITCase)  
> Time elapsed: 3.448 s  <<< ERROR!
> java.io.FileNotFoundException: No such file or directory: 
> s3://[secure]/temp/tests-f37db36e-c116-4c58-a16b-8ca241baae4b/testdir
> 09:38:59.447 [INFO] Running 
> org.apache.flink.fs.s3hadoop.HadoopS3FileSystemBehaviorITCase
> 09:39:01.791 [ERROR] Tests run: 13, Failures: 0, Errors: 13, Skipped: 0, Time 
> elapsed: 6.611 s <<< FAILURE! - in 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase
> 09:39:01.797 [ERROR] 
> testCloseWithNoData(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase)
>   Time elapsed: 2.394 s  <<< ERROR!
> java.lang.NullPointerException
>   at 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testCloseWithNoData(HadoopS3RecoverableWriterITCase.java:186)
> 09:39:01.798 [ERROR] 
> testCommitAfterPersist(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase)
>   Time elapsed: 0.191 s  <<< ERROR!
> java.lang.NullPointerException
>   at 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testCommitAfterPersist(HadoopS3RecoverableWriterITCase.java:208)
> 09:39:01.799 [ERROR] 
> testRecoverWithEmptyState(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase)
>   Time elapsed: 0.235 s  <<< ERROR!
> java.lang.NullPointerException
>   at 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testResumeAfterMultiplePersist(HadoopS3RecoverableWriterITCase.java:384)
>   at 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testResumeAfterMultiplePersistWithSmallData(HadoopS3RecoverableWriterITCase.java:352)
>   at 
> org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testRecoverWithEmptyState(HadoopS3RecoverableWriterITCase.java:302)
> 09:39:01.799 [ERROR] 
> testRecoverFromIntermWithoutAdditionalState(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase)
>   Time elapsed: 0.181 s  <<< ERROR!
> java.lang.NullPointerException