[ https://issues.apache.org/jira/browse/HADOOP-18175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17512578#comment-17512578 ]
Steve Loughran commented on HADOOP-18175: ----------------------------------------- h3. ITestS3AContractUnbuffer fix: getPos() on a closed file to return 0; {code} [ERROR] testUnbufferOnClosedFile(org.apache.hadoop.fs.contract.s3a.ITestS3AContractUnbuffer) Time elapsed: 0.833 s <<< ERROR! java.io.IOException: Stream is closed! at org.apache.hadoop.fs.s3a.read.S3PrefetchingInputStream.throwIfClosed(S3PrefetchingInputStream.java:224) at org.apache.hadoop.fs.s3a.read.S3PrefetchingInputStream.getPos(S3PrefetchingInputStream.java:105) at org.apache.hadoop.fs.FSDataInputStream.getPos(FSDataInputStream.java:81) at org.apache.hadoop.fs.contract.AbstractContractUnbufferTest.unbuffer(AbstractContractUnbufferTest.java:124) at org.apache.hadoop.fs.contract.AbstractContractUnbufferTest.testUnbufferOnClosedFile(AbstractContractUnbufferTest.java:90) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) {code} h3. ITestS3AUnbuffer instanceof and statistics assertions failing. {code} [ERROR] testUnbuffer(org.apache.hadoop.fs.s3a.ITestS3AUnbuffer) Time elapsed: 2.385 s <<< FAILURE! java.lang.AssertionError at org.junit.Assert.fail(Assert.java:87) at org.junit.Assert.assertTrue(Assert.java:42) at org.junit.Assert.assertTrue(Assert.java:53) at org.apache.hadoop.fs.s3a.ITestS3AUnbuffer.testUnbuffer(ITestS3AUnbuffer.java:75) [ERROR] testUnbufferStreamStatistics(org.apache.hadoop.fs.s3a.ITestS3AUnbuffer) Time elapsed: 1.115 s <<< FAILURE! java.lang.AssertionError: Count of stream_read_bytes starting=0 current=0 diff=0: stream_read_bytes expected:<8> but was:<0> at org.junit.Assert.fail(Assert.java:89) at org.junit.Assert.failNotEquals(Assert.java:835) at org.junit.Assert.assertEquals(Assert.java:647) at org.apache.hadoop.fs.s3a.S3ATestUtils$MetricDiff.assertDiffEquals(S3ATestUtils.java:998) at org.apache.hadoop.fs.s3a.S3ATestUtils$MetricDiff.assertDiffEquals(S3ATestUtils.java:1007) at org.apache.hadoop.fs.s3a.ITestS3AUnbuffer.testUnbufferStreamStatistics(ITestS3AUnbuffer.java:149) {code} h3. ITestS3ARequesterPays statistics assertion. {code} [ERROR] testRequesterPaysOptionSuccess(org.apache.hadoop.fs.s3a.ITestS3ARequesterPays) Time elapsed: 2.198 s <<< FAILURE! java.lang.AssertionError: [Counter named stream_read_opened] Expecting: <0L> to be greater than: <1L> at org.apache.hadoop.fs.s3a.ITestS3ARequesterPays.testRequesterPaysOptionSuccess(ITestS3ARequesterPays.java:78) {code} fix: wire up the statistics and/or change assertion based on stream type h3. testStatistics(org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextStatistics) T {code} java.io.EOFException: Attempted to seek or read past the end of the file 512 at org.apache.hadoop.fs.s3a.read.S3InputStream.throwIfInvalidSeek(S3InputStream.java:439) at org.apache.hadoop.fs.s3a.read.S3InputStream.seek(S3InputStream.java:234) at org.apache.hadoop.fs.s3a.read.S3PrefetchingInputStream.seek(S3PrefetchingInputStream.java:161) at org.apache.hadoop.fs.FSInputStream.read(FSInputStream.java:85) at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:98) at org.apache.hadoop.fs.FCStatisticsBaseTest.testStatistics(FCStatisticsBaseTest.java:107) {code} > test failures with prefetching s3a input stream > ----------------------------------------------- > > Key: HADOOP-18175 > URL: https://issues.apache.org/jira/browse/HADOOP-18175 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3, test > Affects Versions: 3.4.0 > Reporter: Steve Loughran > Priority: Major > > identify and fix all test regressions from the prefetching s3a input stream -- This message was sent by Atlassian Jira (v8.20.1#820001) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org