[GitHub] [hadoop] ahmarsuhail commented on a diff in pull request #4212: HADOOP-18175. fix test failures with prefetching s3a input stream

2022-04-28 Thread GitBox


ahmarsuhail commented on code in PR #4212:
URL: https://github.com/apache/hadoop/pull/4212#discussion_r860843620


##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/read/S3PrefetchingInputStream.java:
##
@@ -103,8 +103,7 @@ public synchronized int available() throws IOException {
*/
   @Override
   public synchronized long getPos() throws IOException {

Review Comment:
   this method no longer throws an IOException 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[GitHub] [hadoop] ahmarsuhail commented on a diff in pull request #4212: HADOOP-18175. fix test failures with prefetching s3a input stream

2022-04-26 Thread GitBox


ahmarsuhail commented on code in PR #4212:
URL: https://github.com/apache/hadoop/pull/4212#discussion_r858421027


##
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3ARequesterPays.java:
##
@@ -71,11 +72,16 @@ public void testRequesterPaysOptionSuccess() throws 
Throwable {
   inputStream.seek(0);
   inputStream.readByte();
 
-  // Verify > 1 call was made, so we're sure it is correctly configured 
for each request
-  IOStatisticAssertions
-  .assertThatStatisticCounter(inputStream.getIOStatistics(),
-  StreamStatisticNames.STREAM_READ_OPENED)
-  .isGreaterThan(1);
+  if (conf.getBoolean(PREFETCH_ENABLED_KEY, true)) {
+// For S3PrefetchingInputStream, verify a call was made
+
IOStatisticAssertions.assertThatStatisticCounter(inputStream.getIOStatistics(),
+StreamStatisticNames.STREAM_READ_OPENED).isEqualTo(1);
+  } else {
+// For S3InputStream, verify > 1 call was made,

Review Comment:
   should this be For S3AInputStream ?



##
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AUnbuffer.java:
##
@@ -192,6 +197,12 @@ private boolean isObjectStreamOpen(FSDataInputStream 
inputStream) {
 return ((S3AInputStream) 
inputStream.getWrappedStream()).isObjectStreamOpen();
   }
 
+  private void skipIfCannotUnbuffer(InputStream inputStream) {

Review Comment:
   You should just be able to use 
`inputStream.hasCapability(StreamCapabilities.UNBUFFER)` instead of this method



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org