[ https://issues.apache.org/jira/browse/HADOOP-11270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358829#comment-15358829 ]
Steve Loughran commented on HADOOP-11270: ----------------------------------------- +findbugs warning isn't to be ignored. Skip result should be used to check range skipped, consider some policy if != desired. Warn? repeat seek? {{S3AInputStream}} does a warning > Seek behavior difference between NativeS3FsInputStream and DFSInputStream > ------------------------------------------------------------------------- > > Key: HADOOP-11270 > URL: https://issues.apache.org/jira/browse/HADOOP-11270 > Project: Hadoop Common > Issue Type: Bug > Components: fs/s3 > Affects Versions: 2.5.1 > Reporter: Venkata Puneet Ravuri > Assignee: Venkata Puneet Ravuri > Attachments: HADOOP-11270.02.patch, HADOOP-11270.03.patch, > HADOOP-11270.04.patch, HADOOP-11270.patch > > > There is a difference in behavior while seeking a given file present > in S3 using NativeS3FileSystem$NativeS3FsInputStream and a file present in > HDFS using DFSInputStream. > If we seek to the end of the file incase of NativeS3FsInputStream, it fails > with exception "java.io.EOFException: Attempted to seek or read past the end > of the file". That is because a getObject request is issued on the S3 object > with range start as value of length of file. > This is the complete exception stack:- > Caused by: java.io.EOFException: Attempted to seek or read past the end of > the file > at > org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.processException(Jets3tNativeFileSystemStore.java:462) > at > org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.handleException(Jets3tNativeFileSystemStore.java:411) > at > org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.retrieve(Jets3tNativeFileSystemStore.java:234) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:601) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) > at org.apache.hadoop.fs.s3native.$Proxy17.retrieve(Unknown Source) > at > org.apache.hadoop.fs.s3native.NativeS3FileSystem$NativeS3FsInputStream.seek(NativeS3FileSystem.java:205) > at > org.apache.hadoop.fs.BufferedFSInputStream.seek(BufferedFSInputStream.java:96) > at > org.apache.hadoop.fs.BufferedFSInputStream.skip(BufferedFSInputStream.java:67) > at java.io.DataInputStream.skipBytes(DataInputStream.java:220) > at org.apache.hadoop.hive.ql.io.RCFile$ValueBuffer.readFields(RCFile.java:739) > at > org.apache.hadoop.hive.ql.io.RCFile$Reader.currentValueBuffer(RCFile.java:1720) > at org.apache.hadoop.hive.ql.io.RCFile$Reader.getCurrentRow(RCFile.java:1898) > at > org.apache.hadoop.hive.ql.io.RCFileRecordReader.next(RCFileRecordReader.java:149) > at > org.apache.hadoop.hive.ql.io.RCFileRecordReader.next(RCFileRecordReader.java:44) > at > org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:339) > ... 15 more -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org