[ http://issues.apache.org/jira/browse/HADOOP-519?page=comments#action_12435611 ] Milind Bhandarkar commented on HADOOP-519: ------------------------------------------
> Why not do this now? Because the normal read method has some other deficiencies as well. For example, even for reading just a few bytes from the block, the datanode tries to send out the full block. Also, even when the seeks target is within the same block, the existing connection to the datanode is clossed anyway and a new one opened. Since these changes are related to clean-up of duplicate code, I had planned to do them as a different patch. > HDFS File API should be extended to include positional read > ----------------------------------------------------------- > > Key: HADOOP-519 > URL: http://issues.apache.org/jira/browse/HADOOP-519 > Project: Hadoop > Issue Type: New Feature > Components: dfs > Affects Versions: 0.6.0 > Environment: All > Reporter: Milind Bhandarkar > Assigned To: Milind Bhandarkar > Fix For: 0.7.0 > > Attachments: pread.patch > > > HDFS Input streams should support positional read. Positional read (such as > the pread syscall on linux) allows reading for a specified offset without > affecting the current file offset. Since the underlying file state is not > touched, pread can be used efficiently in multi-threaded programs. > Here is how I plan to implement it. > Provide PositionedReadable interface, with the following methods: > int read(long position, byte[] buffer, int offset, int length); > void readFully(long position, byte[] buffer, int offset, int length); > void readFully(long position, byte[] buffer); > Abstract class FSInputStream would provide default implementation of the > above methods using getPos(), seek() and read() methods. The default > implementation is inefficient in multi-threaded programs since it locks the > object while seeking, reading, and restoring to old state. > DFSClient.DFSInputStream, which extends FSInputStream will provide an > efficient non-synchronized implementation for above calls. > In addition, FSDataInputStream, which is a wrapper around FSInputStream, will > provide wrapper methods for above read methods as well. > Patch forthcoming early next week. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
