[
https://issues.apache.org/jira/browse/HDFS-516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12755048#action_12755048
]
Todd Lipcon commented on HDFS-516:
----------------------------------
I haven't had a chance to look over the patch as of yet, but I have one concern:
Is there a plan for deprecation in the event that HDFS itself achieves similar
performance? I think having an entirely separate FS implementation that only
differs in performance is not a good idea longterm. Using this contrib project
as an experimentation ground sounds great, but I think long term we should
focus on improving DistributedFileSystem's performance itself, and not
bifurcate the code into the "fast version that we dont really support because
it's contrib" and "slow version that we do".
I'll try to find a chance to look over the patch soon, but in the meantime do
you have any thoughts on the above?
> Low Latency distributed reads
> -----------------------------
>
> Key: HDFS-516
> URL: https://issues.apache.org/jira/browse/HDFS-516
> Project: Hadoop HDFS
> Issue Type: New Feature
> Reporter: Jay Booth
> Priority: Minor
> Attachments: hdfs-516-20090912.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> I created a method for low latency random reads using NIO on the server side
> and simulated OS paging with LRU caching and lookahead on the client side.
> Some applications could include lucene searching (term->doc and doc->offset
> mappings are likely to be in local cache, thus much faster than nutch's
> current FsDirectory impl and binary search through record files (bytes at
> 1/2, 1/4, 1/8 marks are likely to be cached)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
