[ https://issues.apache.org/jira/browse/HDFS-14872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16942067#comment-16942067 ]
Erik Krogen commented on HDFS-14872: ------------------------------------ +1 on this idea from me, we have considered building the same internally using the same pre-allocation strategy that you've described. For a large file with numerous blocks, it should be able to make a big difference. > Read HDFS Blocks in Random Order > -------------------------------- > > Key: HDFS-14872 > URL: https://issues.apache.org/jira/browse/HDFS-14872 > Project: Hadoop HDFS > Issue Type: New Feature > Components: hdfs-client > Affects Versions: 2.8.5, 3.2.1 > Reporter: David Mollitor > Priority: Major > > When the HDFS client is downloading (copying) an entire file, allow the > client to download the blocks in random order. If a lot of clients are > reading the same file, in parallel, they will all download the first block, > the second block, and so on, stampeding down the line. > It would be interesting to spread the load across across all the available > DataNodes. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org