[
https://issues.apache.org/jira/browse/HADOOP-2705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12562740#action_12562740
]
Doug Cutting commented on HADOOP-2705:
--------------------------------------
Nutch applications can easily keep hundreds of files open, and each file has
multiple layers of buffers. With 1000 files, each with say 5 buffers, this
would add 160MB of heap, which should not be done lightly.
> io.file.buffer.size should default to a value larger than 4k
> ------------------------------------------------------------
>
> Key: HADOOP-2705
> URL: https://issues.apache.org/jira/browse/HADOOP-2705
> Project: Hadoop Core
> Issue Type: Improvement
> Components: conf
> Reporter: Chris Douglas
> Assignee: Chris Douglas
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: 2705-0.patch
>
>
> Tests using HADOOP-2406 suggest that increasing this to 32k from 4k improves
> read times for block, lzo compressed SequenceFiles by over 40%; 32k is a
> relatively conservative bump.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
