[ http://issues.apache.org/jira/browse/HADOOP-212?page=all ]
Owen O'Malley updated HADOOP-212:
---------------------------------
Attachment: dfs-blocksize.patch
Ok, here is the patch.
Changes dfs block size from a compile time constant to a parameter that is set
when a file is created.
1. FileSystem.getBlockSize becomes getDefaultBlockSize
2. A new method FileSystem.getBlockSize(path) finds the blocksize of a file.
3. Block size is added to FileSystem.create
4. InputFormatBase uses the block size of each file rather than the global
constant.
5. I followed the convention of using DfsPath to cache meta information values
associatied with the dfs file.
6. FileUnderConstruction records the block size
7. Removed check to make sure that the block size was shorter than the global
value.
8. Add a new value
> allow changes to dfs block size
> -------------------------------
>
> Key: HADOOP-212
> URL: http://issues.apache.org/jira/browse/HADOOP-212
> Project: Hadoop
> Type: Improvement
> Components: dfs
> Versions: 0.2
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Priority: Critical
> Fix For: 0.3
> Attachments: dfs-blocksize.patch
>
> Trying to change the DFS block size, led the realization that the 32,000,000
> was hard coded into the source code. I propose:
> 1. Change the default block size to 64 * 1024 * 1024.
> 2. Add the config variable dfs.block.size that sets the default block size.
> 3. Add a parameter to the FileSystem, DFSClient, and ClientProtocol create
> method that let's the user control the block size.
> 4. Rename the FileSystem.getBlockSize to getDefaultBlockSize.
> 5. Add a new method to FileSytem.getBlockSize that takes a pathname.
> 6. Use long for the block size in the API, which is what was used before.
> However, the implementation will not work if block size is set bigger than
> 2**31.
> 7. Have the InputFormatBase use the blocksize of each file to determine the
> split size.
> Thoughts?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
