[
https://issues.apache.org/jira/browse/HDFS-7430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HDFS-7430:
---------------------------------------
Resolution: Fixed
Fix Version/s: 3.0.0
Status: Resolved (was: Patch Available)
Committed to trunk. It took 3 months, and 12 revisions, but we got it! Thanks
for the reviews, Andrew.
As per Nicholas' suggestion, let's let this soak for a week or two in trunk
before we backport to branch-2. If we spot any bugs this will give us a chance
to fix them.
> Rewrite the BlockScanner to use O(1) memory and use multiple threads
> --------------------------------------------------------------------
>
> Key: HDFS-7430
> URL: https://issues.apache.org/jira/browse/HDFS-7430
> Project: Hadoop HDFS
> Issue Type: Improvement
> Affects Versions: 2.7.0
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Fix For: 3.0.0
>
> Attachments: HDFS-7430.002.patch, HDFS-7430.003.patch,
> HDFS-7430.004.patch, HDFS-7430.005.patch, HDFS-7430.006.patch,
> HDFS-7430.007.patch, HDFS-7430.008.patch, HDFS-7430.009.patch,
> HDFS-7430.010.patch, HDFS-7430.011.patch, HDFS-7430.012.patch, memory.png
>
>
> We should update the BlockScanner to use a constant amount of memory by
> keeping track of what block was scanned last, rather than by tracking the
> scan status of all blocks in memory. Also, instead of having just one
> thread, we should have a verification thread per hard disk (or other volume),
> scanning at a configurable rate of bytes per second.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
