[jira] [Updated] (HDFS-5031) BlockScanner scans the block multiple times and on restart scans everything
[ https://issues.apache.org/jira/browse/HDFS-5031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-5031: - Priority: Blocker (was: Major) Fix Version/s: (was: 2.3.0) 2.1.2-beta 3.0.0 I pulled this into branch-2.1-beta. This seems to fix HDFS-5225. BlockScanner scans the block multiple times and on restart scans everything --- Key: HDFS-5031 URL: https://issues.apache.org/jira/browse/HDFS-5031 Project: Hadoop HDFS Issue Type: Bug Components: datanode Affects Versions: 3.0.0, 2.1.0-beta Reporter: Vinay Assignee: Vinay Priority: Blocker Fix For: 3.0.0, 2.1.2-beta Attachments: HDFS-5031.patch, HDFS-5031.patch, HDFS-5031.patch BlockScanner scans the block twice, also on restart of datanode scans everything. Steps: 1. Write blocks with interval of more than 5 seconds. write new block on completion of scan for written block. Each time datanode scans new block, it also scans, previous block which is already scanned. Now after restart, datanode scans all blocks again. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-5031) BlockScanner scans the block multiple times and on restart scans everything
[ https://issues.apache.org/jira/browse/HDFS-5031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-5031: Resolution: Fixed Fix Version/s: 2.3.0 Status: Resolved (was: Patch Available) BlockScanner scans the block multiple times and on restart scans everything --- Key: HDFS-5031 URL: https://issues.apache.org/jira/browse/HDFS-5031 Project: Hadoop HDFS Issue Type: Bug Components: datanode Affects Versions: 3.0.0, 2.1.0-beta Reporter: Vinay Assignee: Vinay Fix For: 2.3.0 Attachments: HDFS-5031.patch, HDFS-5031.patch, HDFS-5031.patch BlockScanner scans the block twice, also on restart of datanode scans everything. Steps: 1. Write blocks with interval of more than 5 seconds. write new block on completion of scan for written block. Each time datanode scans new block, it also scans, previous block which is already scanned. Now after restart, datanode scans all blocks again. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-5031) BlockScanner scans the block multiple times and on restart scans everything
[ https://issues.apache.org/jira/browse/HDFS-5031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinay updated HDFS-5031: Attachment: HDFS-5031.patch Updated the patch. 1. Kept equals() and hashCode() back. 2. Added block comparison also in LAST_SCAN_TIME_COMPARATOR, as TreeMap will use comparator if available to check the existence of the element before adding. Which will make block scan getting delayed if multiple blocks have same scantime ( this is possible if blocks are small in size ) BlockScanner scans the block multiple times and on restart scans everything --- Key: HDFS-5031 URL: https://issues.apache.org/jira/browse/HDFS-5031 Project: Hadoop HDFS Issue Type: Bug Components: datanode Affects Versions: 3.0.0, 2.1.0-beta Reporter: Vinay Assignee: Vinay Attachments: HDFS-5031.patch, HDFS-5031.patch, HDFS-5031.patch BlockScanner scans the block twice, also on restart of datanode scans everything. Steps: 1. Write blocks with interval of more than 5 seconds. write new block on completion of scan for written block. Each time datanode scans new block, it also scans, previous block which is already scanned. Now after restart, datanode scans all blocks again. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-5031) BlockScanner scans the block multiple times and on restart scans everything
[ https://issues.apache.org/jira/browse/HDFS-5031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinay updated HDFS-5031: Attachment: HDFS-5031.patch Updated patch with comments BlockScanner scans the block multiple times and on restart scans everything --- Key: HDFS-5031 URL: https://issues.apache.org/jira/browse/HDFS-5031 Project: Hadoop HDFS Issue Type: Bug Components: datanode Affects Versions: 3.0.0, 2.1.0-beta Reporter: Vinay Assignee: Vinay Attachments: HDFS-5031.patch, HDFS-5031.patch BlockScanner scans the block twice, also on restart of datanode scans everything. Steps: 1. Write blocks with interval of more than 5 seconds. write new block on completion of scan for written block. Each time datanode scans new block, it also scans, previous block which is already scanned. Now after restart, datanode scans all blocks again. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-5031) BlockScanner scans the block multiple times and on restart scans everything
[ https://issues.apache.org/jira/browse/HDFS-5031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-5031: Priority: Major (was: Blocker) BlockScanner scans the block multiple times and on restart scans everything --- Key: HDFS-5031 URL: https://issues.apache.org/jira/browse/HDFS-5031 Project: Hadoop HDFS Issue Type: Bug Components: datanode Affects Versions: 3.0.0, 2.1.0-beta Reporter: Vinay Assignee: Vinay Attachments: HDFS-5031.patch BlockScanner scans the block twice, also on restart of datanode scans everything. Steps: 1. Write blocks with interval of more than 5 seconds. write new block on completion of scan for written block. Each time datanode scans new block, it also scans, previous block which is already scanned. Now after restart, datanode scans all blocks again. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-5031) BlockScanner scans the block multiple times and on restart scans everything
[ https://issues.apache.org/jira/browse/HDFS-5031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinay updated HDFS-5031: Status: Patch Available (was: Open) BlockScanner scans the block multiple times and on restart scans everything --- Key: HDFS-5031 URL: https://issues.apache.org/jira/browse/HDFS-5031 Project: Hadoop HDFS Issue Type: Bug Components: datanode Affects Versions: 3.0.0, 2.1.0-beta Reporter: Vinay Assignee: Vinay Priority: Blocker Attachments: HDFS-5031.patch BlockScanner scans the block twice, also on restart of datanode scans everything. Steps: 1. Write blocks with interval of more than 5 seconds. write new block on completion of scan for written block. Each time datanode scans new block, it also scans, previous block which is already scanned. Now after restart, datanode scans all blocks again. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-5031) BlockScanner scans the block multiple times and on restart scans everything
[ https://issues.apache.org/jira/browse/HDFS-5031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinay updated HDFS-5031: Attachment: HDFS-5031.patch Attached the patch BlockScanner scans the block multiple times and on restart scans everything --- Key: HDFS-5031 URL: https://issues.apache.org/jira/browse/HDFS-5031 Project: Hadoop HDFS Issue Type: Bug Components: datanode Affects Versions: 3.0.0, 2.1.0-beta Reporter: Vinay Assignee: Vinay Priority: Blocker Attachments: HDFS-5031.patch BlockScanner scans the block twice, also on restart of datanode scans everything. Steps: 1. Write blocks with interval of more than 5 seconds. write new block on completion of scan for written block. Each time datanode scans new block, it also scans, previous block which is already scanned. Now after restart, datanode scans all blocks again. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
