[jira] [Updated] (HDFS-3765) Namenode INITIALIZESHAREDEDITS should be able to initialize all shared storages
[ https://issues.apache.org/jira/browse/HDFS-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated HDFS-3765: -- Attachment: hdfs-3765-branch-2.txt Attaching branch-2 patch. It's the same except for some resolved conflicts on the imports. Will commit both patches momentarily. Thanks, Vinay. Namenode INITIALIZESHAREDEDITS should be able to initialize all shared storages --- Key: HDFS-3765 URL: https://issues.apache.org/jira/browse/HDFS-3765 Project: Hadoop HDFS Issue Type: Improvement Components: ha Affects Versions: 2.1.0-alpha, 3.0.0 Reporter: Vinay Assignee: Vinay Attachments: hdfs-3765-branch-2.txt, HDFS-3765.patch, HDFS-3765.patch, HDFS-3765.patch, hdfs-3765.txt, hdfs-3765.txt, hdfs-3765.txt Currently, NameNode INITIALIZESHAREDEDITS provides ability to copy the edits files to file schema based shared storages when moving cluster from Non-HA environment to HA enabled environment. This Jira focuses on the following * Generalizing the logic of copying the edits to new shared storage so that any schema based shared storage can initialized for HA cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3765) Namenode INITIALIZESHAREDEDITS should be able to initialize all shared storages
[ https://issues.apache.org/jira/browse/HDFS-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated HDFS-3765: -- Attachment: hdfs-3765.txt Trying patch upload again... this applies clean on trunk for me. Namenode INITIALIZESHAREDEDITS should be able to initialize all shared storages --- Key: HDFS-3765 URL: https://issues.apache.org/jira/browse/HDFS-3765 Project: Hadoop HDFS Issue Type: Improvement Components: ha Affects Versions: 2.1.0-alpha, 3.0.0 Reporter: Vinay Assignee: Vinay Attachments: HDFS-3765.patch, HDFS-3765.patch, HDFS-3765.patch, hdfs-3765.txt, hdfs-3765.txt, hdfs-3765.txt Currently, NameNode INITIALIZESHAREDEDITS provides ability to copy the edits files to file schema based shared storages when moving cluster from Non-HA environment to HA enabled environment. This Jira focuses on the following * Generalizing the logic of copying the edits to new shared storage so that any schema based shared storage can initialized for HA cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3765) Namenode INITIALIZESHAREDEDITS should be able to initialize all shared storages
[ https://issues.apache.org/jira/browse/HDFS-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinay updated HDFS-3765: Attachment: HDFS-3765.patch Attaching the patch with fixed comments Namenode INITIALIZESHAREDEDITS should be able to initialize all shared storages --- Key: HDFS-3765 URL: https://issues.apache.org/jira/browse/HDFS-3765 Project: Hadoop HDFS Issue Type: Improvement Components: ha Affects Versions: 2.1.0-alpha, 3.0.0 Reporter: Vinay Assignee: Vinay Attachments: HDFS-3765.patch, HDFS-3765.patch, HDFS-3765.patch, hdfs-3765.txt Currently, NameNode INITIALIZESHAREDEDITS provides ability to copy the edits files to file schema based shared storages when moving cluster from Non-HA environment to HA enabled environment. This Jira focuses on the following * Generalizing the logic of copying the edits to new shared storage so that any schema based shared storage can initialized for HA cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3765) Namenode INITIALIZESHAREDEDITS should be able to initialize all shared storages
[ https://issues.apache.org/jira/browse/HDFS-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated HDFS-3765: -- Attachment: hdfs-3765.txt New rev based on Vinay's, with a few more fixes: - include the reset method for ExitUtil so the test can pass - dont allow InitializeSharedEdits to run when no shared edits are configured - fix for initializeSharedEdits to work when checkpoints are not on segment boundaries -- the previous assertion there wasn't correct - add debug/trace logging, which was helpful for debugging the above - fix a bug where an extra END_LOG_SEGMENT would get inserted when the source logs had an END_LOG_SEGMENT - call recoverClosedStreams() before writing to shared edits - necessary for QJM, since the recovery phase has to run before a write phase - unset the shared edits in the source image - otherwise it will fail to read from an unformatted journal for some JMs I've been testing this with QJM and it seems to work well. Namenode INITIALIZESHAREDEDITS should be able to initialize all shared storages --- Key: HDFS-3765 URL: https://issues.apache.org/jira/browse/HDFS-3765 Project: Hadoop HDFS Issue Type: Improvement Components: ha Affects Versions: 2.1.0-alpha, 3.0.0 Reporter: Vinay Assignee: Vinay Attachments: HDFS-3765.patch, HDFS-3765.patch, HDFS-3765.patch, hdfs-3765.txt, hdfs-3765.txt Currently, NameNode INITIALIZESHAREDEDITS provides ability to copy the edits files to file schema based shared storages when moving cluster from Non-HA environment to HA enabled environment. This Jira focuses on the following * Generalizing the logic of copying the edits to new shared storage so that any schema based shared storage can initialized for HA cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3765) Namenode INITIALIZESHAREDEDITS should be able to initialize all shared storages
[ https://issues.apache.org/jira/browse/HDFS-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinay updated HDFS-3765: Description: Currently, NameNode INITIALIZESHAREDEDITS provides ability to copy the edits files to file schema based shared storages when moving cluster from Non-HA environment to HA enabled environment. This Jira focuses on the following * Generalizing the logic of copying the edits to new shared storage so that any schema based shared storage can initialized for HA cluster. was: Currently, NameNode INITIALIZESHAREDEDITS provides ability to copy the edits files to file schema based shared storages when moving cluster from Non-HA environment to HA enabled environment. This Jira focuses on the following * Generalizing the logic of copying the edits to new shared storage so that any schema based shared storage can initialized for HA cluster. * Ability to Initialize new shared storage from existing shared storage when moving from One shared storage to another shared storage (Might be because of cost, performance, etc. For ex: Moving from NFS to BKJM/QJM). Namenode INITIALIZESHAREDEDITS should be able to initialize all shared storages --- Key: HDFS-3765 URL: https://issues.apache.org/jira/browse/HDFS-3765 Project: Hadoop HDFS Issue Type: Improvement Components: ha Affects Versions: 2.1.0-alpha, 3.0.0 Reporter: Vinay Assignee: Vinay Attachments: HDFS-3765.patch, HDFS-3765.patch Currently, NameNode INITIALIZESHAREDEDITS provides ability to copy the edits files to file schema based shared storages when moving cluster from Non-HA environment to HA enabled environment. This Jira focuses on the following * Generalizing the logic of copying the edits to new shared storage so that any schema based shared storage can initialized for HA cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3765) Namenode INITIALIZESHAREDEDITS should be able to initialize all shared storages
[ https://issues.apache.org/jira/browse/HDFS-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinay updated HDFS-3765: Attachment: HDFS-3765.patch Attaching the patch with comments fixed. Namenode INITIALIZESHAREDEDITS should be able to initialize all shared storages --- Key: HDFS-3765 URL: https://issues.apache.org/jira/browse/HDFS-3765 Project: Hadoop HDFS Issue Type: Improvement Components: ha Affects Versions: 2.1.0-alpha, 3.0.0 Reporter: Vinay Assignee: Vinay Attachments: HDFS-3765.patch, HDFS-3765.patch Currently, NameNode INITIALIZESHAREDEDITS provides ability to copy the edits files to file schema based shared storages when moving cluster from Non-HA environment to HA enabled environment. This Jira focuses on the following * Generalizing the logic of copying the edits to new shared storage so that any schema based shared storage can initialized for HA cluster. * Ability to Initialize new shared storage from existing shared storage when moving from One shared storage to another shared storage (Might be because of cost, performance, etc. For ex: Moving from NFS to BKJM/QJM). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3765) Namenode INITIALIZESHAREDEDITS should be able to initialize all shared storages
[ https://issues.apache.org/jira/browse/HDFS-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinay updated HDFS-3765: Status: Patch Available (was: Open) Namenode INITIALIZESHAREDEDITS should be able to initialize all shared storages --- Key: HDFS-3765 URL: https://issues.apache.org/jira/browse/HDFS-3765 Project: Hadoop HDFS Issue Type: Improvement Components: ha Affects Versions: 2.1.0-alpha, 3.0.0 Reporter: Vinay Assignee: Vinay Attachments: HDFS-3765.patch, HDFS-3765.patch Currently, NameNode INITIALIZESHAREDEDITS provides ability to copy the edits files to file schema based shared storages when moving cluster from Non-HA environment to HA enabled environment. This Jira focuses on the following * Generalizing the logic of copying the edits to new shared storage so that any schema based shared storage can initialized for HA cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3765) Namenode INITIALIZESHAREDEDITS should be able to initialize all shared storages
[ https://issues.apache.org/jira/browse/HDFS-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated HDFS-3765: -- Attachment: hdfs-3765.txt here's a rev with some of the above changes. Also fixed TestBookKeeperAsHASharedDir to properly reset the ExitException in a @Before method. Namenode INITIALIZESHAREDEDITS should be able to initialize all shared storages --- Key: HDFS-3765 URL: https://issues.apache.org/jira/browse/HDFS-3765 Project: Hadoop HDFS Issue Type: Improvement Components: ha Affects Versions: 2.1.0-alpha, 3.0.0 Reporter: Vinay Assignee: Vinay Attachments: HDFS-3765.patch, HDFS-3765.patch, hdfs-3765.txt Currently, NameNode INITIALIZESHAREDEDITS provides ability to copy the edits files to file schema based shared storages when moving cluster from Non-HA environment to HA enabled environment. This Jira focuses on the following * Generalizing the logic of copying the edits to new shared storage so that any schema based shared storage can initialized for HA cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3765) Namenode INITIALIZESHAREDEDITS should be able to initialize all shared storages
[ https://issues.apache.org/jira/browse/HDFS-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinay updated HDFS-3765: Attachment: HDFS-3765.patch Attached the patch for the same. Namenode INITIALIZESHAREDEDITS should be able to initialize all shared storages --- Key: HDFS-3765 URL: https://issues.apache.org/jira/browse/HDFS-3765 Project: Hadoop HDFS Issue Type: Improvement Components: ha Affects Versions: 2.1.0-alpha, 3.0.0 Reporter: Vinay Assignee: Vinay Attachments: HDFS-3765.patch Currently, NameNode INITIALIZESHAREDEDITS provides ability to copy the edits files to file schema based shared storages when moving cluster from Non-HA environment to HA enabled environment. This Jira focuses on the following * Generalizing the logic of copying the edits to new shared storage so that any schema based shared storage can initialized for HA cluster. * Ability to Initialize new shared storage from existing shared storage when moving from One shared storage to another shared storage (Might be because of cost, performance, etc. For ex: Moving from NFS to BKJM/QJM). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
