[jira] [Updated] (HDFS-5732) Separate memory space between BM and NN
[ https://issues.apache.org/jira/browse/HDFS-5732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Bortnikov updated HDFS-5732: --- Attachment: Remote BM.pdf Updated design of the remote BM operation. Separate memory space between BM and NN --- Key: HDFS-5732 URL: https://issues.apache.org/jira/browse/HDFS-5732 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Reporter: Amir Langer Attachments: 0002-Separation-of-BM-from-NN-Step-2-Separate-memory-spac.patch, Remote BM.pdf Change created APIs to not rely on the same instance being shared in both BM and NN. Use immutable objects / keep state in sync. BM and NN will still exist in the same VM work on a new BM service as an independent process is deferred to later tasks. Also, a one to one relation between BM and NN is assumed. This task should maintain backward compatibility. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HDFS-5732) Separate memory space between BM and NN
[ https://issues.apache.org/jira/browse/HDFS-5732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amir Langer updated HDFS-5732: -- Attachment: 0002-Separation-of-BM-from-NN-Step-2-Separate-memory-spac.patch incremental patch after patch 0001 of sub task 1 (HDFS-5731) Separate memory space between BM and NN --- Key: HDFS-5732 URL: https://issues.apache.org/jira/browse/HDFS-5732 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Reporter: Amir Langer Attachments: 0002-Separation-of-BM-from-NN-Step-2-Separate-memory-spac.patch Change created APIs to not rely on the same instance being shared in both BM and NN. Use immutable objects / keep state in sync. BM and NN will still exist in the same VM work on a new BM service as an independent process is deferred to later tasks. Also, a one to one relation between BM and NN is assumed. This task should maintain backward compatibility. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HDFS-5732) Separate memory space between BM and NN
[ https://issues.apache.org/jira/browse/HDFS-5732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amir Langer updated HDFS-5732: -- Attachment: (was: 0002-Separation-of-BM-from-NN-Step-2-Separate-memory-spac.patch) Separate memory space between BM and NN --- Key: HDFS-5732 URL: https://issues.apache.org/jira/browse/HDFS-5732 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Reporter: Amir Langer Change created APIs to not rely on the same instance being shared in both BM and NN. Use immutable objects / keep state in sync. BM and NN will still exist in the same VM work on a new BM service as an independent process is deferred to later tasks. Also, a one to one relation between BM and NN is assumed. This task should maintain backward compatibility. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HDFS-5732) Separate memory space between BM and NN
[ https://issues.apache.org/jira/browse/HDFS-5732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amir Langer updated HDFS-5732: -- Attachment: 0002-Separation-of-BM-from-NN-Step-2-Separate-memory-spac.patch Separation of memory space between BM and NN. This is an incremental patch on top of the patch for the first sub task (HDFS-5731) NN=NameNode BM=BlockManager DN=DataNode This patch mainly separates the block instances between BM and NN. This means that Inodes contain different BlockInfo instances from the BlocksMap. The replications information is only maintained by the instances in the BlocksMap - and those no longer contain a reference to the Inode but the id instead. (The new object is called BlockReplicationInfo). Improved API between BM and NN to pass more ids instead of references between the two (Although not complete, still work to invest there). Also - Separation of memory space meant there are a few upcalls - calls where the BM calls the NN. Some of these are normal (when the call originated from the DN and changed the state of the block). But some should be eliminated as they create a circular call NN-BM-NN. Currently not a problem (and was always there) - but will become problematic when we try and separate BM to its own process. The next sub-task should deal with those. Separate memory space between BM and NN --- Key: HDFS-5732 URL: https://issues.apache.org/jira/browse/HDFS-5732 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Reporter: Amir Langer Attachments: 0002-Separation-of-BM-from-NN-Step-2-Separate-memory-spac.patch Change created APIs to not rely on the same instance being shared in both BM and NN. Use immutable objects / keep state in sync. BM and NN will still exist in the same VM work on a new BM service as an independent process is deferred to later tasks. Also, a one to one relation between BM and NN is assumed. This task should maintain backward compatibility. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
