[jira] [Updated] (HDFS-6659) Create a Block List
[ https://issues.apache.org/jira/browse/HDFS-6659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-6659: - Labels: perfomance (was: BB2015-05-TBR perfomance) > Create a Block List > --- > > Key: HDFS-6659 > URL: https://issues.apache.org/jira/browse/HDFS-6659 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: namenode >Affects Versions: 2.4.1 >Reporter: Amir Langer >Assignee: Amir Langer > Labels: perfomance > Attachments: HDFS-6659.patch > > > BlockList - An efficient array based list that can extend its capacity with > two main features: > 1. Gaps (result of remove operations) are managed internally without the need > for extra memory - We create a linked list of gaps by using the array index > as references + An int to the head of the gaps list. In every insert > operation, we first use any available gap before extending the array. > 2. Array extension is done by chaining different arrays, not by allocating a > larger array and copying all its data across. This is a lot less heavy in > terms of latency for that particular call. It also avoids having large amount > of contiguous heap space and so behaves nicer with garbage collection. > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-6659) Create a Block List
[ https://issues.apache.org/jira/browse/HDFS-6659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-6659: - Target Version/s: (was: 2.8.0) > Create a Block List > --- > > Key: HDFS-6659 > URL: https://issues.apache.org/jira/browse/HDFS-6659 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: namenode >Affects Versions: 2.4.1 >Reporter: Amir Langer >Assignee: Amir Langer > Labels: perfomance > Attachments: HDFS-6659.patch > > > BlockList - An efficient array based list that can extend its capacity with > two main features: > 1. Gaps (result of remove operations) are managed internally without the need > for extra memory - We create a linked list of gaps by using the array index > as references + An int to the head of the gaps list. In every insert > operation, we first use any available gap before extending the array. > 2. Array extension is done by chaining different arrays, not by allocating a > larger array and copying all its data across. This is a lot less heavy in > terms of latency for that particular call. It also avoids having large amount > of contiguous heap space and so behaves nicer with garbage collection. > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-6659) Create a Block List
[ https://issues.apache.org/jira/browse/HDFS-6659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HDFS-6659: --- Labels: BB2015-05-TBR perfomance (was: perfomance) Create a Block List --- Key: HDFS-6659 URL: https://issues.apache.org/jira/browse/HDFS-6659 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Affects Versions: 2.4.1 Reporter: Amir Langer Assignee: Amir Langer Labels: BB2015-05-TBR, perfomance Attachments: HDFS-6659.patch BlockList - An efficient array based list that can extend its capacity with two main features: 1. Gaps (result of remove operations) are managed internally without the need for extra memory - We create a linked list of gaps by using the array index as references + An int to the head of the gaps list. In every insert operation, we first use any available gap before extending the array. 2. Array extension is done by chaining different arrays, not by allocating a larger array and copying all its data across. This is a lot less heavy in terms of latency for that particular call. It also avoids having large amount of contiguous heap space and so behaves nicer with garbage collection. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-6659) Create a Block List
[ https://issues.apache.org/jira/browse/HDFS-6659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amir Langer updated HDFS-6659: -- Labels: perfomance (was: ) Status: Patch Available (was: Open) Create a Block List --- Key: HDFS-6659 URL: https://issues.apache.org/jira/browse/HDFS-6659 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Affects Versions: 2.4.1 Reporter: Amir Langer Assignee: Amir Langer Labels: perfomance Attachments: 0001-BlockList-a-list-of-Blocks-that-saves-memory-by-mana.patch BlockList - An efficient array based list that can extend its capacity with two main features: 1. Gaps (result of remove operations) are managed internally without the need for extra memory - We create a linked list of gaps by using the array index as references + An int to the head of the gaps list. In every insert operation, we first use any available gap before extending the array. 2. Array extension is done by chaining different arrays, not by allocating a larger array and copying all its data across. This is a lot less heavy in terms of latency for that particular call. It also avoids having large amount of contiguous heap space and so behaves nicer with garbage collection. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-6659) Create a Block List
[ https://issues.apache.org/jira/browse/HDFS-6659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amir Langer updated HDFS-6659: -- Attachment: HDFS-6659.patch Create a Block List --- Key: HDFS-6659 URL: https://issues.apache.org/jira/browse/HDFS-6659 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Affects Versions: 2.4.1 Reporter: Amir Langer Assignee: Amir Langer Labels: perfomance Attachments: HDFS-6659.patch BlockList - An efficient array based list that can extend its capacity with two main features: 1. Gaps (result of remove operations) are managed internally without the need for extra memory - We create a linked list of gaps by using the array index as references + An int to the head of the gaps list. In every insert operation, we first use any available gap before extending the array. 2. Array extension is done by chaining different arrays, not by allocating a larger array and copying all its data across. This is a lot less heavy in terms of latency for that particular call. It also avoids having large amount of contiguous heap space and so behaves nicer with garbage collection. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-6659) Create a Block List
[ https://issues.apache.org/jira/browse/HDFS-6659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amir Langer updated HDFS-6659: -- Attachment: (was: 0001-BlockList-a-list-of-Blocks-that-saves-memory-by-mana.patch) Create a Block List --- Key: HDFS-6659 URL: https://issues.apache.org/jira/browse/HDFS-6659 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Affects Versions: 2.4.1 Reporter: Amir Langer Assignee: Amir Langer Labels: perfomance Attachments: HDFS-6659.patch BlockList - An efficient array based list that can extend its capacity with two main features: 1. Gaps (result of remove operations) are managed internally without the need for extra memory - We create a linked list of gaps by using the array index as references + An int to the head of the gaps list. In every insert operation, we first use any available gap before extending the array. 2. Array extension is done by chaining different arrays, not by allocating a larger array and copying all its data across. This is a lot less heavy in terms of latency for that particular call. It also avoids having large amount of contiguous heap space and so behaves nicer with garbage collection. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-6659) Create a Block List
[ https://issues.apache.org/jira/browse/HDFS-6659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amir Langer updated HDFS-6659: -- Attachment: 0001-BlockList-a-list-of-Blocks-that-saves-memory-by-mana.patch Create a Block List --- Key: HDFS-6659 URL: https://issues.apache.org/jira/browse/HDFS-6659 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Affects Versions: 2.4.1 Reporter: Amir Langer Fix For: 3.0.0, 2.5.0 Attachments: 0001-BlockList-a-list-of-Blocks-that-saves-memory-by-mana.patch BlockList - An efficient array based list that can extend its capacity with two main features: 1. Gaps (result of remove operations) are managed internally without the need for extra memory - We create a linked list of gaps by using the array index as references + An int to the head of the gaps list. In every insert operation, we first use any available gap before extending the array. 2. Array extension is done by chaining different arrays, not by allocating a larger array and copying all its data across. This is a lot less heavy in terms of latency for that particular call. It also avoids having large amount of contiguous heap space and so behaves nicer with garbage collection. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HDFS-6659) Create a Block List
[ https://issues.apache.org/jira/browse/HDFS-6659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-6659: - Target Version/s: 2.6.0 Fix Version/s: (was: 2.5.0) (was: 3.0.0) Create a Block List --- Key: HDFS-6659 URL: https://issues.apache.org/jira/browse/HDFS-6659 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Affects Versions: 2.4.1 Reporter: Amir Langer Attachments: 0001-BlockList-a-list-of-Blocks-that-saves-memory-by-mana.patch BlockList - An efficient array based list that can extend its capacity with two main features: 1. Gaps (result of remove operations) are managed internally without the need for extra memory - We create a linked list of gaps by using the array index as references + An int to the head of the gaps list. In every insert operation, we first use any available gap before extending the array. 2. Array extension is done by chaining different arrays, not by allocating a larger array and copying all its data across. This is a lot less heavy in terms of latency for that particular call. It also avoids having large amount of contiguous heap space and so behaves nicer with garbage collection. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HDFS-6659) Create a Block List
[ https://issues.apache.org/jira/browse/HDFS-6659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-6659: - Assignee: Amir Langer Create a Block List --- Key: HDFS-6659 URL: https://issues.apache.org/jira/browse/HDFS-6659 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Affects Versions: 2.4.1 Reporter: Amir Langer Assignee: Amir Langer Attachments: 0001-BlockList-a-list-of-Blocks-that-saves-memory-by-mana.patch BlockList - An efficient array based list that can extend its capacity with two main features: 1. Gaps (result of remove operations) are managed internally without the need for extra memory - We create a linked list of gaps by using the array index as references + An int to the head of the gaps list. In every insert operation, we first use any available gap before extending the array. 2. Array extension is done by chaining different arrays, not by allocating a larger array and copying all its data across. This is a lot less heavy in terms of latency for that particular call. It also avoids having large amount of contiguous heap space and so behaves nicer with garbage collection. -- This message was sent by Atlassian JIRA (v6.2#6252)
