[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14380725#comment-14380725
]
Daryn Sharp commented on HDFS-6658:
---
bq. It would be nice if this could help with the goa
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14380666#comment-14380666
]
Daryn Sharp commented on HDFS-6658:
---
bq. Why did you roll your own bitset instead of usi
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376532#comment-14376532
]
Charles Lamb commented on HDFS-6658:
[~daryn],
I spent a couple of hours making a firs
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14372508#comment-14372508
]
Colin Patrick McCabe commented on HDFS-6658:
Daryn, I apologize for not being m
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14371612#comment-14371612
]
Daryn Sharp commented on HDFS-6658:
---
Thanks for your comments! The goal is significant m
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14368324#comment-14368324
]
Colin Patrick McCabe commented on HDFS-6658:
I didn't get a chance to look at t
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14367179#comment-14367179
]
Daryn Sharp commented on HDFS-6658:
---
Would anyone comment on the approach? We are eager
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14359021#comment-14359021
]
Daryn Sharp commented on HDFS-6658:
---
I should mention I'll split this patch into as many
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14357672#comment-14357672
]
Charles Lamb commented on HDFS-6658:
Hi [~daryn],
Colin and I read over the design doc
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14357683#comment-14357683
]
Daryn Sharp commented on HDFS-6658:
---
I've added many preconditions, not asserts, to avoid
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14357661#comment-14357661
]
Hadoop QA commented on HDFS-6658:
-
{color:red}-1 overall{color}. Here are the results of t
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14357635#comment-14357635
]
Hadoop QA commented on HDFS-6658:
-
{color:red}-1 overall{color}. Here are the results of t
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14356996#comment-14356996
]
Hadoop QA commented on HDFS-6658:
-
{color:red}-1 overall{color}. Here are the results of t
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14356162#comment-14356162
]
Hadoop QA commented on HDFS-6658:
-
{color:red}-1 overall{color}. Here are the results of t
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14355932#comment-14355932
]
Daryn Sharp commented on HDFS-6658:
---
Sorry, didn't see your reply, but great minds think
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14350936#comment-14350936
]
Colin Patrick McCabe commented on HDFS-6658:
[~clamb] and I have been discussin
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14349489#comment-14349489
]
Daryn Sharp commented on HDFS-6658:
---
No worries, discussion is good. Will be even better
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14345572#comment-14345572
]
Todd Lipcon commented on HDFS-6658:
---
{quote}
1 the balancer
2 datanode manager start/stop
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14345360#comment-14345360
]
Daryn Sharp commented on HDFS-6658:
---
Great questions, Todd. Both are initial goals that
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14343530#comment-14343530
]
Todd Lipcon commented on HDFS-6658:
---
Hey Daryn... jumping in unsolicited with a random th
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14340483#comment-14340483
]
Hadoop QA commented on HDFS-6658:
-
{color:red}-1 overall{color}. Here are the results of t
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14340444#comment-14340444
]
Hadoop QA commented on HDFS-6658:
-
{color:red}-1 overall{color}. Here are the results of t
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14180682#comment-14180682
]
Konstantin Shvachko commented on HDFS-6658:
---
I agree usually people remove data i
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14178078#comment-14178078
]
Amir Langer commented on HDFS-6658:
---
Thanks [~shv]
I agree that the scenario of a remove
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14175788#comment-14175788
]
Konstantin Shvachko commented on HDFS-6658:
---
Pretty neat data structure, Amir. Co
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14128533#comment-14128533
]
Hadoop QA commented on HDFS-6658:
-
{color:red}-1 overall{color}. Here are the results of t
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14119041#comment-14119041
]
Colin Patrick McCabe commented on HDFS-6658:
I didn't realize the patches were
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14113773#comment-14113773
]
Edward Bortnikov commented on HDFS-6658:
Looks convincing Amir! Seems that everybod
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14075574#comment-14075574
]
Amir Langer commented on HDFS-6658:
---
Re [~drankye]: Using on or off heap memory is an or
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14070028#comment-14070028
]
Kai Zheng commented on HDFS-6658:
-
So would it be better to put the BlockList stuffs off he
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14067231#comment-14067231
]
Colin Patrick McCabe commented on HDFS-6658:
bq. Separately, I'll open a JIRA a
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14064241#comment-14064241
]
Colin Patrick McCabe commented on HDFS-6658:
bq. I'm not totally convinced of t
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14064095#comment-14064095
]
Nathan Roberts commented on HDFS-6658:
--
{quote}
I guess my argument is that (in the sh
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14063913#comment-14063913
]
Colin Patrick McCabe commented on HDFS-6658:
bq. What you suggesting (if I unde
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14063077#comment-14063077
]
Amir Langer commented on HDFS-6658:
---
[~ cmccabe] When I looked at off heap I was using th
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14063072#comment-14063072
]
Amir Langer commented on HDFS-6658:
---
~szetszwo - yes I thought about using just one and c
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14062648#comment-14062648
]
Tsz Wo Nicholas Sze commented on HDFS-6658:
---
Let me try it in HDFS-5464.
> Namen
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14062593#comment-14062593
]
Tsz Wo Nicholas Sze commented on HDFS-6658:
---
Nice idea!
For processing block rep
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14062514#comment-14062514
]
Colin Patrick McCabe commented on HDFS-6658:
bq. If we don't have caching, we n
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14062498#comment-14062498
]
Amir Langer commented on HDFS-6658:
---
We explored the idea of off-heap memory for the Name
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14061381#comment-14061381
]
Colin Patrick McCabe commented on HDFS-6658:
I think that we need to take a mor
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14061374#comment-14061374
]
Colin Patrick McCabe commented on HDFS-6658:
bq. As opposed to block-management
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14061213#comment-14061213
]
Edward Bortnikov commented on HDFS-6658:
As opposed to block-management-as-a-servic
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14061069#comment-14061069
]
Colin Patrick McCabe commented on HDFS-6658:
2**31 (2 billion) possible replica
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060740#comment-14060740
]
Nathan Roberts commented on HDFS-6658:
--
Maybe have a simple fragmentation metric and i
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060023#comment-14060023
]
Amir Langer commented on HDFS-6658:
---
Hi [~kihwal] - In response to the scenario of massiv
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14057858#comment-14057858
]
Nathan Roberts commented on HDFS-6658:
--
Thanks Amir. Very attractive memory savings.
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14057619#comment-14057619
]
Kihwal Lee commented on HDFS-6658:
--
Is there any way to compact or free a set of sparse bl
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14057575#comment-14057575
]
Kihwal Lee commented on HDFS-6658:
--
[~langera], I've added you as contributor and assigned
[
https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14057385#comment-14057385
]
Amir Langer commented on HDFS-6658:
---
Attached patches for every sub-task.
> Namenode mem
50 matches
Mail list logo
