[jira] [Commented] (HDFS-3092) Enable journal protocol based editlog streaming for standby namenode

2012-04-16 Thread dhruba borthakur (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254889#comment-13254889 ] dhruba borthakur commented on HDFS-3092: I still somehow like the BK approach, espe

[jira] [Commented] (HDFS-3051) A zero-copy ScatterGatherRead api from FSDataInputStream

2012-03-23 Thread dhruba borthakur (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13237185#comment-13237185 ] dhruba borthakur commented on HDFS-3051: Hi colin, I agree that scatter/gather typi

[jira] [Commented] (HDFS-3051) A zero-copy read api from FSDataInputStream

2012-03-06 Thread dhruba borthakur (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223096#comment-13223096 ] dhruba borthakur commented on HDFS-3051: The new API can be a new method in FSData

[jira] [Commented] (HDFS-2834) ByteBuffer-based read API for DFSInputStream

2012-03-06 Thread dhruba borthakur (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223093#comment-13223093 ] dhruba borthakur commented on HDFS-2834: Agreed Todd. Instead of subverting this ji

[jira] [Commented] (HDFS-2834) ByteBuffer-based read API for DFSInputStream

2012-03-05 Thread dhruba borthakur (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223023#comment-13223023 ] dhruba borthakur commented on HDFS-2834: In our own variant of hdfs, we have introd

[jira] [Commented] (HDFS-1623) High Availability Framework for HDFS NN

2012-02-15 Thread dhruba borthakur (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13208599#comment-13208599 ] dhruba borthakur commented on HDFS-1623: > I think for the first cut we should disa

[jira] [Commented] (HDFS-2791) If block report races with closing of file, replica is incorrectly marked corrupt

2012-01-24 Thread dhruba borthakur (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13192800#comment-13192800 ] dhruba borthakur commented on HDFS-2791: +1 > If block report rac

[jira] [Commented] (HDFS-2791) If block report races with closing of file, replica is incorrectly marked corrupt

2012-01-24 Thread dhruba borthakur (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13192340#comment-13192340 ] dhruba borthakur commented on HDFS-2791: > Case 1) if the block has a too-low gener

[jira] [Commented] (HDFS-2691) HA: Tests and fixes for pipeline targets and replica recovery

2012-01-20 Thread dhruba borthakur (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13190339#comment-13190339 ] dhruba borthakur commented on HDFS-2691: In our implementation, we have implemented

[jira] [Commented] (HDFS-2744) Extend FSDataInputStream to allow fadvise

2012-01-03 Thread dhruba borthakur (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13179012#comment-13179012 ] dhruba borthakur commented on HDFS-2744: {code} /** * Tells the system to not

[jira] [Commented] (HDFS-140) When a file is deleted, its blocks remain in the blocksmap till the next block report from Datanode

2011-12-27 Thread dhruba borthakur (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13176461#comment-13176461 ] dhruba borthakur commented on HDFS-140: --- hi uma, your technical point makes sense, but

[jira] [Commented] (HDFS-2699) Store data and checksums together in block file

2011-12-19 Thread dhruba borthakur (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13172804#comment-13172804 ] dhruba borthakur commented on HDFS-2699: Suresh: it provides great relief for an hb

[jira] [Commented] (HDFS-2699) Store data and checksums together in block file

2011-12-18 Thread dhruba borthakur (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13171976#comment-13171976 ] dhruba borthakur commented on HDFS-2699: Thanks srivas for your comments. > a bloc

[jira] [Commented] (HDFS-2699) Store data and checksums together in block file

2011-12-18 Thread dhruba borthakur (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13171951#comment-13171951 ] dhruba borthakur commented on HDFS-2699: Thanks for your comments Scott, Andrew, To

[jira] [Commented] (HDFS-2699) Store data and checksums together in block file

2011-12-17 Thread dhruba borthakur (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13171653#comment-13171653 ] dhruba borthakur commented on HDFS-2699: Hi andrew, all of the points you mentioned

[jira] [Commented] (HDFS-2699) Store data and checksums together in block file

2011-12-17 Thread dhruba borthakur (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13171604#comment-13171604 ] dhruba borthakur commented on HDFS-2699: Another option that I am going to try is t

[jira] [Commented] (HDFS-2699) Store data and checksums together in block file

2011-12-17 Thread dhruba borthakur (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13171601#comment-13171601 ] dhruba borthakur commented on HDFS-2699: There are various alternatie like you prop

[jira] [Commented] (HDFS-2699) Store data and checksums together in block file

2011-12-17 Thread dhruba borthakur (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13171493#comment-13171493 ] dhruba borthakur commented on HDFS-2699: The number of random reads issued by HBase

[jira] [Commented] (HDFS-1972) HA: Datanode fencing mechanism

2011-12-13 Thread dhruba borthakur (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169072#comment-13169072 ] dhruba borthakur commented on HDFS-1972: > would it be possible to defer block repl

[jira] [Commented] (HDFS-1972) HA: Datanode fencing mechanism

2011-12-13 Thread dhruba borthakur (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13168624#comment-13168624 ] dhruba borthakur commented on HDFS-1972: findOverReplicatedReplicas runs with the r

[jira] [Commented] (HDFS-2246) Shortcut a local client reads to a Datanodes files directly

2011-12-09 Thread dhruba borthakur (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13166520#comment-13166520 ] dhruba borthakur commented on HDFS-2246: Hi JD, I agree that this is a scenario tha

[jira] [Commented] (HDFS-233) Support for snapshots

2011-12-08 Thread dhruba borthakur (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165901#comment-13165901 ] dhruba borthakur commented on HDFS-233: --- Hi Matt, without a design approval, the work

[jira] [Commented] (HDFS-1972) HA: Datanode fencing mechanism

2011-12-01 Thread dhruba borthakur (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161329#comment-13161329 ] dhruba borthakur commented on HDFS-1972: I recollect that we prevent primary nameno

[jira] [Commented] (HDFS-2601) Proposal to store edits and checkpoints inside HDFS itself for namenode HA

2011-11-29 Thread dhruba borthakur (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159853#comment-13159853 ] dhruba borthakur commented on HDFS-2601: Can you pl detail the high-level changes n

[jira] [Commented] (HDFS-2246) Shortcut a local client reads to a Datanodes files directly

2011-11-05 Thread dhruba borthakur (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144766#comment-13144766 ] dhruba borthakur commented on HDFS-2246: > synchronized LinkedHashMap Why not a co

[jira] [Commented] (HDFS-140) When a file is deleted, its blocks remain in the blocksmap till the next block report from Datanode

2011-10-12 Thread dhruba borthakur (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13126136#comment-13126136 ] dhruba borthakur commented on HDFS-140: --- I would like to agree with Todd too. Uma: do

[jira] [Commented] (HDFS-2246) Shortcut a local client reads to a Datanodes files directly

2011-10-03 Thread dhruba borthakur (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13119411#comment-13119411 ] dhruba borthakur commented on HDFS-2246: Hi Milind/Sanjay, I was assuming that this