[jira] [Commented] (HDFS-3004) Implement Recovery Mode

2012-03-09 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13226583#comment-13226583 ] Eli Collins commented on HDFS-3004: --- HDFS-3004.008.patch has a bunch of other stuff in it

[jira] [Commented] (HDFS-3066) cap space usage of default log4j rolling policy (hdfs specific changes)

2012-03-09 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13226399#comment-13226399 ] Eli Collins commented on HDFS-3066: --- +1 pending jenkins > cap space usag

[jira] [Commented] (HDFS-2303) jsvc needs to be recompilable

2012-03-09 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13226395#comment-13226395 ] Eli Collins commented on HDFS-2303: --- +1 Latest patch (HDFS-2303-5-trunk.patch) looks gre

[jira] [Commented] (HDFS-3004) Implement Recovery Mode

2012-03-09 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13226378#comment-13226378 ] Eli Collins commented on HDFS-3004: --- Your comments above make sense, thanks for the expla

[jira] [Commented] (HDFS-1623) High Availability Framework for HDFS NN

2012-03-09 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13226273#comment-13226273 ] Eli Collins commented on HDFS-1623: --- +1 branch 23 patch looks good to me

[jira] [Commented] (HDFS-2303) jsvc needs to be recompilable

2012-03-08 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225327#comment-13225327 ] Eli Collins commented on HDFS-2303: --- Mingjie, - Have you verified you can set JSVC_HOME

[jira] [Commented] (HDFS-2303) jsvc needs to be recompilable

2012-03-07 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13224940#comment-13224940 ] Eli Collins commented on HDFS-2303: --- Sounds good, it's only for the error message since w

[jira] [Commented] (HDFS-2288) Replicas awaiting recovery should return a full visible length

2012-03-07 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13224814#comment-13224814 ] Eli Collins commented on HDFS-2288: --- Ah, never mind this is the TestLogRolling#testLogRol

[jira] [Commented] (HDFS-2288) Replicas awaiting recovery should return a full visible length

2012-03-07 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13224813#comment-13224813 ] Eli Collins commented on HDFS-2288: --- Hey Todd, I think you're right, there's a bug in th

[jira] [Commented] (HDFS-2303) jsvc needs to be recompilable

2012-03-07 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13224764#comment-13224764 ] Eli Collins commented on HDFS-2303: --- Agree w Tucu, Mingjie, Roman, ATM, on the approach.

[jira] [Commented] (HDFS-3004) Implement Recovery Mode

2012-03-07 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13224698#comment-13224698 ] Eli Collins commented on HDFS-3004: --- Overall approach looks good. - Wr edit logs in the

[jira] [Commented] (HDFS-2872) Add sanity checks during edits loading that generation stamps are non-decreasing

2012-03-07 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13224567#comment-13224567 ] Eli Collins commented on HDFS-2872: --- +1 looks good. I ran a build and reloaded the edits

[jira] [Commented] (HDFS-2872) Add sanity checks during edits loading that generation stamps are non-decreasing

2012-03-06 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223759#comment-13223759 ] Eli Collins commented on HDFS-2872: --- Some nits, otherwise looks great. - I'd throw IOE i

[jira] [Commented] (HDFS-3048) Small race in BlockManager#close

2012-03-05 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222585#comment-13222585 ] Eli Collins commented on HDFS-3048: --- Here's an example NPE seen in a unit test. {noforma

[jira] [Commented] (HDFS-3035) HA: fix failure of TestFileAppendRestart due to OP_UPDATE_BLOCKS

2012-03-01 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220228#comment-13220228 ] Eli Collins commented on HDFS-3035: --- +1 looks good > HA: fix failure of

[jira] [Commented] (HDFS-3025) Automatic log sync shouldn't happen inside logEdit path

2012-02-29 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13219667#comment-13219667 ] Eli Collins commented on HDFS-3025: --- +1 looks good > Automatic log sync

[jira] [Commented] (HDFS-2979) HA: Balancer should use logical uri for creating failover proxy with HA enabled.

2012-02-29 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13219657#comment-13219657 ] Eli Collins commented on HDFS-2979: --- +1 > HA: Balancer should use logica

[jira] [Commented] (HDFS-2979) HA: Balancer should use logical uri for creating failover proxy with HA enabled.

2012-02-29 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13219622#comment-13219622 ] Eli Collins commented on HDFS-2979: --- Let's add a wrapper for getNameServiceUris, somethin

[jira] [Commented] (HDFS-3023) Optimize entries in edits log for persistBlocks calls

2012-02-29 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13219611#comment-13219611 ] Eli Collins commented on HDFS-3023: --- Hey Todd, reviewed the patch again. +1, thanks for a

[jira] [Commented] (HDFS-3027) HA: Implement a simple NN health check

2012-02-29 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13219578#comment-13219578 ] Eli Collins commented on HDFS-3027: --- +1 > HA: Implement a simple NN heal

[jira] [Commented] (HDFS-3020) Auto-logSync based on edit log buffer size broken

2012-02-29 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13219453#comment-13219453 ] Eli Collins commented on HDFS-3020: --- +1 updated patch lgtm > Auto-logSyn

[jira] [Commented] (HDFS-2979) HA: Balancer should use logical uri for creating failover proxy with HA enabled.

2012-02-29 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13219385#comment-13219385 ] Eli Collins commented on HDFS-2979: --- - The new getNNUris method seems overloaded, eg it o

[jira] [Commented] (HDFS-3004) Create Offline NameNode recovery tool

2012-02-28 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218881#comment-13218881 ] Eli Collins commented on HDFS-3004: --- Hey Colin, Nice writeup. Worth mentioning the foc

[jira] [Commented] (HDFS-2992) Edit log failure trace should include transaction ID of error

2012-02-28 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218874#comment-13218874 ] Eli Collins commented on HDFS-2992: --- Hey Colin, Looks like TestFSEditLogLoader.testDispl

[jira] [Commented] (HDFS-3027) HA: Implement a simple NN health check

2012-02-28 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218828#comment-13218828 ] Eli Collins commented on HDFS-3027: --- Looks like NNResourceChecker#hasAvailableDiskSpace d

[jira] [Commented] (HDFS-3025) Automatic log sync shouldn't happen inside logEdit path

2012-02-28 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218773#comment-13218773 ] Eli Collins commented on HDFS-3025: --- +1 pending jenkins > Automatic log

[jira] [Commented] (HDFS-3026) HA: Handle failure during HA state transition

2012-02-28 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218753#comment-13218753 ] Eli Collins commented on HDFS-3026: --- bq. bq. Wrt delayed shutdown, we likely have (or sho

[jira] [Commented] (HDFS-3027) HA: Implement a simple NN health check

2012-02-28 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218729#comment-13218729 ] Eli Collins commented on HDFS-3027: --- Yea, feel free to punt the specific message to anoth

[jira] [Commented] (HDFS-2920) HA: fix remaining TODO items

2012-02-28 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218692#comment-13218692 ] Eli Collins commented on HDFS-2920: --- +1 > HA: fix remaining TODO items >

[jira] [Commented] (HDFS-3020) Auto-logSync based on edit log buffer size broken

2012-02-28 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218453#comment-13218453 ] Eli Collins commented on HDFS-3020: --- +1 update looks good > Auto-logSync

[jira] [Commented] (HDFS-3023) Optimize entries in edits log for persistBlocks calls

2012-02-28 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218437#comment-13218437 ] Eli Collins commented on HDFS-3023: --- Thanks for the info wrt perf, makes sense. Will be i

[jira] [Commented] (HDFS-3023) Optimize entries in edits log for persistBlocks calls

2012-02-28 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218383#comment-13218383 ] Eli Collins commented on HDFS-3023: --- Thanks for the info. By option #2 vs option #3 abov

[jira] [Commented] (HDFS-3023) Optimize entries in edits log for persistBlocks calls

2012-02-28 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218382#comment-13218382 ] Eli Collins commented on HDFS-3023: --- Thanks for the info. By option #2 vs option #3 abov

[jira] [Commented] (HDFS-3023) Optimize entries in edits log for persistBlocks calls

2012-02-28 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218378#comment-13218378 ] Eli Collins commented on HDFS-3023: --- The patch looks good btw, though why does updating b

[jira] [Commented] (HDFS-3023) Optimize entries in edits log for persistBlocks calls

2012-02-28 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218365#comment-13218365 ] Eli Collins commented on HDFS-3023: --- Is log size a proxy for performance? Ie do we have

[jira] [Commented] (HDFS-2958) HA: Sweep for remaining proxy construction which doesn't go through failover path

2012-02-28 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218355#comment-13218355 ] Eli Collins commented on HDFS-2958: --- +1 looks great. This is much nicer! I tried to see i

[jira] [Commented] (HDFS-2992) Edit log failure trace should include transaction ID of error

2012-02-28 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218336#comment-13218336 ] Eli Collins commented on HDFS-2992: --- +1 looks good > Edit log failure tr

[jira] [Commented] (HDFS-2920) HA: fix remaining TODO items

2012-02-28 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218320#comment-13218320 ] Eli Collins commented on HDFS-2920: --- Hey ATM, how about putting the NN health check imple

[jira] [Commented] (HDFS-3024) Improve performance of stringification in addStoredBlock

2012-02-28 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218301#comment-13218301 ] Eli Collins commented on HDFS-3024: --- +1 looks good > Improve performance

[jira] [Commented] (HDFS-3006) Webhdfs "SETOWNER" call returns incorrect content-type

2012-02-26 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13216995#comment-13216995 ] Eli Collins commented on HDFS-3006: --- bq. - reverts the change in HttpFs since the simple

[jira] [Commented] (HDFS-3006) Webhdfs "SETOWNER" call returns incorrect content-type

2012-02-23 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13215361#comment-13215361 ] Eli Collins commented on HDFS-3006: --- Is this true of HttpFS as well? > W

[jira] [Commented] (HDFS-2995) start-dfs.sh should only start the 2NN for namenodes with dfs.namenode.secondary.http-address configured

2012-02-23 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13214938#comment-13214938 ] Eli Collins commented on HDFS-2995: --- Verified with a tarball that a 2NN is not started/st

[jira] [Commented] (HDFS-2920) HA: fix remaining TODO items

2012-02-22 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13214174#comment-13214174 ] Eli Collins commented on HDFS-2920: --- +1 looks good > HA: fix remaining T

[jira] [Commented] (HDFS-2922) HA: close out operation categories

2012-02-22 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13214080#comment-13214080 ] Eli Collins commented on HDFS-2922: --- Forgot to mention, I'm re-running the hdfs tests for

[jira] [Commented] (HDFS-2971) some improvements to the manual NN metadata recovery tools

2012-02-22 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13213932#comment-13213932 ] Eli Collins commented on HDFS-2971: --- Suresh, The highest gen stamp in an image or log te

[jira] [Commented] (HDFS-2943) Expose last checkpoint time and transaction stats as JMX metrics

2012-02-13 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207523#comment-13207523 ] Eli Collins commented on HDFS-2943: --- +1 looks great > Expose last checkp

[jira] [Commented] (HDFS-2947) HA: On startup NN throws an NPE in the metrics system

2012-02-13 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207518#comment-13207518 ] Eli Collins commented on HDFS-2947: --- +1 looks good > HA: On startup NN t

[jira] [Commented] (HDFS-2942) HA: TestActiveStandbyElectorRealZK fails if build dir does not exist

2012-02-13 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207515#comment-13207515 ] Eli Collins commented on HDFS-2942: --- +1 looks good > HA: TestActiveStand

[jira] [Commented] (HDFS-2944) Typo in hdfs-default.xml causes dfs.client.block.write.replace-datanode-on-failure.enable to be mistakenly disabled

2012-02-13 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207146#comment-13207146 ] Eli Collins commented on HDFS-2944: --- +1 > Typo in hdfs-default.xml cause

[jira] [Commented] (HDFS-2935) Shared edits dir property should be suffixed with nameservice and namenodeID

2012-02-09 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205265#comment-13205265 ] Eli Collins commented on HDFS-2935: --- Why nnId? Since the shared dir is shared it should j

[jira] [Commented] (HDFS-2878) TestBlockRecovery does not compile

2012-02-09 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205122#comment-13205122 ] Eli Collins commented on HDFS-2878: --- +1 looks good > TestBlockRecovery d

[jira] [Commented] (HDFS-2781) Add client protocol and DFSadmin for command to restore failed storage

2012-02-09 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204931#comment-13204931 ] Eli Collins commented on HDFS-2781: --- Can we define this away? Eg if a standby loses conne

[jira] [Commented] (HDFS-2781) Add client protocol and DFSadmin for command to restore failed storage

2012-02-09 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204918#comment-13204918 ] Eli Collins commented on HDFS-2781: --- If we change the behavior such that the NN drops int

[jira] [Commented] (HDFS-2917) HA: haadmin should not work if run by regular user

2012-02-09 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204809#comment-13204809 ] Eli Collins commented on HDFS-2917: --- Thank you both for the reviews. @Jitendra, filed HD

[jira] [Commented] (HDFS-2579) Starting delegation token manager during safemode fails

2012-02-08 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204293#comment-13204293 ] Eli Collins commented on HDFS-2579: --- +1 latest patch and testing looks good.

[jira] [Commented] (HDFS-2923) Namenode IPC handler count uses the wrong configuration key

2012-02-08 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204272#comment-13204272 ] Eli Collins commented on HDFS-2923: --- +1 I looked for other similar mistakes in HDFS-1763

[jira] [Commented] (HDFS-2764) TestBackupNode is racy

2012-02-07 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13203208#comment-13203208 ] Eli Collins commented on HDFS-2764: --- +1 nice find. I'd add a comment like the following

[jira] [Commented] (HDFS-2362) More Improvements on NameNode Scalability

2012-02-07 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13203121#comment-13203121 ] Eli Collins commented on HDFS-2362: --- Not for 23.1, which is getting cut soon. We'll merge

[jira] [Commented] (HDFS-2911) Gracefully handle OutOfMemoryErrors

2012-02-07 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13203064#comment-13203064 ] Eli Collins commented on HDFS-2911: --- HDFS isn't really an application. If we labor on sub

[jira] [Commented] (HDFS-2579) Starting delegation token manager during safemode fails

2012-02-07 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13202913#comment-13202913 ] Eli Collins commented on HDFS-2579: --- +1 updated patch lgtm Jitendra, if you're going to

[jira] [Commented] (HDFS-2902) Allow new shared edit logs dir to be configured while NN is running

2012-02-06 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201923#comment-13201923 ] Eli Collins commented on HDFS-2902: --- Correct, the checkpoint may be eg hourly so the admi

[jira] [Commented] (HDFS-2782) HA: Support multiple shared edits dirs

2012-02-06 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201897#comment-13201897 ] Eli Collins commented on HDFS-2782: --- There's also a variant of solution #2 where you don'

[jira] [Commented] (HDFS-2794) HA: Active NN may purge edit log files before standby NN has a chance to read them

2012-02-06 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201828#comment-13201828 ] Eli Collins commented on HDFS-2794: --- Ignore my previous comment, was on a stale page. La

[jira] [Commented] (HDFS-2794) HA: Active NN may purge edit log files before standby NN has a chance to read them

2012-02-06 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201826#comment-13201826 ] Eli Collins commented on HDFS-2794: --- Todd, new comment looks good. Per last comment the p

[jira] [Commented] (HDFS-2794) HA: Active NN may purge edit log files before standby NN has a chance to read them

2012-02-06 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201666#comment-13201666 ] Eli Collins commented on HDFS-2794: --- Nit: the new parameter in hdfs-default.xml is missin

[jira] [Commented] (HDFS-2733) Document HA configuration and CLI

2012-02-06 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201160#comment-13201160 ] Eli Collins commented on HDFS-2733: --- Looks good. Comments follow. * Might make sense to

[jira] [Commented] (HDFS-2794) HA: Active NN may purge edit log files before standby NN has a chance to read them

2012-02-06 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201128#comment-13201128 ] Eli Collins commented on HDFS-2794: --- I like this approach better too. Patch looks good.

[jira] [Commented] (HDFS-2819) Document new HA-related configs in hdfs-default.xml

2012-02-05 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201120#comment-13201120 ] Eli Collins commented on HDFS-2819: --- Oops, meant HDFS-2733 in that last comment.

[jira] [Commented] (HDFS-2579) Starting delegation token manager during safemode fails

2012-02-05 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201021#comment-13201021 ] Eli Collins commented on HDFS-2579: --- +1 looks good Nits: - All the SecretManager start/s

[jira] [Commented] (HDFS-2894) HA: disable 2NN when HA is enabled

2012-02-05 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200951#comment-13200951 ] Eli Collins commented on HDFS-2894: --- The bug here is that in the HA branch we're now trea

[jira] [Commented] (HDFS-2579) Starting delegation token manager during safemode fails

2012-02-05 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200880#comment-13200880 ] Eli Collins commented on HDFS-2579: --- Sounds good to me. > Starting deleg

[jira] [Commented] (HDFS-2868) Add number of active transfer threads to the DataNode status

2012-02-05 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200837#comment-13200837 ] Eli Collins commented on HDFS-2868: --- +1 thanks Harsh! > Add number of a

[jira] [Commented] (HDFS-2868) Add number of active transfer threads to the DataNode status

2012-02-05 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200825#comment-13200825 ] Eli Collins commented on HDFS-2868: --- Looks good Harsh. Nits: - I'd make the comment "@Ov

[jira] [Commented] (HDFS-2893) The 2NN won't start if dfs.namenode.secondary.http-address is default or specified with a wildcard IP and port

2012-02-04 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200587#comment-13200587 ] Eli Collins commented on HDFS-2893: --- Actually, I don't see why sbin/start-dfs.sh checks t

[jira] [Commented] (HDFS-2769) HA: When HA is enabled with a shared edits dir, that dir should be marked required

2012-02-03 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200228#comment-13200228 ] Eli Collins commented on HDFS-2769: --- Shared storage does not imply a single point of fail

[jira] [Commented] (HDFS-2792) HA: Make fsck work

2012-02-02 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13199495#comment-13199495 ] Eli Collins commented on HDFS-2792: --- How about the following? - hdfs fsck should not requ

[jira] [Commented] (HDFS-2769) HA: When HA is enabled with a shared edits dir, that dir should be marked required

2012-02-02 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13199489#comment-13199489 ] Eli Collins commented on HDFS-2769: --- +1 > HA: When HA is enabled with a

[jira] [Commented] (HDFS-2860) HA: TestDFSRollback#testRollback is failing

2012-02-02 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13199441#comment-13199441 ] Eli Collins commented on HDFS-2860: --- +1 sorry should have caught this when filing!

[jira] [Commented] (HDFS-2808) HA: Use logical names in haadmin

2012-02-02 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13199059#comment-13199059 ] Eli Collins commented on HDFS-2808: --- s/"slash"/"dash"/ > HA: Use logical

[jira] [Commented] (HDFS-2861) HA: checkpointing should verify that the dfs.http.address has been configured to a non-loopback for peer NN

2012-02-02 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13198941#comment-13198941 ] Eli Collins commented on HDFS-2861: --- +1 lgtm > HA: checkpointing should

[jira] [Commented] (HDFS-2870) HA: Remove some INFO level logging accidentally left around

2012-02-01 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13198046#comment-13198046 ] Eli Collins commented on HDFS-2870: --- +1 > HA: Remove some INFO level log

[jira] [Commented] (HDFS-2861) HA: checkpointing should verify that the dfs.http.address has been configured to a non-loopback for peer NN

2012-01-31 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13197597#comment-13197597 ] Eli Collins commented on HDFS-2861: --- +1 Nit, would add an assert or comment that HTTP_A

[jira] [Commented] (HDFS-2742) HA: observed dataloss in replication stress test

2012-01-31 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13197589#comment-13197589 ] Eli Collins commented on HDFS-2742: --- Todd, thanks for the detailed explanation! +1 to th

[jira] [Commented] (HDFS-2292) HA: HTTP fail-over

2012-01-31 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13197458#comment-13197458 ] Eli Collins commented on HDFS-2292: --- There are three separate cases to handle: # web UI #

[jira] [Commented] (HDFS-2857) Cleanup BlockInfo class

2012-01-31 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13197090#comment-13197090 ] Eli Collins commented on HDFS-2857: --- +1 lgtm. Mind merging to 23? Would be good to have t

[jira] [Commented] (HDFS-2860) HA: TestDFSRollback#testRollback is failing

2012-01-30 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13196668#comment-13196668 ] Eli Collins commented on HDFS-2860: --- Here's the failing assert: {noformat} junit.framewo

[jira] [Commented] (HDFS-2853) HA: NN fails to start if the shared edits dir is marked required

2012-01-30 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13196648#comment-13196648 ] Eli Collins commented on HDFS-2853: --- +1 Nit: MiniDFSCluster#formatSharedEditsDir might b

[jira] [Commented] (HDFS-2742) HA: observed dataloss in replication stress test

2012-01-29 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13195900#comment-13195900 ] Eli Collins commented on HDFS-2742: --- bq. I don't entirely follow what you're getting at h

[jira] [Commented] (HDFS-2791) If block report races with closing of file, replica is incorrectly marked corrupt

2012-01-29 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13195897#comment-13195897 ] Eli Collins commented on HDFS-2791: --- bq. Eli: >... overloading the notion of a corrupt bl

[jira] [Commented] (HDFS-2691) HA: Tests and fixes for pipeline targets and replica recovery

2012-01-29 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13195895#comment-13195895 ] Eli Collins commented on HDFS-2691: --- +1 looks great > HA: Tests and fix

[jira] [Commented] (HDFS-2840) TestHostnameFilter should work with localhost or localhost.localdomain

2012-01-27 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13195247#comment-13195247 ] Eli Collins commented on HDFS-2840: --- +1 lgtm > TestHostnameFilter should

[jira] [Commented] (HDFS-2851) After Balancer runs, usedSpace is not balancing correctly.

2012-01-27 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13195215#comment-13195215 ] Eli Collins commented on HDFS-2851: --- Wonder if this is fixed by HDFS-1105. Would be good

[jira] [Commented] (HDFS-2833) Add GETMERGE operation to httpfs

2012-01-27 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13195002#comment-13195002 ] Eli Collins commented on HDFS-2833: --- getmerge is an FsShell API, not a FileSystem/Context

[jira] [Commented] (HDFS-2838) NPE in FSNamesystem when in safe mode

2012-01-26 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13194287#comment-13194287 ] Eli Collins commented on HDFS-2838: --- +1 nice test. > NPE in FSNamesyste

[jira] [Commented] (HDFS-2691) HA: Tests and fixes for pipeline targets and replica recovery

2012-01-25 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13193442#comment-13193442 ] Eli Collins commented on HDFS-2691: --- Sorry for chiming in late. I think solution #1 is pr

[jira] [Commented] (HDFS-2791) If block report races with closing of file, replica is incorrectly marked corrupt

2012-01-25 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13193418#comment-13193418 ] Eli Collins commented on HDFS-2791: --- I think the current patch is preferable, am +1 on it

[jira] [Commented] (HDFS-2742) HA: observed dataloss in replication stress test

2012-01-25 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13193341#comment-13193341 ] Eli Collins commented on HDFS-2742: --- Todd, Approach in the latest patch looks good to me

[jira] [Commented] (HDFS-2838) NPE in FSNamesystem when in safe mode

2012-01-24 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13192826#comment-13192826 ] Eli Collins commented on HDFS-2838: --- No worries. Greg is going to take a stab at moving t

[jira] [Commented] (HDFS-2838) NPE in FSNamesystem when in safe mode

2012-01-24 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13192808#comment-13192808 ] Eli Collins commented on HDFS-2838: --- +1 > NPE in FSNamesystem when in sa

[jira] [Commented] (HDFS-2804) SBN should not mark blocks under-replicated when exiting safemode

2012-01-23 Thread Eli Collins (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191539#comment-13191539 ] Eli Collins commented on HDFS-2804: --- Still looks good =) > SBN should no

<    1   2   3   4   >