[jira] [Commented] (HDFS-1623) High Availability Framework for HDFS NN

2012-03-14 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13229534#comment-13229534 ] Todd Lipcon commented on HDFS-1623: --- Hi Avik. The multi-SBN approach is actually very nea

[jira] [Commented] (HDFS-3077) Quorum-based protocol for reading and writing edit logs

2012-03-14 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13229529#comment-13229529 ] Todd Lipcon commented on HDFS-3077: --- bq. Sequencing writes between different writers is t

[jira] [Commented] (HDFS-3077) Quorum-based protocol for reading and writing edit logs

2012-03-14 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13229527#comment-13229527 ] Todd Lipcon commented on HDFS-3077: --- bq. These arguments seem very much to be a case of N

[jira] [Commented] (HDFS-3077) Quorum-based protocol for reading and writing edit logs

2012-03-14 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13229343#comment-13229343 ] Todd Lipcon commented on HDFS-3077: --- Hi Suresh, Apologies in advance for the length of t

[jira] [Commented] (HDFS-3077) Quorum-based protocol for reading and writing edit logs

2012-03-14 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13229050#comment-13229050 ] Todd Lipcon commented on HDFS-3077: --- I should mention another advantage of this system ov

[jira] [Commented] (HDFS-3090) Broaden NN#monitorHealth checks

2012-03-13 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13228987#comment-13228987 ] Todd Lipcon commented on HDFS-3090: --- Konstantin also pointed out that we should monitor t

[jira] [Commented] (HDFS-3078) 2NN https port setting is broken in Hadoop 1.0

2012-03-12 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13227868#comment-13227868 ] Todd Lipcon commented on HDFS-3078: --- +1 > 2NN https port setting is brok

[jira] [Commented] (HDFS-3077) Quorum-based protocol for reading and writing edit logs

2012-03-12 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13227780#comment-13227780 ] Todd Lipcon commented on HDFS-3077: --- I plan to post a more thorough design doc in the nex

[jira] [Commented] (HDFS-3062) Fail to submit mapred job on a secured-HA-HDFS: logic URI cannot be picked up by job submission.

2012-03-11 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13227293#comment-13227293 ] Todd Lipcon commented on HDFS-3062: --- Thanks for the patch, Mingjie! A small formatting ni

[jira] [Commented] (HDFS-3064) Allow datanodes to start with non-privileged ports for testing.

2012-03-08 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225442#comment-13225442 ] Todd Lipcon commented on HDFS-3064: --- Sure, I agree with you whole-heartedly :) But Jakob

[jira] [Commented] (HDFS-3065) HA: Newly active NameNode does not recognize decommissioning DataNode

2012-03-08 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225438#comment-13225438 ] Todd Lipcon commented on HDFS-3065: --- I think the solution here is that "refreshNodes" sho

[jira] [Commented] (HDFS-3064) Allow datanodes to start with non-privileged ports for testing.

2012-03-08 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225351#comment-13225351 ] Todd Lipcon commented on HDFS-3064: --- FWIW, I suggested this a while back and people seeme

[jira] [Commented] (HDFS-3061) Cached directory size in INodeDirectory can get permantently out of sync with computed size, causing quota issues

2012-03-07 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13224879#comment-13224879 ] Todd Lipcon commented on HDFS-3061: --- Possible this is HDFS-1189 or HDFS-1377? Can you rep

[jira] [Commented] (HDFS-3051) A zero-copy ScatterGatherRead api from FSDataInputStream

2012-03-06 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223116#comment-13223116 ] Todd Lipcon commented on HDFS-3051: --- One interesting thing to take note of: in Linux prio

[jira] [Commented] (HDFS-2834) ByteBuffer-based read API for DFSInputStream

2012-03-06 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223099#comment-13223099 ] Todd Lipcon commented on HDFS-2834: --- Thanks Dhruba. I opened HDFS-3053 for further discus

[jira] [Commented] (HDFS-2834) ByteBuffer-based read API for DFSInputStream

2012-03-05 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223052#comment-13223052 ] Todd Lipcon commented on HDFS-2834: --- Hi Dhruba. Looks like your implementation provides f

[jira] [Commented] (HDFS-2731) HA: Autopopulate standby name dirs if they're empty

2012-03-05 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222693#comment-13222693 ] Todd Lipcon commented on HDFS-2731: --- Here's an example output: {code} ==

[jira] [Commented] (HDFS-3048) Small race in BlockManager#close

2012-03-05 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222595#comment-13222595 ] Todd Lipcon commented on HDFS-3048: --- I think you'd need to do a join() on the replication

[jira] [Commented] (HDFS-2834) ByteBuffer-based read API for DFSInputStream

2012-03-03 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13221785#comment-13221785 ] Todd Lipcon commented on HDFS-2834: --- Hey Henry. Thanks for getting test-patch clean. I'll

[jira] [Commented] (HDFS-3038) Add FSEditLog.metrics to findbugs exclude list

2012-03-01 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220556#comment-13220556 ] Todd Lipcon commented on HDFS-3038: --- Failed tests are obviously unrelated, since this jus

[jira] [Commented] (HDFS-3037) TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy

2012-03-01 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220437#comment-13220437 ] Todd Lipcon commented on HDFS-3037: --- +1 pending Hudson > TestMulitipleNN

[jira] [Commented] (HDFS-3020) Auto-logSync based on edit log buffer size broken

2012-03-01 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220354#comment-13220354 ] Todd Lipcon commented on HDFS-3020: --- The latest patch was against a non-trunk branch so i

[jira] [Commented] (HDFS-3036) Remove unused method DFSUtil#isDefaultNamenodeAddress

2012-03-01 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220294#comment-13220294 ] Todd Lipcon commented on HDFS-3036: --- +1 > Remove unused method DFSUtil#i

[jira] [Commented] (HDFS-3023) Optimize entries in edits log for persistBlocks calls

2012-03-01 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220212#comment-13220212 ] Todd Lipcon commented on HDFS-3023: --- Committed the delta (r1295721) > Op

[jira] [Commented] (HDFS-3023) Optimize entries in edits log for persistBlocks calls

2012-03-01 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220206#comment-13220206 ] Todd Lipcon commented on HDFS-3023: --- Accidentally missed one place to add the new LV: {co

[jira] [Commented] (HDFS-2979) HA: Balancer should use logical uri for creating failover proxy with HA enabled.

2012-03-01 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13219924#comment-13219924 ] Todd Lipcon commented on HDFS-2979: --- +1 lgtm (I reviewed only the delta from the previous

[jira] [Commented] (HDFS-2979) HA: Balancer should use logical uri for creating failover proxy with HA enabled.

2012-03-01 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13219918#comment-13219918 ] Todd Lipcon commented on HDFS-2979: --- Small nits: - please rename {{getNameNodeInfo}} to {

[jira] [Commented] (HDFS-3031) HA: Error (failed to close file) when uploading large file + kill active NN + manual failover

2012-02-29 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13219634#comment-13219634 ] Todd Lipcon commented on HDFS-3031: --- Seems like we have two problems: 1) we don't notice

[jira] [Commented] (HDFS-2984) S-live: Rate operation count for delete is worse than 0.20.204 by 28.8%

2012-02-29 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13219548#comment-13219548 ] Todd Lipcon commented on HDFS-2984: --- Hi Amol. Can you include the git hash or svn branch/

[jira] [Commented] (HDFS-2991) failure to load edits: ClassCastException

2012-02-29 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13219407#comment-13219407 ] Todd Lipcon commented on HDFS-2991: --- Thanks. I committed this to 23.2, branch-23, and tru

[jira] [Commented] (HDFS-309) FSEditLog should log progress during replay

2012-02-28 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218823#comment-13218823 ] Todd Lipcon commented on HDFS-309: -- {code} +LOG.info("Acquiring write lock to replay ed

[jira] [Commented] (HDFS-1623) High Availability Framework for HDFS NN

2012-02-28 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218813#comment-13218813 ] Todd Lipcon commented on HDFS-1623: --- With HDFS-3020, HDFS-3023, HDFS-3024, HDFS-3025, I g

[jira] [Commented] (HDFS-3023) Optimize entries in edits log for persistBlocks calls

2012-02-28 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218806#comment-13218806 ] Todd Lipcon commented on HDFS-3023: --- I've been running this patch on a cluster. With this

[jira] [Commented] (HDFS-3028) DFS rollback broken

2012-02-28 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218795#comment-13218795 ] Todd Lipcon commented on HDFS-3028: --- I don't think so - I ran the rollback with the old v

[jira] [Commented] (HDFS-2991) failure to load edits: ClassCastException

2012-02-28 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218790#comment-13218790 ] Todd Lipcon commented on HDFS-2991: --- Test failure is because it's missing the correspondi

[jira] [Commented] (HDFS-3028) DFS rollback broken

2012-02-28 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218772#comment-13218772 ] Todd Lipcon commented on HDFS-3028: --- Aha, sorry for the many comments, but: the blockpool

[jira] [Commented] (HDFS-3028) DFS rollback broken

2012-02-28 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218770#comment-13218770 ] Todd Lipcon commented on HDFS-3028: --- Strangely, I don't seem to have a {{previous}} dir i

[jira] [Commented] (HDFS-3028) DFS rollback broken

2012-02-28 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218767#comment-13218767 ] Todd Lipcon commented on HDFS-3028: --- When I run {{start-dfs.sh -rollback}} the DNs do pro

[jira] [Commented] (HDFS-3020) Auto-logSync based on edit log buffer size broken

2012-02-28 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218718#comment-13218718 ] Todd Lipcon commented on HDFS-3020: --- Suresh, mind if I address those unnecessary IOEs sep

[jira] [Commented] (HDFS-3023) Optimize entries in edits log for persistBlocks calls

2012-02-28 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218449#comment-13218449 ] Todd Lipcon commented on HDFS-3023: --- I'll file a follow-up JIRA about that TODO I added.

[jira] [Commented] (HDFS-3004) Create Offline NameNode recovery tool

2012-02-28 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218435#comment-13218435 ] Todd Lipcon commented on HDFS-3004: --- bq. The editlog entry in the middle is corrupt, foll

[jira] [Commented] (HDFS-1623) High Availability Framework for HDFS NN

2012-02-28 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218402#comment-13218402 ] Todd Lipcon commented on HDFS-1623: --- Yep, I plan to re-run these benchmarks hopefully tod

[jira] [Commented] (HDFS-3023) Optimize entries in edits log for persistBlocks calls

2012-02-28 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218386#comment-13218386 ] Todd Lipcon commented on HDFS-3023: --- bq. By option #2 vs option #3 above I meant componen

[jira] [Commented] (HDFS-3023) Optimize entries in edits log for persistBlocks calls

2012-02-28 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218384#comment-13218384 ] Todd Lipcon commented on HDFS-3023: --- bq. though why does updating block info also now upd

[jira] [Commented] (HDFS-3020) Auto-logSync based on edit log buffer size broken

2012-02-28 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218381#comment-13218381 ] Todd Lipcon commented on HDFS-3020: --- bq. Does this mean that the editlog is being synced

[jira] [Commented] (HDFS-3023) Optimize entries in edits log for persistBlocks calls

2012-02-28 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218375#comment-13218375 ] Todd Lipcon commented on HDFS-3023: --- I do think the log size is the major difference caus

[jira] [Commented] (HDFS-3004) Create Offline NameNode recovery tool

2012-02-28 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218364#comment-13218364 ] Todd Lipcon commented on HDFS-3004: --- Thanks for the doc. Makes sense. The only addition I

[jira] [Commented] (HDFS-3023) Optimize entries in edits log for persistBlocks calls

2012-02-27 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13217895#comment-13217895 ] Todd Lipcon commented on HDFS-3023: --- Apparently I am bad at counting. The above should re

[jira] [Commented] (HDFS-3019) TestEditLogJournalFailures silently failing

2012-02-27 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13217802#comment-13217802 ] Todd Lipcon commented on HDFS-3019: --- Apparently the patch needs some alteration against t

[jira] [Commented] (HDFS-3011) move webapps/ into the JAR

2012-02-27 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13217677#comment-13217677 ] Todd Lipcon commented on HDFS-3011: --- Yea, I used to do that, then I found it annoying so

[jira] [Commented] (HDFS-3013) HA: NameNode format doesn't pick up dfs.namenode.name.dir.NameServiceId configure

2012-02-27 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13217661#comment-13217661 ] Todd Lipcon commented on HDFS-3013: --- +1, assuming tests pass. Running a smattering of the

[jira] [Commented] (HDFS-3011) move webapps/ into the JAR

2012-02-27 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13217606#comment-13217606 ] Todd Lipcon commented on HDFS-3011: --- Hm, I think I do that, but on my version of Eclipse

[jira] [Commented] (HDFS-309) FSEditLog should log progress during replay

2012-02-27 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13217554#comment-13217554 ] Todd Lipcon commented on HDFS-309: -- Having this trace information seems useful -- but a bit

[jira] [Commented] (HDFS-2978) The NameNode should expose name dir statuses via JMX

2012-02-24 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13216190#comment-13216190 ] Todd Lipcon commented on HDFS-2978: --- I agree with Hitesh - JMX is supposed to support arb

[jira] [Commented] (HDFS-3013) NameNode format doesn't pick up dfs.namenode.name.dir.NameServiceId configure

2012-02-24 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13216185#comment-13216185 ] Todd Lipcon commented on HDFS-3013: --- Nice find, Mingjie. Want to take a crack at a patch?

[jira] [Commented] (HDFS-2904) HA: Client support for getting delegation tokens to an HA cluster

2012-02-24 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13216134#comment-13216134 ] Todd Lipcon commented on HDFS-2904: --- Cool, I agree with your points above. Let's continue

[jira] [Commented] (HDFS-3011) move webapps/ into the JAR

2012-02-24 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13215969#comment-13215969 ] Todd Lipcon commented on HDFS-3011: --- While you're at it, another thing I've noticed is th

[jira] [Commented] (HDFS-2904) HA: Client support for getting delegation tokens to an HA cluster

2012-02-24 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13215812#comment-13215812 ] Todd Lipcon commented on HDFS-2904: --- bq. I think we should not couple the idea of logical

[jira] [Commented] (HDFS-3010) Get performance on HA branch to match trunk

2012-02-24 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13215806#comment-13215806 ] Todd Lipcon commented on HDFS-3010: --- There are three possible components to the perf issu

[jira] [Commented] (HDFS-1623) High Availability Framework for HDFS NN

2012-02-23 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13215309#comment-13215309 ] Todd Lipcon commented on HDFS-1623: --- I've completed a round of preliminary performance te

[jira] [Commented] (HDFS-2991) failure to load edits: ClassCastException

2012-02-23 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13215050#comment-13215050 ] Todd Lipcon commented on HDFS-2991: --- bq. So you allow OP_CLOSE without the preceding OP_A

[jira] [Commented] (HDFS-2987) DNA_SHUTDOWN command is never sent by the NN

2012-02-23 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13214971#comment-13214971 ] Todd Lipcon commented on HDFS-2987: --- Do you think it actually works? Given it's never bee

[jira] [Commented] (HDFS-2991) failure to load edits: ClassCastException

2012-02-23 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13214966#comment-13214966 ] Todd Lipcon commented on HDFS-2991: --- Good catch on #1. bq. You do not need to logOpenFil

[jira] [Commented] (HDFS-2995) start-dfs.sh tries to start 2NN everywhere

2012-02-23 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13214919#comment-13214919 ] Todd Lipcon commented on HDFS-2995: --- Seems likely. Eli, I see you're watching this ticket

[jira] [Commented] (HDFS-2993) HA: BackupNode#checkOperation should permit CHECKPOINT operations

2012-02-22 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13214426#comment-13214426 ] Todd Lipcon commented on HDFS-2993: --- +1 > HA: BackupNode#checkOperation

[jira] [Commented] (HDFS-2991) failure to load edits: ClassCastException

2012-02-22 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13214406#comment-13214406 ] Todd Lipcon commented on HDFS-2991: --- One thing worth testing here before commit, I just t

[jira] [Commented] (HDFS-2994) If lease is recovered successfully inline with create, create can fail

2012-02-22 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13214400#comment-13214400 ] Todd Lipcon commented on HDFS-2994: --- Not entirely sure how I got into the state where all

[jira] [Commented] (HDFS-2991) failure to load edits: ClassCastException

2012-02-22 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13214358#comment-13214358 ] Todd Lipcon commented on HDFS-2991: --- bq. org.apache.hadoop.hdfs.server.common.TestDistrib

[jira] [Commented] (HDFS-2991) failure to load edits: ClassCastException

2012-02-22 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13214238#comment-13214238 ] Todd Lipcon commented on HDFS-2991: --- I've verified this bug is present on the 0.23.1-rc2

[jira] [Commented] (HDFS-2991) failure to load edits: ClassCastException

2012-02-22 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13214220#comment-13214220 ] Todd Lipcon commented on HDFS-2991: --- Inspecting the code, it seems like this bug is prese

[jira] [Commented] (HDFS-2991) failure to load edits: ClassCastException

2012-02-22 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13214210#comment-13214210 ] Todd Lipcon commented on HDFS-2991: --- Correlating OEV output vs audit log output, this see

[jira] [Commented] (HDFS-2983) Relax the build version check to permit rolling upgrades within a release

2012-02-22 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13213955#comment-13213955 ] Todd Lipcon commented on HDFS-2983: --- Would be nice to, at the same time or separately, ad

[jira] [Commented] (HDFS-2981) The default value of dfs.client.block.write.replace-datanode-on-failure.enable should be true

2012-02-21 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13213228#comment-13213228 ] Todd Lipcon commented on HDFS-2981: --- Oh, my bad. Sorry for not looking. +1

[jira] [Commented] (HDFS-2981) The default value of dfs.client.block.write.replace-datanode-on-failure.enable should be true

2012-02-21 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13213206#comment-13213206 ] Todd Lipcon commented on HDFS-2981: --- Oh, I see. That makes sense, then... But I think yo

[jira] [Commented] (HDFS-2982) Startup performance suffers when there are many edit log segments

2012-02-21 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13213169#comment-13213169 ] Todd Lipcon commented on HDFS-2982: --- With 9000 files over NFS it took 20 minutes to call

[jira] [Commented] (HDFS-2981) The default value of dfs.client.block.write.replace-datanode-on-failure.enable should be true

2012-02-21 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13213103#comment-13213103 ] Todd Lipcon commented on HDFS-2981: --- I disagree that it should be true by default. Apps l

[jira] [Commented] (HDFS-2904) HA: Client support for getting delegation tokens to an HA cluster

2012-02-21 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13213101#comment-13213101 ] Todd Lipcon commented on HDFS-2904: --- {quote} > logical URI is "hdfs://my-ha-cluster/" You

[jira] [Commented] (HDFS-2977) HA: TestBalancerWithHANameNodes is failing

2012-02-20 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13212313#comment-13212313 ] Todd Lipcon commented on HDFS-2977: --- Uma: I didn't quite follow your comment. Do you thin

[jira] [Commented] (HDFS-2977) HA: TestBalancerWithHANameNodes is failing

2012-02-20 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13212136#comment-13212136 ] Todd Lipcon commented on HDFS-2977: --- +1 > HA: TestBalancerWithHANameNode

[jira] [Commented] (HDFS-2974) HA: MiniDFSCluster does not delete standby NN name dirs during format

2012-02-19 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13211637#comment-13211637 ] Todd Lipcon commented on HDFS-2974: --- +1 > HA: MiniDFSCluster does not de

[jira] [Commented] (HDFS-2952) HA: NN should not start with upgrade option or with a pending an unfinalized upgrade

2012-02-17 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210753#comment-13210753 ] Todd Lipcon commented on HDFS-2952: --- +1 > HA: NN should not start with u

[jira] [Commented] (HDFS-2904) HA: Client support for getting delegation tokens to an HA cluster

2012-02-17 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210581#comment-13210581 ] Todd Lipcon commented on HDFS-2904: --- bq. Currently, hftp and webhdfs actually set a diffe

[jira] [Commented] (HDFS-2968) Protocol translator for BlockRecoveryCommand broken when multiple blocks need recovery

2012-02-17 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210522#comment-13210522 ] Todd Lipcon commented on HDFS-2968: --- 0.23.1 doesn't have protobufs in place yet, right?

[jira] [Commented] (HDFS-2969) ExtendedBlock.equals is incorrectly implemented

2012-02-17 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210509#comment-13210509 ] Todd Lipcon commented on HDFS-2969: --- Following code is obviously wrong: {code} Extend

[jira] [Commented] (HDFS-2968) Protocol translator for BlockRecoveryCommand broken when multiple blocks need recovery

2012-02-17 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210500#comment-13210500 ] Todd Lipcon commented on HDFS-2968: --- The PBHelper code has: {code} for (int i = 0; i

[jira] [Commented] (HDFS-2929) HA: stress test and fixes for block synchronization

2012-02-16 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209963#comment-13209963 ] Todd Lipcon commented on HDFS-2929: --- Looping the test for a while I got it to fail, saw t

[jira] [Commented] (HDFS-2904) HA: Client support for getting delegation tokens to an HA cluster

2012-02-16 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209878#comment-13209878 ] Todd Lipcon commented on HDFS-2904: --- bq. Though having a comma separated list of hostname

[jira] [Commented] (HDFS-2926) HA: enable hadoop security authorization for haadmin / protocols

2012-02-16 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209817#comment-13209817 ] Todd Lipcon commented on HDFS-2926: --- Given that we checkSuperuserPrivileges, I think it's

[jira] [Commented] (HDFS-2781) Add client protocol and DFSadmin for command to restore failed storage

2012-02-16 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209811#comment-13209811 ] Todd Lipcon commented on HDFS-2781: --- We can move this to be a non-HA ticket right?

[jira] [Commented] (HDFS-2754) HA: enable dfs.namenode.name.dir.restore if HA is enabled

2012-02-16 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209807#comment-13209807 ] Todd Lipcon commented on HDFS-2754: --- Since we currently don't support multiple shared edi

[jira] [Commented] (HDFS-2960) HA: TestDFSUtil is failing

2012-02-16 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209527#comment-13209527 ] Todd Lipcon commented on HDFS-2960: --- I think we have a couple other cases in the Hadoop t

[jira] [Commented] (HDFS-2902) HA: Allow new (shared) edits log dir to be configured while NN is running

2012-02-16 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209521#comment-13209521 ] Todd Lipcon commented on HDFS-2902: --- Can we convert this to a non-HA JIRA? Seems like add

[jira] [Commented] (HDFS-2910) HA: Active NN reports Bad state: BETWEEN_LOG_SEGMENTS when shared edits dir is inaccessible during log roll

2012-02-16 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209520#comment-13209520 ] Todd Lipcon commented on HDFS-2910: --- Maybe we can convert this to not be an HA subtask/co

[jira] [Commented] (HDFS-2910) HA: Active NN reports Bad state: BETWEEN_LOG_SEGMENTS when shared edits dir is inaccessible during log roll

2012-02-15 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209176#comment-13209176 ] Todd Lipcon commented on HDFS-2910: --- Hey Bikas. Is this still an issue after your other c

[jira] [Commented] (HDFS-2958) HA: Sweep for remaining proxy construction which doesn't go through failover path

2012-02-15 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209141#comment-13209141 ] Todd Lipcon commented on HDFS-2958: --- We should deprecate/remove DFSUtil.createNamenode an

[jira] [Commented] (HDFS-2960) HA: TestDFSUtil is failing

2012-02-15 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209080#comment-13209080 ] Todd Lipcon commented on HDFS-2960: --- Odd, that IP is part of "China Unicom". Do you have

[jira] [Commented] (HDFS-2904) HA: Client support for getting delegation tokens to an HA cluster

2012-02-15 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209010#comment-13209010 ] Todd Lipcon commented on HDFS-2904: --- {quote} when the JT renews delegation tokens on beha

[jira] [Commented] (HDFS-1846) Don't fill preallocated portion of edits log with 0x00

2012-02-15 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13208984#comment-13208984 ] Todd Lipcon commented on HDFS-1846: --- Good point - the bug in the previous code was that w

[jira] [Commented] (HDFS-2586) Add protobuf service and implementation for HAServiceProtocol

2012-02-15 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13208723#comment-13208723 ] Todd Lipcon commented on HDFS-2586: --- The bug is here -- missing "break"s: {code} +HAS

[jira] [Commented] (HDFS-2938) Recursive delete of a large directory makes namenode unresponsive

2012-02-15 Thread Todd Lipcon (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13208710#comment-13208710 ] Todd Lipcon commented on HDFS-2938: --- +1, looks good. One nit: missing a '.' in the javado

<    1   2   3   4   5   6   7   8   >