[jira] [Commented] (HDFS-7034) Archival Storage: Fix TestBlockPlacement and TestStorageMover

2014-09-11 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14129731#comment-14129731 ] Tsz Wo Nicholas Sze commented on HDFS-7034: --- Since the new conf properties are

[jira] [Updated] (HDFS-7039) Archival Storage: fix Balancer tests

2014-09-11 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-7039: -- Attachment: h7039_20140911.patch I added checking target space for scheduling in

[jira] [Updated] (HDFS-6584) Support Archival Storage

2014-09-11 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6584: -- Attachment: h7039_20140911.patch h7039_20140911.patch: with HDFS-7034 and HDFS-7039.

[jira] [Commented] (HDFS-6584) Support Archival Storage

2014-09-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14129748#comment-14129748 ] Hadoop QA commented on HDFS-6584: - {color:red}-1 overall{color}. Here are the results of

[jira] [Updated] (HDFS-6867) For DFSOutputStream, do pipeline recovery for a single block in the background

2014-09-11 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-6867: Attachment: HDFS-6867-20140910.patch New patch to implement updated design. For DFSOutputStream, do

[jira] [Created] (HDFS-7044) Support retention policy based on access time and modify time, use XAttr to store policy

2014-09-11 Thread zhaoyunjiong (JIRA)
zhaoyunjiong created HDFS-7044: -- Summary: Support retention policy based on access time and modify time, use XAttr to store policy Key: HDFS-7044 URL: https://issues.apache.org/jira/browse/HDFS-7044

[jira] [Created] (HDFS-7045) Fix deadlock of open file (in some cases)

2014-09-11 Thread Yi Liu (JIRA)
Yi Liu created HDFS-7045: Summary: Fix deadlock of open file (in some cases) Key: HDFS-7045 URL: https://issues.apache.org/jira/browse/HDFS-7045 Project: Hadoop HDFS Issue Type: Bug

[jira] [Commented] (HDFS-7045) Fix deadlock of open file (in some cases)

2014-09-11 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14129795#comment-14129795 ] Yi Liu commented on HDFS-7045: -- Will attach the patch later. Fix deadlock of open file (in

[jira] [Updated] (HDFS-6584) Support Archival Storage

2014-09-11 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6584: -- Attachment: h6584_20140911.patch Oops, uploaded a wrong file. Here is the correct one.

[jira] [Updated] (HDFS-6584) Support Archival Storage

2014-09-11 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6584: -- Attachment: (was: h7039_20140911.patch) Support Archival Storage

[jira] [Commented] (HDFS-7040) HDFS dangerously uses @Beta methods from very old versions of Guava

2014-09-11 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14129853#comment-14129853 ] Steve Loughran commented on HDFS-7040: -- This is the long-standing hadoop branch-2

Namenode read large editlogs very slow when starting

2014-09-11 Thread Liu Cong
Hi all, Is it appropriate that sending my issue to this mailing list? I'm using hadoop v2.2 on a cluster of 16 servers. It have about 5,000,000 files. 10 days ago, the second namenode downed because OOM, and I didnot noticed, my fault. Yesterday I changed some configs and restarted hadoop,

[jira] [Commented] (HDFS-6867) For DFSOutputStream, do pipeline recovery for a single block in the background

2014-09-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14129881#comment-14129881 ] Hadoop QA commented on HDFS-6867: - {color:red}-1 overall{color}. Here are the results of

[jira] [Updated] (HDFS-7045) Fix deadlock of open file (in some cases)

2014-09-11 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7045: - Description: {{resolvePath}} could throw exception in following case, then deadlock. * open file under path

[jira] [Updated] (HDFS-7045) Fix deadlock of open file (in some cases)

2014-09-11 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7045: - Description: {{resolvePath}} could throw exception in following case, then deadlock. * open file under path

[jira] [Updated] (HDFS-7045) Fix deadlock of open file (in some cases)

2014-09-11 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7045: - Attachment: HDFS-7045.001.patch The patch includes a test to reproduce this issue. Fix deadlock of open file (in

[jira] [Updated] (HDFS-7045) Fix deadlock of open file (in some cases)

2014-09-11 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7045: - Attachment: HDFS-7045.001.patch Fix deadlock of open file (in some cases)

[jira] [Updated] (HDFS-7045) Fix deadlock of open file (in some cases)

2014-09-11 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7045: - Attachment: (was: HDFS-7045.001.patch) Fix deadlock of open file (in some cases)

[jira] [Updated] (HDFS-7045) Fix deadlock of open file (in some cases)

2014-09-11 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7045: - Status: Patch Available (was: Open) Fix deadlock of open file (in some cases)

[jira] [Updated] (HDFS-7045) Fix NameNode deadlock when opening file under /.reserved path

2014-09-11 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7045: - Summary: Fix NameNode deadlock when opening file under /.reserved path (was: Fix deadlock of open file (in some

[jira] [Commented] (HDFS-7044) Support retention policy based on access time and modify time, use XAttr to store policy

2014-09-11 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130001#comment-14130001 ] Allen Wittenauer commented on HDFS-7044: How does this relate to HDFS-6382?

[jira] [Updated] (HDFS-7032) Add WebHDFS support for reading and writing to encryption zones

2014-09-11 Thread Charles Lamb (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charles Lamb updated HDFS-7032: --- Attachment: HDFS-7032.001.patch Submitting patch for test-patch run. Add WebHDFS support for reading

[jira] [Created] (HDFS-7046) HA NN can NPE upon transition to active

2014-09-11 Thread Daryn Sharp (JIRA)
Daryn Sharp created HDFS-7046: - Summary: HA NN can NPE upon transition to active Key: HDFS-7046 URL: https://issues.apache.org/jira/browse/HDFS-7046 Project: Hadoop HDFS Issue Type: Bug

[jira] [Commented] (HDFS-7046) HA NN can NPE upon transition to active

2014-09-11 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130072#comment-14130072 ] Daryn Sharp commented on HDFS-7046: --- Stack trace from an internal 2.5.1-ish build. Line

[jira] [Commented] (HDFS-7046) HA NN can NPE upon transition to active

2014-09-11 Thread Aaron T. Myers (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130079#comment-14130079 ] Aaron T. Myers commented on HDFS-7046: -- Good find, Daryn. Given that you marked this

[jira] [Commented] (HDFS-6584) Support Archival Storage

2014-09-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130102#comment-14130102 ] Hadoop QA commented on HDFS-6584: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-7046) HA NN can NPE upon transition to active

2014-09-11 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130107#comment-14130107 ] Daryn Sharp commented on HDFS-7046: --- I actually do not know when the issue was

[jira] [Commented] (HDFS-7046) HA NN can NPE upon transition to active

2014-09-11 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130111#comment-14130111 ] Daryn Sharp commented on HDFS-7046: --- We've had 2 incidents. Both were delete ops. One

[jira] [Commented] (HDFS-7045) Fix NameNode deadlock when opening file under /.reserved path

2014-09-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130132#comment-14130132 ] Hadoop QA commented on HDFS-7045: - {color:red}-1 overall{color}. Here are the results of

[jira] [Updated] (HDFS-6965) NN continues to issue block locations for DNs with full disks

2014-09-11 Thread Rushabh S Shah (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rushabh S Shah updated HDFS-6965: - Attachment: HDFS-6965.patch Attaching a patch with the changes mentioned in comments by Daryn and

[jira] [Assigned] (HDFS-6965) NN continues to issue block locations for DNs with full disks

2014-09-11 Thread Rushabh S Shah (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rushabh S Shah reassigned HDFS-6965: Assignee: Rushabh S Shah (was: chang li) NN continues to issue block locations for DNs

[jira] [Updated] (HDFS-6965) NN continues to issue block locations for DNs with full disks

2014-09-11 Thread Rushabh S Shah (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rushabh S Shah updated HDFS-6965: - Status: Patch Available (was: Open) NN continues to issue block locations for DNs with full

[jira] [Commented] (HDFS-6194) Create new tests for ByteRangeInputStream

2014-09-11 Thread Juan Yu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130230#comment-14130230 ] Juan Yu commented on HDFS-6194: --- Hi [~wheat9], did you merge the new test to branch-2? the

[jira] [Updated] (HDFS-7044) Support retention policy based on access time and modify time, use XAttr to store policy

2014-09-11 Thread zhaoyunjiong (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhaoyunjiong updated HDFS-7044: --- Attachment: Retention policy design.pdf Attach a simple design document. The major difference between

[jira] [Commented] (HDFS-6867) For DFSOutputStream, do pipeline recovery for a single block in the background

2014-09-11 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130284#comment-14130284 ] Zhe Zhang commented on HDFS-6867: - The test failure seems unrelated. The recovery worker is

[jira] [Commented] (HDFS-6194) Create new tests for ByteRangeInputStream

2014-09-11 Thread Akira AJISAKA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130288#comment-14130288 ] Akira AJISAKA commented on HDFS-6194: - Thanks [~j...@cloudera.com] for the report. I

[jira] [Created] (HDFS-7047) Expose FileStatus#isEncrypted in libhdfs

2014-09-11 Thread Andrew Wang (JIRA)
Andrew Wang created HDFS-7047: - Summary: Expose FileStatus#isEncrypted in libhdfs Key: HDFS-7047 URL: https://issues.apache.org/jira/browse/HDFS-7047 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Commented] (HDFS-6194) Create new tests for ByteRangeInputStream

2014-09-11 Thread Juan Yu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130309#comment-14130309 ] Juan Yu commented on HDFS-6194: --- why not branch-2? btw, the old test that use HftpFileSystem

[jira] [Commented] (HDFS-6306) Standby NN can hold FSDirectory's writeLock for a long time under heavy load

2014-09-11 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130325#comment-14130325 ] Daryn Sharp commented on HDFS-6306: --- I like both approaches. I think #1 might be easier.

[jira] [Commented] (HDFS-6966) Add additional unit tests for encryption zones

2014-09-11 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130329#comment-14130329 ] Andrew Wang commented on HDFS-6966: --- +1 LGTM, will commit shortly. Add additional unit

[jira] [Commented] (HDFS-6621) Hadoop Balancer prematurely exits iterations

2014-09-11 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130337#comment-14130337 ] Andrew Wang commented on HDFS-6621: --- +1 LGTM, I'll commit this shortly. Hadoop Balancer

[jira] [Updated] (HDFS-6966) Add additional unit tests for encryption zones

2014-09-11 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6966: -- Resolution: Fixed Fix Version/s: 2.6.0 Status: Resolved (was: Patch Available)

[jira] [Assigned] (HDFS-6621) Hadoop Balancer prematurely exits iterations

2014-09-11 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang reassigned HDFS-6621: - Assignee: Rafal Wojdyla Hadoop Balancer prematurely exits iterations

[jira] [Updated] (HDFS-6621) Hadoop Balancer prematurely exits iterations

2014-09-11 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6621: -- Resolution: Fixed Fix Version/s: 2.6.0 Status: Resolved (was: Patch Available)

[jira] [Updated] (HDFS-6978) Directory scanner should correctly reconcile blocks on RAM disk

2014-09-11 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-6978: Attachment: HDFS-6978.01.patch Attaching patch: # Fixes DirectoryScanner.scan to support multiple

[jira] [Commented] (HDFS-6621) Hadoop Balancer prematurely exits iterations

2014-09-11 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130355#comment-14130355 ] Andrew Wang commented on HDFS-6621: --- Filed HDFS-7048 for problem 2, hopefully I captured

[jira] [Created] (HDFS-7048) Incorrect Balancer#Source wait/notify leads to early termination

2014-09-11 Thread Andrew Wang (JIRA)
Andrew Wang created HDFS-7048: - Summary: Incorrect Balancer#Source wait/notify leads to early termination Key: HDFS-7048 URL: https://issues.apache.org/jira/browse/HDFS-7048 Project: Hadoop HDFS

[jira] [Commented] (HDFS-6621) Hadoop Balancer prematurely exits iterations

2014-09-11 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130381#comment-14130381 ] Yongjun Zhang commented on HDFS-6621: - Thanks [~andrew.wang], and all of you!

[jira] [Commented] (HDFS-6194) Create new tests for ByteRangeInputStream

2014-09-11 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130389#comment-14130389 ] Haohui Mai commented on HDFS-6194: -- Sorry for the confusion. It looks like that the

[jira] [Commented] (HDFS-7045) Fix NameNode deadlock when opening file under /.reserved path

2014-09-11 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130397#comment-14130397 ] Andrew Wang commented on HDFS-7045: --- +1, nice find Yi. I'll commit this shortly. Fix

[jira] [Commented] (HDFS-6194) Create new tests for ByteRangeInputStream

2014-09-11 Thread Juan Yu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130416#comment-14130416 ] Juan Yu commented on HDFS-6194: --- it's not just reintroduce, the tests are also re-written.

[jira] [Updated] (HDFS-7034) Archival Storage: Fix TestBlockPlacement and TestStorageMover

2014-09-11 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-7034: Attachment: HDFS-7034.002.patch Thanks for the review, Nicholas! Update the patch to address your comments.

[jira] [Updated] (HDFS-7045) Fix NameNode deadlock when opening file under /.reserved path

2014-09-11 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-7045: -- Resolution: Fixed Fix Version/s: 2.6.0 Status: Resolved (was: Patch Available)

[jira] [Commented] (HDFS-6194) Create new tests for ByteRangeInputStream

2014-09-11 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130424#comment-14130424 ] Haohui Mai commented on HDFS-6194: -- Sure. Please feel free to create a jira to track the

[jira] [Created] (HDFS-7049) TestByteRangeInputStream.testPropagatedClose fails and throw NPE on branch-2

2014-09-11 Thread Juan Yu (JIRA)
Juan Yu created HDFS-7049: - Summary: TestByteRangeInputStream.testPropagatedClose fails and throw NPE on branch-2 Key: HDFS-7049 URL: https://issues.apache.org/jira/browse/HDFS-7049 Project: Hadoop HDFS

[jira] [Commented] (HDFS-6982) nntop: topĀ­-like tool for name node users

2014-09-11 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130474#comment-14130474 ] Andrew Wang commented on HDFS-6982: --- Sorry for coming late to this discussion. Big +1

[jira] [Updated] (HDFS-7042) Upgrade fails for Windows HA cluster due to file locks held during rename in JournalNode.

2014-09-11 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HDFS-7042: Resolution: Fixed Fix Version/s: 2.6.0 Status: Resolved (was: Patch Available) I

[jira] [Commented] (HDFS-7046) HA NN can NPE upon transition to active

2014-09-11 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130565#comment-14130565 ] Daryn Sharp commented on HDFS-7046: --- Looks like the incremental tracking of block that is

[jira] [Created] (HDFS-7050) Implementation of NameNodeMXBean.getLiveNodes() skips DataNodes started on the same host

2014-09-11 Thread Przemyslaw Pretki (JIRA)
Przemyslaw Pretki created HDFS-7050: --- Summary: Implementation of NameNodeMXBean.getLiveNodes() skips DataNodes started on the same host Key: HDFS-7050 URL: https://issues.apache.org/jira/browse/HDFS-7050

[jira] [Commented] (HDFS-4946) Allow preferLocalNode in BlockPlacementPolicyDefault to be configurable

2014-09-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130668#comment-14130668 ] Hadoop QA commented on HDFS-4946: - {color:red}-1 overall{color}. Here are the results of

[jira] [Resolved] (HDFS-7034) Archival Storage: Fix TestBlockPlacement and TestStorageMover

2014-09-11 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-7034. - Resolution: Fixed Hadoop Flags: Reviewed I've committed this. Archival Storage: Fix

[jira] [Commented] (HDFS-6965) NN continues to issue block locations for DNs with full disks

2014-09-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130712#comment-14130712 ] Hadoop QA commented on HDFS-6965: - {color:red}-1 overall{color}. Here are the results of

[jira] [Resolved] (HDFS-7039) Archival Storage: fix Balancer tests

2014-09-11 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-7039. - Resolution: Fixed Hadoop Flags: Reviewed +1. I've committed this. Archival Storage: fix Balancer

[jira] [Commented] (HDFS-5946) Webhdfs DN choosing code is flawed

2014-09-11 Thread Ming Ma (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130734#comment-14130734 ] Ming Ma commented on HDFS-5946: --- We ran into the similar issue Daryn mentioned. It seems

[jira] [Commented] (HDFS-6965) NN continues to issue block locations for DNs with full disks

2014-09-11 Thread Rushabh S Shah (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130763#comment-14130763 ] Rushabh S Shah commented on HDFS-6965: -- I ran all the above failed tests on my local

[jira] [Updated] (HDFS-6584) Support Archival Storage

2014-09-11 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-6584: Attachment: h6584_20140911b.patch Support Archival Storage Key:

[jira] [Commented] (HDFS-5946) Webhdfs DN choosing code is flawed

2014-09-11 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130778#comment-14130778 ] Haohui Mai commented on HDFS-5946: -- Looking at the branch-2 code, it looks to me that it

[jira] [Commented] (HDFS-7046) HA NN can NPE upon transition to active

2014-09-11 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130806#comment-14130806 ] Kihwal Lee commented on HDFS-7046: -- Incremental tracking is okay. Even during transition

[jira] [Commented] (HDFS-7046) HA NN can NPE upon transition to active

2014-09-11 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130841#comment-14130841 ] Kihwal Lee commented on HDFS-7046: -- Since the secret manager is the one who is trying to

[jira] [Created] (HDFS-7051) TestDataNodeRollingUpgrade#isBlockFileInPrevious assumes Unix file path separator.

2014-09-11 Thread Chris Nauroth (JIRA)
Chris Nauroth created HDFS-7051: --- Summary: TestDataNodeRollingUpgrade#isBlockFileInPrevious assumes Unix file path separator. Key: HDFS-7051 URL: https://issues.apache.org/jira/browse/HDFS-7051

[jira] [Commented] (HDFS-6880) Adding tracing to DataNode data transfer protocol

2014-09-11 Thread Nick Dimiduk (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130844#comment-14130844 ] Nick Dimiduk commented on HDFS-6880: bq. We need finer granularity. Does that make

[jira] [Updated] (HDFS-7051) TestDataNodeRollingUpgrade#isBlockFileInPrevious assumes Unix file path separator.

2014-09-11 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HDFS-7051: Status: Patch Available (was: Open) TestDataNodeRollingUpgrade#isBlockFileInPrevious assumes Unix

[jira] [Updated] (HDFS-7051) TestDataNodeRollingUpgrade#isBlockFileInPrevious assumes Unix file path separator.

2014-09-11 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HDFS-7051: Attachment: HDFS-7051.1.patch I'm attaching a patch. With this, the test passes for me on both Mac

[jira] [Commented] (HDFS-7051) TestDataNodeRollingUpgrade#isBlockFileInPrevious assumes Unix file path separator.

2014-09-11 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130853#comment-14130853 ] Haohui Mai commented on HDFS-7051: -- +1 TestDataNodeRollingUpgrade#isBlockFileInPrevious

[jira] [Commented] (HDFS-7051) TestDataNodeRollingUpgrade#isBlockFileInPrevious assumes Unix file path separator.

2014-09-11 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130856#comment-14130856 ] Arpit Agarwal commented on HDFS-7051: - +1, thanks for fixing this Chris.

[jira] [Updated] (HDFS-7035) Refactor DataStorage and BlockSlicePoolStorage

2014-09-11 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HDFS-7035: Attachment: HDFS-7035.002.patch Rework against trunk directly. Refactor DataStorage and

[jira] [Updated] (HDFS-6867) For DFSOutputStream, do pipeline recovery for a single block in the background

2014-09-11 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-6867: Attachment: HDFS-6867-20140911.patch I figured out that {{stage}} is set to the correct value

[jira] [Commented] (HDFS-6584) Support Archival Storage

2014-09-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130934#comment-14130934 ] Hadoop QA commented on HDFS-6584: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-7045) Fix NameNode deadlock when opening file under /.reserved path

2014-09-11 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130950#comment-14130950 ] Yi Liu commented on HDFS-7045: -- Thanks [~andrew.wang] for review and committing. Fix

[jira] [Updated] (HDFS-6808) Add command line option to ask DataNode reload configuration.

2014-09-11 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HDFS-6808: Attachment: HDFS-7035.002.patch [~cmccabe] Thank you very much for your suggestions! I have updated

[jira] [Updated] (HDFS-6808) Add command line option to ask DataNode reload configuration.

2014-09-11 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HDFS-6808: Attachment: (was: HDFS-7035.002.patch) Add command line option to ask DataNode reload

[jira] [Updated] (HDFS-6808) Add command line option to ask DataNode reload configuration.

2014-09-11 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HDFS-6808: Attachment: HDFS-6808.002.patch Add command line option to ask DataNode reload configuration.

[jira] [Updated] (HDFS-6808) Add command line option to ask DataNode reload configuration.

2014-09-11 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HDFS-6808: Attachment: (was: HDFS-6808.002.patch) Add command line option to ask DataNode reload

[jira] [Updated] (HDFS-6808) Add command line option to ask DataNode reload configuration.

2014-09-11 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HDFS-6808: Attachment: HDFS-6808.003.patch Add command line option to ask DataNode reload configuration.

[jira] [Updated] (HDFS-6808) Add command line option to ask DataNode reload configuration.

2014-09-11 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HDFS-6808: Attachment: HDFS-6808.003.combo.txt Add command line option to ask DataNode reload configuration.

[jira] [Commented] (HDFS-7040) HDFS dangerously uses @Beta methods from very old versions of Guava

2014-09-11 Thread Christopher Tubbs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130970#comment-14130970 ] Christopher Tubbs commented on HDFS-7040: - bq. .is there any easy way to switch off

[jira] [Commented] (HDFS-6584) Support Archival Storage

2014-09-11 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130975#comment-14130975 ] Andrew Wang commented on HDFS-6584: --- Hi all, Nice work here! A few of us here at

[jira] [Commented] (HDFS-7040) HDFS dangerously uses @Beta methods from very old versions of Guava

2014-09-11 Thread Christopher Tubbs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130979#comment-14130979 ] Christopher Tubbs commented on HDFS-7040: - To the comment about bumping the Guava

[jira] [Commented] (HDFS-7051) TestDataNodeRollingUpgrade#isBlockFileInPrevious assumes Unix file path separator.

2014-09-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131009#comment-14131009 ] Hadoop QA commented on HDFS-7051: - {color:red}-1 overall{color}. Here are the results of

[jira] [Updated] (HDFS-6779) hdfs version subcommand is missing

2014-09-11 Thread Akira AJISAKA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated HDFS-6779: Labels: newbie scripts (was: scripts) hdfs version subcommand is missing

[jira] [Commented] (HDFS-6867) For DFSOutputStream, do pipeline recovery for a single block in the background

2014-09-11 Thread Colin Patrick McCabe (JIRA)
, HDFS-6867-20140828-2.patch, HDFS-6867-20140910.patch, HDFS-6867-20140911.patch, HDFS-6867-design-20140820.pdf, HDFS-6867-design-20140821.pdf, HDFS-6867-design-20140822.pdf, HDFS-6867-design-20140827.pdf, HDFS-6867-design-20140910.pdf For DFSOutputStream, we should be able to do pipeline

[jira] [Commented] (HDFS-7035) Refactor DataStorage and BlockSlicePoolStorage

2014-09-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131028#comment-14131028 ] Hadoop QA commented on HDFS-7035: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-7001) Tests in TestTracing depends on the order of execution

2014-09-11 Thread Akira AJISAKA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131037#comment-14131037 ] Akira AJISAKA commented on HDFS-7001: - Hi [~iwasakims], {code} +

[jira] [Commented] (HDFS-6982) nntop: topĀ­-like tool for name node users

2014-09-11 Thread Maysam Yabandeh (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131039#comment-14131039 ] Maysam Yabandeh commented on HDFS-6982: --- Thanks [~andrew.wang]. Yeah the main reason

[jira] [Commented] (HDFS-6808) Add command line option to ask DataNode reload configuration.

2014-09-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131071#comment-14131071 ] Hadoop QA commented on HDFS-6808: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-6867) For DFSOutputStream, do pipeline recovery for a single block in the background

2014-09-11 Thread Hadoop QA (JIRA)
of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12668215/HDFS-6867-20140911.patch against trunk revision 17ffbe0. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears

[jira] [Updated] (HDFS-7051) TestDataNodeRollingUpgrade#isBlockFileInPrevious assumes Unix file path separator.

2014-09-11 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HDFS-7051: Resolution: Fixed Fix Version/s: 2.6.0 Hadoop Flags: Reviewed Status: Resolved

[jira] [Commented] (HDFS-6584) Support Archival Storage

2014-09-11 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131142#comment-14131142 ] Jing Zhao commented on HDFS-6584: - Thanks a lot for the great comments, [~andrew.wang]! Let