[jira] [Created] (HDFS-7359) NameNode in secured HA cluster fails to start if dfs.namenode.secondary.http-address cannot be interpreted as a network address.

2014-11-04 Thread Chris Nauroth (JIRA)
Chris Nauroth created HDFS-7359: --- Summary: NameNode in secured HA cluster fails to start if dfs.namenode.secondary.http-address cannot be interpreted as a network address. Key: HDFS-7359 URL: https://issues.apache.o

[jira] [Commented] (HDFS-7347) Configurable erasure coding policy for individual files and directories

2014-11-04 Thread Vinayakumar B (JIRA)
t; Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Zhe Zhang >Assignee: Zhe Zhang > Attachments: HDFS-7347-20141104.patch > > > HDFS users and admins should be able to turn on and off erasure coding for > individual files

[jira] [Updated] (HDFS-7347) Configurable erasure coding policy for individual files and directories

2014-11-04 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-7347: Attachment: HDFS-7347-20141104.patch This patch adds a new policy to the current policy spectrum. While we

[jira] [Updated] (HDFS-7347) Configurable erasure coding policy for individual files and directories

2014-11-04 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-7347: Status: Patch Available (was: Open) > Configurable erasure coding policy for individual files and directorie

[jira] [Updated] (HDFS-7279) Use netty to implement DatanodeWebHdfsMethods

2014-11-04 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-7279: - Attachment: HDFS-7279.006.patch > Use netty to implement DatanodeWebHdfsMethods > -

[jira] [Commented] (HDFS-7358) Clients may get stuck waiting when using ByteArrayManager

2014-11-04 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197613#comment-14197613 ] Hadoop QA commented on HDFS-7358: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (HDFS-7331) Add Datanode network counts to datanode jmx page

2014-11-04 Thread Aaron T. Myers (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197449#comment-14197449 ] Aaron T. Myers commented on HDFS-7331: -- bq. The size of the map needs to be bounded. A

[jira] [Commented] (HDFS-7279) Use netty to implement DatanodeWebHdfsMethods

2014-11-04 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197443#comment-14197443 ] Hadoop QA commented on HDFS-7279: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (HDFS-7335) Redundant checkOperation() in FSN.analyzeFileState()

2014-11-04 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197414#comment-14197414 ] Hadoop QA commented on HDFS-7335: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (HDFS-7331) Add Datanode network counts to datanode jmx page

2014-11-04 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197393#comment-14197393 ] Andrew Wang commented on HDFS-7331: --- Instead of a Map, couldn't we just use an object? Fa

[jira] [Updated] (HDFS-7357) FSNamesystem.checkFileProgress should log file path

2014-11-04 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-7357: - Resolution: Fixed Fix Version/s: 2.7.0 Hadoop Flags: Reviewed Status: Resolved (was:

[jira] [Updated] (HDFS-7056) Snapshot support for truncate

2014-11-04 Thread Konstantin Shvachko (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-7056: -- Attachment: HDFS-7056.patch (1) Done. (2) No duplicate blocks in InvalidateBlocks, becaus

[jira] [Commented] (HDFS-7017) Implement OutputStream for libhdfs3

2014-11-04 Thread Zhanwei Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197355#comment-14197355 ] Zhanwei Wang commented on HDFS-7017: Add log when lease renewer exit caused by OOM, add

[jira] [Updated] (HDFS-7017) Implement OutputStream for libhdfs3

2014-11-04 Thread Zhanwei Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhanwei Wang updated HDFS-7017: --- Attachment: HDFS-7017-pnative.004.patch > Implement OutputStream for libhdfs3 > ---

[jira] [Commented] (HDFS-7357) FSNamesystem.checkFileProgress should log file path

2014-11-04 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197334#comment-14197334 ] Haohui Mai commented on HDFS-7357: -- The failure is unrelated. I'll commit it shortly. > F

[jira] [Commented] (HDFS-7325) Prevent thundering herd problem in ByteArrayManager by using notify not notifyAll

2014-11-04 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197333#comment-14197333 ] Tsz Wo Nicholas Sze commented on HDFS-7325: --- Thanks for the advise about the thun

[jira] [Updated] (HDFS-7358) Clients may get stuck waiting when using ByteArrayManager

2014-11-04 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-7358: -- Attachment: h7358_20141104.patch Tried to change TestByteArrayManager but not able to rep

[jira] [Updated] (HDFS-7358) Clients may get stuck waiting when using ByteArrayManager

2014-11-04 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-7358: -- Status: Patch Available (was: Open) > Clients may get stuck waiting when using ByteArray

[jira] [Commented] (HDFS-7017) Implement OutputStream for libhdfs3

2014-11-04 Thread Zhanwei Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197308#comment-14197308 ] Zhanwei Wang commented on HDFS-7017: Unfortunately we are the 0.0001% users who disable

[jira] [Commented] (HDFS-7358) Clients may get stuck waiting when using ByteArrayManager

2014-11-04 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197288#comment-14197288 ] Tsz Wo Nicholas Sze commented on HDFS-7358: --- Since the clients still get stuck wi

[jira] [Commented] (HDFS-7358) Clients may get stuck waiting when using ByteArrayManager

2014-11-04 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197254#comment-14197254 ] stack commented on HDFS-7358: - With the patch in place, it takes longer to hit the stall and th

[jira] [Commented] (HDFS-7357) FSNamesystem.checkFileProgress should log file path

2014-11-04 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197207#comment-14197207 ] Hadoop QA commented on HDFS-7357: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (HDFS-7233) NN logs unnecessary org.apache.hadoop.hdfs.protocol.UnresolvedPathException

2014-11-04 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197190#comment-14197190 ] Hudson commented on HDFS-7233: -- FAILURE: Integrated in Hadoop-trunk-Commit #6441 (See [https:

[jira] [Updated] (HDFS-7233) NN logs unnecessary org.apache.hadoop.hdfs.protocol.UnresolvedPathException

2014-11-04 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-7233: Component/s: namenode > NN logs unnecessary org.apache.hadoop.hdfs.protocol.UnresolvedPathException > ---

[jira] [Updated] (HDFS-7233) NN logs unnecessary org.apache.hadoop.hdfs.protocol.UnresolvedPathException

2014-11-04 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-7233: Issue Type: Improvement (was: Bug) > NN logs unnecessary org.apache.hadoop.hdfs.protocol.UnresolvedPathExcep

[jira] [Updated] (HDFS-7233) NN logs unnecessary org.apache.hadoop.hdfs.protocol.UnresolvedPathException

2014-11-04 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-7233: Resolution: Fixed Fix Version/s: 2.6.0 Hadoop Flags: Reviewed Status: Resolved (was: P

[jira] [Commented] (HDFS-7276) Limit the number of byte arrays used by DFSOutputStream

2014-11-04 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197183#comment-14197183 ] stack commented on HDFS-7276: - I tried applying [~cmccabe] 's HDFS-7325 but to no avail. > Lim

[jira] [Updated] (HDFS-7358) Clients may get stuck waiting when using ByteArrayManager

2014-11-04 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-7358: -- Attachment: h7358_20141104_wait_timeout.patch [~stack], let's add a wait timeout to see i

[jira] [Commented] (HDFS-7233) NN logs unnecessary org.apache.hadoop.hdfs.protocol.UnresolvedPathException

2014-11-04 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197171#comment-14197171 ] Jing Zhao commented on HDFS-7233: - +1. I will commit the patch shortly. > NN logs unnecess

[jira] [Updated] (HDFS-7279) Use netty to implement DatanodeWebHdfsMethods

2014-11-04 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-7279: - Attachment: HDFS-7279.005.patch > Use netty to implement DatanodeWebHdfsMethods > -

[jira] [Updated] (HDFS-7358) Clients may get stuck waiting when using ByteArrayManager

2014-11-04 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-7358: -- Description: [~stack] reported that clients might get stuck waiting when using ByteArrayM

[jira] [Updated] (HDFS-7358) Clients may get stuck waiting when using ByteArrayManager

2014-11-04 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-7358: -- Description: [~stack] reported that clients might get stuck waiting when using ByteArrayM

[jira] [Commented] (HDFS-7233) NN logs unnecessary org.apache.hadoop.hdfs.protocol.UnresolvedPathException

2014-11-04 Thread Rushabh S Shah (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197153#comment-14197153 ] Rushabh S Shah commented on HDFS-7233: -- It is difficult to parse the log files so no t

[jira] [Commented] (HDFS-7233) NN logs unnecessary org.apache.hadoop.hdfs.protocol.UnresolvedPathException

2014-11-04 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197142#comment-14197142 ] Hadoop QA commented on HDFS-7233: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (HDFS-7276) Limit the number of byte arrays used by DFSOutputStream

2014-11-04 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197140#comment-14197140 ] Tsz Wo Nicholas Sze commented on HDFS-7276: --- [~stack], thanks for trying this fea

[jira] [Created] (HDFS-7358) Clients may get stuck waiting when using ByteArrayManager

2014-11-04 Thread Tsz Wo Nicholas Sze (JIRA)
Tsz Wo Nicholas Sze created HDFS-7358: - Summary: Clients may get stuck waiting when using ByteArrayManager Key: HDFS-7358 URL: https://issues.apache.org/jira/browse/HDFS-7358 Project: Hadoop HDFS

[jira] [Commented] (HDFS-7355) TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails on Windows, because we cannot deny access to the file owner.

2014-11-04 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197127#comment-14197127 ] Chris Nauroth commented on HDFS-7355: - Thank you for the reviews and the commit, Ming a

[jira] [Commented] (HDFS-7199) DFSOutputStream can silently drop data if DataStreamer crashes with a non-I/O exception

2014-11-04 Thread Rushabh S Shah (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197119#comment-14197119 ] Rushabh S Shah commented on HDFS-7199: -- 1. Both the tests runs successfully on my loca

[jira] [Commented] (HDFS-7276) Limit the number of byte arrays used by DFSOutputStream

2014-11-04 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197115#comment-14197115 ] stack commented on HDFS-7276: - This seems to be holding up progress: {code} "sync.0" #35 prio=5

[jira] [Commented] (HDFS-7355) TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails on Windows, because we cannot deny access to the file owner.

2014-11-04 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197105#comment-14197105 ] Hudson commented on HDFS-7355: -- FAILURE: Integrated in Hadoop-trunk-Commit #6440 (See [https:

[jira] [Updated] (HDFS-7355) TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails on Windows, because we cannot deny access to the file owner.

2014-11-04 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-7355: - Resolution: Fixed Fix Version/s: 2.6.0 Hadoop Flags: Reviewed Status: Resolved (was:

[jira] [Commented] (HDFS-7276) Limit the number of byte arrays used by DFSOutputStream

2014-11-04 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197089#comment-14197089 ] stack commented on HDFS-7276: - I was thinking it was because 20480 * 130, my packet size, was t

[jira] [Commented] (HDFS-7356) Use DirectoryListing.hasMore() directly in nfs

2014-11-04 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197083#comment-14197083 ] Hudson commented on HDFS-7356: -- FAILURE: Integrated in Hadoop-trunk-Commit #6439 (See [https:

[jira] [Updated] (HDFS-7356) Use DirectoryListing.hasMore() directly in nfs

2014-11-04 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-7356: Issue Type: Improvement (was: Bug) > Use DirectoryListing.hasMore() directly in nfs > --

[jira] [Updated] (HDFS-7356) Use DirectoryListing.hasMore() directly in nfs

2014-11-04 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-7356: Resolution: Fixed Fix Version/s: 2.7.0 Hadoop Flags: Reviewed Status: Resolved (was: P

[jira] [Updated] (HDFS-7356) Use DirectoryListing.hasMore() directly in nfs

2014-11-04 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-7356: Component/s: nfs > Use DirectoryListing.hasMore() directly in nfs > -

[jira] [Commented] (HDFS-7355) TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails on Windows, because we cannot deny access to the file owner.

2014-11-04 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197068#comment-14197068 ] Haohui Mai commented on HDFS-7355: -- The test failure is unrelated. I'll commit it shortly.

[jira] [Commented] (HDFS-7355) TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails on Windows, because we cannot deny access to the file owner.

2014-11-04 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197065#comment-14197065 ] Hadoop QA commented on HDFS-7355: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (HDFS-7314) Aborted DFSClient's impact on long running service like YARN

2014-11-04 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197061#comment-14197061 ] Hadoop QA commented on HDFS-7314: - {color:green}+1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-7334) Fix periodic failures of TestCheckpoint#testTooManyEditReplayFailures

2014-11-04 Thread Charles Lamb (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197060#comment-14197060 ] Charles Lamb commented on HDFS-7334: Thanks for the review and the commit [~wheat9]. >

[jira] [Commented] (HDFS-7276) Limit the number of byte arrays used by DFSOutputStream

2014-11-04 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197036#comment-14197036 ] stack commented on HDFS-7276: - A few comments on this nice new feature: I'd suggest you add a

[jira] [Commented] (HDFS-3107) HDFS truncate

2014-11-04 Thread Plamen Jeliazkov (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197027#comment-14197027 ] Plamen Jeliazkov commented on HDFS-3107: I checked FindBugs since Jenkins messed up

[jira] [Commented] (HDFS-7334) Fix periodic failures of TestCheckpoint#testTooManyEditReplayFailures

2014-11-04 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197011#comment-14197011 ] Hudson commented on HDFS-7334: -- SUCCESS: Integrated in Hadoop-trunk-Commit #6437 (See [https:

[jira] [Updated] (HDFS-7334) Fix periodic failures of TestCheckpoint#testTooManyEditReplayFailures

2014-11-04 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-7334: - Resolution: Fixed Fix Version/s: 2.6.0 Hadoop Flags: Reviewed Status: Resolved (was:

[jira] [Updated] (HDFS-7334) Fix periodic failures of TestCheckpoint#testTooManyEditReplayFailures

2014-11-04 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-7334: - Target Version/s: (was: 2.7.0) > Fix periodic failures of TestCheckpoint#testTooManyEditReplayFailures >

[jira] [Updated] (HDFS-7335) Redundant checkOperation() in FSN.analyzeFileState()

2014-11-04 Thread Milan Desai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Milan Desai updated HDFS-7335: -- Status: Open (was: Patch Available) > Redundant checkOperation() in FSN.analyzeFileState() > ---

[jira] [Updated] (HDFS-7335) Redundant checkOperation() in FSN.analyzeFileState()

2014-11-04 Thread Milan Desai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Milan Desai updated HDFS-7335: -- Status: Patch Available (was: Open) > Redundant checkOperation() in FSN.analyzeFileState() > ---

[jira] [Commented] (HDFS-7356) Use DirectoryListing.hasMore() directly in nfs

2014-11-04 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196967#comment-14196967 ] Hadoop QA commented on HDFS-7356: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (HDFS-7279) Use netty to implement DatanodeWebHdfsMethods

2014-11-04 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196968#comment-14196968 ] Hadoop QA commented on HDFS-7279: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (HDFS-7335) Redundant checkOperation() in FSN.analyzeFileState()

2014-11-04 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196969#comment-14196969 ] Hadoop QA commented on HDFS-7335: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (HDFS-7333) Improve log message in Storage.tryLock()

2014-11-04 Thread Konstantin Shvachko (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196954#comment-14196954 ] Konstantin Shvachko commented on HDFS-7333: --- Don't see any Java warnings in Stora

[jira] [Commented] (HDFS-7334) Fix periodic failures of TestCheckpoint#testTooManyEditReplayFailures

2014-11-04 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196944#comment-14196944 ] Haohui Mai commented on HDFS-7334: -- The test report does not have the failure. I'll commit

[jira] [Commented] (HDFS-7334) Fix periodic failures of TestCheckpoint#testTooManyEditReplayFailures

2014-11-04 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196934#comment-14196934 ] Hadoop QA commented on HDFS-7334: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (HDFS-7199) DFSOutputStream can silently drop data if DataStreamer crashes with a non-I/O exception

2014-11-04 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196933#comment-14196933 ] Hadoop QA commented on HDFS-7199: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Updated] (HDFS-7279) Use netty to implement DatanodeWebHdfsMethods

2014-11-04 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-7279: - Attachment: HDFS-7279.004.patch > Use netty to implement DatanodeWebHdfsMethods > -

[jira] [Updated] (HDFS-7335) Redundant checkOperation() in FSN.analyzeFileState()

2014-11-04 Thread Milan Desai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Milan Desai updated HDFS-7335: -- Attachment: HDFS-7335.patch New patch removes git diff prefix > Redundant checkOperation() in FSN.analyz

[jira] [Updated] (HDFS-7231) rollingupgrade needs some guard rails

2014-11-04 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HDFS-7231: --- Target Version/s: 2.6.0 > rollingupgrade needs some guard rails > -

[jira] [Commented] (HDFS-7295) Support arbitrary max expiration times for delegation token

2014-11-04 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196896#comment-14196896 ] Allen Wittenauer commented on HDFS-7295: FWIW, I *do* think the max lifespan should

[jira] [Commented] (HDFS-7056) Snapshot support for truncate

2014-11-04 Thread Konstantin Shvachko (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196891#comment-14196891 ] Konstantin Shvachko commented on HDFS-7056: --- > Actually Guo and I have finished t

[jira] [Commented] (HDFS-7355) TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails on Windows, because we cannot deny access to the file owner.

2014-11-04 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196855#comment-14196855 ] Haohui Mai commented on HDFS-7355: -- +1 pending jenkins. > TestDataNodeVolumeFailure#testU

[jira] [Commented] (HDFS-7355) TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails on Windows, because we cannot deny access to the file owner.

2014-11-04 Thread Ming Ma (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196849#comment-14196849 ] Ming Ma commented on HDFS-7355: --- Thanks, [~cnauroth]. The patch looks good. BTW, it seems som

[jira] [Commented] (HDFS-7357) FSNamesystem.checkFileProgress should log file path

2014-11-04 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196834#comment-14196834 ] Haohui Mai commented on HDFS-7357: -- +1 pending jenkins. > FSNamesystem.checkFileProgress

[jira] [Updated] (HDFS-7357) FSNamesystem.checkFileProgress should log file path

2014-11-04 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-7357: -- Attachment: h7357_20141104.patch h7357_20141104.patch: - add path and other info to the

[jira] [Updated] (HDFS-7357) FSNamesystem.checkFileProgress should log file path

2014-11-04 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-7357: -- Status: Patch Available (was: Open) > FSNamesystem.checkFileProgress should log file pat

[jira] [Commented] (HDFS-7356) Use DirectoryListing.hasMore() directly in nfs

2014-11-04 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196821#comment-14196821 ] Jing Zhao commented on HDFS-7356: - +1 pending Jenkins > Use DirectoryListing.hasMore() dir

[jira] [Updated] (HDFS-7356) Use DirectoryListing.hasMore() directly in nfs

2014-11-04 Thread Li Lu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Lu updated HDFS-7356: Status: Patch Available (was: Open) > Use DirectoryListing.hasMore() directly in nfs > -

[jira] [Updated] (HDFS-7356) Use DirectoryListing.hasMore() directly in nfs

2014-11-04 Thread Li Lu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Lu updated HDFS-7356: Attachment: HDFS-7356-110414.patch Hi [~wheat9], I've fixed this in my patch. If you have time please feel free to h

[jira] [Created] (HDFS-7357) FSNamesystem.checkFileProgress should log file path

2014-11-04 Thread Tsz Wo Nicholas Sze (JIRA)
Tsz Wo Nicholas Sze created HDFS-7357: - Summary: FSNamesystem.checkFileProgress should log file path Key: HDFS-7357 URL: https://issues.apache.org/jira/browse/HDFS-7357 Project: Hadoop HDFS

[jira] [Commented] (HDFS-7233) NN logs unnecessary org.apache.hadoop.hdfs.protocol.UnresolvedPathException

2014-11-04 Thread Rushabh S Shah (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196806#comment-14196806 ] Rushabh S Shah commented on HDFS-7233: -- All the tests passing on my local setup. > NN

[jira] [Assigned] (HDFS-7356) Use DirectoryListing.hasMore() directly in nfs

2014-11-04 Thread Li Lu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Lu reassigned HDFS-7356: --- Assignee: Li Lu > Use DirectoryListing.hasMore() directly in nfs > ---

[jira] [Created] (HDFS-7356) Use DirectoryListing.hasMore() directly in nfs

2014-11-04 Thread Haohui Mai (JIRA)
Haohui Mai created HDFS-7356: Summary: Use DirectoryListing.hasMore() directly in nfs Key: HDFS-7356 URL: https://issues.apache.org/jira/browse/HDFS-7356 Project: Hadoop HDFS Issue Type: Bug

[jira] [Commented] (HDFS-7335) Redundant checkOperation() in FSN.analyzeFileState()

2014-11-04 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196781#comment-14196781 ] Hadoop QA commented on HDFS-7335: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (HDFS-7355) TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails on Windows, because we cannot deny access to the file owner.

2014-11-04 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196734#comment-14196734 ] Chris Nauroth commented on HDFS-7355: - http://technet.microsoft.com/en-us/library/cc783

[jira] [Commented] (HDFS-7208) NN doesn't schedule replication when a DN storage fails

2014-11-04 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196752#comment-14196752 ] Chris Nauroth commented on HDFS-7208: - The new test cannot work correctly on Windows.

[jira] [Updated] (HDFS-7335) Redundant checkOperation() in FSN.analyzeFileState()

2014-11-04 Thread Milan Desai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Milan Desai updated HDFS-7335: -- Attachment: HDFS-7335.patch Attaching a patch that removes the checkOperation call in FSNamesystem.analy

[jira] [Updated] (HDFS-7355) TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails on Windows, because we cannot deny access to the file owner.

2014-11-04 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HDFS-7355: Attachment: HDFS-7355.1.patch The attached patch skips the test. > TestDataNodeVolumeFailure#testUnd

[jira] [Created] (HDFS-7355) TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails on Windows, because we cannot deny access to the file owner.

2014-11-04 Thread Chris Nauroth (JIRA)
Chris Nauroth created HDFS-7355: --- Summary: TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails on Windows, because we cannot deny access to the file owner. Key: HDFS-7355 URL: https://issues.apache.

[jira] [Updated] (HDFS-7355) TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails on Windows, because we cannot deny access to the file owner.

2014-11-04 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HDFS-7355: Status: Patch Available (was: Open) > TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure

[jira] [Updated] (HDFS-7335) Redundant checkOperation() in FSN.analyzeFileState()

2014-11-04 Thread Milan Desai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Milan Desai updated HDFS-7335: -- Status: Patch Available (was: Open) > Redundant checkOperation() in FSN.analyzeFileState() > ---

[jira] [Commented] (HDFS-7343) A comprehensive and flexible storage policy engine

2014-11-04 Thread Andrew Purtell (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196759#comment-14196759 ] Andrew Purtell commented on HDFS-7343: -- Most of the ideas mentioned in the description

[jira] [Updated] (HDFS-7056) Snapshot support for truncate

2014-11-04 Thread Plamen Jeliazkov (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Plamen Jeliazkov updated HDFS-7056: --- Attachment: HDFS-3107-HDFS-7056-combined.patch Attaching combined patch here as well. > Snapsh

[jira] [Commented] (HDFS-3107) HDFS truncate

2014-11-04 Thread Plamen Jeliazkov (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196716#comment-14196716 ] Plamen Jeliazkov commented on HDFS-3107: I will attach it to HDFS-7056 since it has

[jira] [Updated] (HDFS-7314) Aborted DFSClient's impact on long running service like YARN

2014-11-04 Thread Ming Ma (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated HDFS-7314: -- Attachment: HDFS-7314-2.patch Thanks, [~cmccabe]. I have updated the patch based on your suggestion. > Aborted D

[jira] [Commented] (HDFS-7199) DFSOutputStream can silently drop data if DataStreamer crashes with a non-I/O exception

2014-11-04 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1419#comment-1419 ] Colin Patrick McCabe commented on HDFS-7199: +1 pending jenkins > DFSOutputStr

[jira] [Commented] (HDFS-6982) nntop: top­-like tool for name node users

2014-11-04 Thread Maysam Yabandeh (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196664#comment-14196664 ] Maysam Yabandeh commented on HDFS-6982: --- Let me take an example. Time period: 60 sec

[jira] [Commented] (HDFS-7056) Snapshot support for truncate

2014-11-04 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196655#comment-14196655 ] Jing Zhao commented on HDFS-7056: - Thanks for working on this, [~shv] and [~zero45]. So far

[jira] [Commented] (HDFS-7334) Fix periodic failures of TestCheckpoint#testTooManyEditReplayFailures

2014-11-04 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196649#comment-14196649 ] Haohui Mai commented on HDFS-7334: -- +1 pending jenkins. > Fix periodic failures of TestCh

[jira] [Commented] (HDFS-6982) nntop: top­-like tool for name node users

2014-11-04 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196644#comment-14196644 ] Haohui Mai commented on HDFS-6982: -- bq. Before we start adding events to that (which will

[jira] [Updated] (HDFS-7334) Fix periodic failures of TestCheckpoint#testTooManyEditReplayFailures

2014-11-04 Thread Charles Lamb (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charles Lamb updated HDFS-7334: --- Attachment: HDFS-7334.002.patch [~wheat9], Thanks for the review! The .002 patch changes the set to s

[jira] [Commented] (HDFS-6982) nntop: top­-like tool for name node users

2014-11-04 Thread Maysam Yabandeh (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196632#comment-14196632 ] Maysam Yabandeh commented on HDFS-6982: --- Thanks [~wheat9] for the comment. Let me exp

  1   2   >