[jira] [Commented] (HDFS-9063) Correctly handle snapshot path for getContentSummary

2015-09-17 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791678#comment-14791678 ] Yi Liu commented on HDFS-9063: -- Thanks [~jingzhao] for working on this. Jenkins' report has issue,

[jira] [Updated] (HDFS-9089) Balancer and Mover should use ".system" as reserved inode name instead of "system"

2015-09-17 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-9089: -- Hadoop Flags: Incompatible change This looks like an incompatible change. > Balancer

[jira] [Commented] (HDFS-9040) Erasure coding: Refactor DFSStripedOutputStream (Move Namenode RPC Requests to Coordinator)

2015-09-17 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791655#comment-14791655 ] Zhe Zhang commented on HDFS-9040: - Thanks Jing for the new patch! The structure looks much cleaner now.

[jira] [Commented] (HDFS-9022) Move NameNode.getAddress() and NameNode.getUri() to hadoop-hdfs-client

2015-09-17 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791677#comment-14791677 ] Mingliang Liu commented on HDFS-9022: - The only failing test {{TestReplaceDatanodeOnFailure}} fails

[jira] [Updated] (HDFS-8341) (Summary & Description may be invalid) HDFS mover stuck in loop after failing to move block, doesn't move rest of blocks, can't get data back off decommissioning external

2015-09-17 Thread Surendra Singh Lilhore (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surendra Singh Lilhore updated HDFS-8341: - Assignee: (was: Surendra Singh Lilhore) > (Summary & Description may be

[jira] [Commented] (HDFS-9092) Nfs silently drops overlapping write requests, thus data copying can't complete

2015-09-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791684#comment-14791684 ] Hadoop QA commented on HDFS-9092: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Commented] (HDFS-9089) Balancer and Mover should use ".system" as reserved inode name instead of "system"

2015-09-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791692#comment-14791692 ] Hadoop QA commented on HDFS-9089: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Commented] (HDFS-9040) Erasure coding: Refactor DFSStripedOutputStream (Move Namenode RPC Requests to Coordinator)

2015-09-17 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791739#comment-14791739 ] Walter Su commented on HDFS-9040: - bq. should we just do updatePipeline when completing the block? 1. In

[jira] [Commented] (HDFS-9089) Balancer and Mover should use ".system" as reserved inode name instead of "system"

2015-09-17 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791760#comment-14791760 ] Tsz Wo Nicholas Sze commented on HDFS-9089: --- > Its better to make ".system" as reserved inode for

[jira] [Resolved] (HDFS-8341) (Summary & Description may be invalid) HDFS mover stuck in loop after failing to move block, doesn't move rest of blocks, can't get data back off decommissioning external

2015-09-17 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze resolved HDFS-8341. --- Resolution: Invalid Resolving as invalid. Please feel free to reopen if you disagree.

[jira] [Commented] (HDFS-8550) Erasure Coding: Fix FindBugs Multithreaded correctness Warning

2015-09-17 Thread Rakesh R (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14802814#comment-14802814 ] Rakesh R commented on HDFS-8550: Hi [~zhz], Attached patch to resolve the findbug warnings, would be great

[jira] [Issue Comment Deleted] (HDFS-9053) Support large directories efficiently using B-Tree

2015-09-17 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-9053: - Comment: was deleted (was: \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime ||

[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

2015-09-17 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14802729#comment-14802729 ] Yi Liu commented on HDFS-9053: -- The failed one test is not related to this patch. > Support large directories

[jira] [Commented] (HDFS-8341) (Summary & Description may be invalid) HDFS mover stuck in loop after failing to move block, doesn't move rest of blocks, can't get data back off decommissioning externa

2015-09-17 Thread Hari Sekhon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14802746#comment-14802746 ] Hari Sekhon commented on HDFS-8341: --- [~szetszwo] I believe this ticket is still valid: There were holes

[jira] [Reopened] (HDFS-8341) (Summary & Description may be invalid) HDFS mover stuck in loop after failing to move block, doesn't move rest of blocks, can't get data back off decommissioning external

2015-09-17 Thread Hari Sekhon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Sekhon reopened HDFS-8341: --- > (Summary & Description may be invalid) HDFS mover stuck in loop after failing > to move block, doesn't

[jira] [Updated] (HDFS-8341) HDFS mover stuck in loop trying to move corrupt block with no other valid replicas, doesn't move rest of other data blocks

2015-09-17 Thread Hari Sekhon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Sekhon updated HDFS-8341: -- Summary: HDFS mover stuck in loop trying to move corrupt block with no other valid replicas, doesn't

[jira] [Commented] (HDFS-8550) Erasure Coding: Fix FindBugs Multithreaded correctness Warning

2015-09-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14802743#comment-14802743 ] Hadoop QA commented on HDFS-8550: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Updated] (HDFS-8341) HDFS mover stuck in loop trying to move corrupt block with no other valid replicas, doesn't move rest of other data blocks, can't get data back off decommissioning externa

2015-09-17 Thread Hari Sekhon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Sekhon updated HDFS-8341: -- Summary: HDFS mover stuck in loop trying to move corrupt block with no other valid replicas, doesn't

[jira] [Commented] (HDFS-3107) HDFS truncate

2015-09-17 Thread Konstantin Shvachko (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791779#comment-14791779 ] Konstantin Shvachko commented on HDFS-3107: --- I would say absolutely go for it. As that was one of

[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

2015-09-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791829#comment-14791829 ] Hadoop QA commented on HDFS-9053: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Updated] (HDFS-8632) Erasure Coding: Add InterfaceAudience annotation to the erasure coding classes

2015-09-17 Thread Rakesh R (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated HDFS-8632: --- Status: Patch Available (was: Open) > Erasure Coding: Add InterfaceAudience annotation to the erasure coding

[jira] [Updated] (HDFS-8873) throttle directoryScanner

2015-09-17 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HDFS-8873: --- Attachment: HDFS-8873.003.patch Reposting patch. > throttle directoryScanner >

[jira] [Commented] (HDFS-8873) throttle directoryScanner

2015-09-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14803149#comment-14803149 ] Hadoop QA commented on HDFS-8873: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Commented] (HDFS-9095) RPC client should fail gracefully when the connection is timed out or reset

2015-09-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14803187#comment-14803187 ] Hadoop QA commented on HDFS-9095: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Updated] (HDFS-8632) Erasure Coding: Add InterfaceAudience annotation to the erasure coding classes

2015-09-17 Thread Rakesh R (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated HDFS-8632: --- Attachment: HDFS-8632-HDFS-7285-03.patch > Erasure Coding: Add InterfaceAudience annotation to the erasure

[jira] [Commented] (HDFS-9022) Move NameNode.getAddress() and NameNode.getUri() to hadoop-hdfs-client

2015-09-17 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14803170#comment-14803170 ] Haohui Mai commented on HDFS-9022: -- The patch looks good to me. bq. This is expected as we will file a

[jira] [Commented] (HDFS-9093) Initialize protobuf fields in RemoteBlockReaderTest

2015-09-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14803192#comment-14803192 ] Hadoop QA commented on HDFS-9093: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Updated] (HDFS-9095) RPC client should fail gracefully when the connection is timed out or reset

2015-09-17 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-9095: - Status: Patch Available (was: Open) > RPC client should fail gracefully when the connection is timed out

[jira] [Updated] (HDFS-9093) Initialize protobuf fields in RemoteBlockReaderTest

2015-09-17 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-9093: - Status: Patch Available (was: Open) > Initialize protobuf fields in RemoteBlockReaderTest >

[jira] [Commented] (HDFS-8632) Erasure Coding: Add InterfaceAudience annotation to the erasure coding classes

2015-09-17 Thread Rakesh R (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14803173#comment-14803173 ] Rakesh R commented on HDFS-8632: It seems there are few [findbug warnings on LocatedStripedBlock

[jira] [Commented] (HDFS-8873) throttle directoryScanner

2015-09-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14802910#comment-14802910 ] Kihwal Lee commented on HDFS-8873: -- Shall we target 2.7.2? > throttle directoryScanner >

[jira] [Commented] (HDFS-8632) Erasure Coding: Add InterfaceAudience annotation to the erasure coding classes

2015-09-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14802949#comment-14802949 ] Hadoop QA commented on HDFS-8632: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Updated] (HDFS-8873) throttle directoryScanner

2015-09-17 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HDFS-8873: --- Attachment: (was: HDFS-8873.003.patch) > throttle directoryScanner >

[jira] [Created] (HDFS-9097) Erasure coding: update EC command "-s" flag to "-p" when specifying policy

2015-09-17 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-9097: --- Summary: Erasure coding: update EC command "-s" flag to "-p" when specifying policy Key: HDFS-9097 URL: https://issues.apache.org/jira/browse/HDFS-9097 Project: Hadoop HDFS

[jira] [Commented] (HDFS-8647) Abstract BlockManager's rack policy into BlockPlacementPolicy

2015-09-17 Thread Brahma Reddy Battula (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14803246#comment-14803246 ] Brahma Reddy Battula commented on HDFS-8647: {{TestBlocksWithNotEnoughRacks}} are failing due

[jira] [Updated] (HDFS-9097) Erasure coding: update EC command "-s" flag to "-p" when specifying policy

2015-09-17 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-9097: Status: Patch Available (was: Open) > Erasure coding: update EC command "-s" flag to "-p" when specifying

[jira] [Updated] (HDFS-9097) Erasure coding: update EC command "-s" flag to "-p" when specifying policy

2015-09-17 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-9097: Attachment: HDFS-9097-HDFS-7285.00.patch > Erasure coding: update EC command "-s" flag to "-p" when

[jira] [Commented] (HDFS-9063) Correctly handle snapshot path for getContentSummary

2015-09-17 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14803241#comment-14803241 ] Jing Zhao commented on HDFS-9063: - Thanks Yi! {{getContentSummary(dir's current path)}} should include (all

[jira] [Updated] (HDFS-5802) NameNode does not check for inode type before traversing down a path

2015-09-17 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-5802: Attachment: (was: HDFS-5802.002.patch) > NameNode does not check for inode type before traversing down a

[jira] [Created] (HDFS-9098) Erasure coding: emulate race conditions among striped streamers in write pipeline

2015-09-17 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-9098: --- Summary: Erasure coding: emulate race conditions among striped streamers in write pipeline Key: HDFS-9098 URL: https://issues.apache.org/jira/browse/HDFS-9098 Project: Hadoop

[jira] [Commented] (HDFS-9040) Erasure coding: Refactor DFSStripedOutputStream (Move Namenode RPC Requests to Coordinator)

2015-09-17 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14803316#comment-14803316 ] Zhe Zhang commented on HDFS-9040: - bq. 3. We might want to add the logic to replace a failed

[jira] [Commented] (HDFS-9089) Balancer and Mover should use ".system" as reserved inode name instead of "system"

2015-09-17 Thread Surendra Singh Lilhore (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14803367#comment-14803367 ] Surendra Singh Lilhore commented on HDFS-9089: -- bq. Why ".system" is better than "system"? The

[jira] [Commented] (HDFS-8632) Erasure Coding: Add InterfaceAudience annotation to the erasure coding classes

2015-09-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14803375#comment-14803375 ] Hadoop QA commented on HDFS-8632: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Updated] (HDFS-5802) NameNode does not check for inode type before traversing down a path

2015-09-17 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-5802: Status: Patch Available (was: Open) > NameNode does not check for inode type before traversing down a path

[jira] [Updated] (HDFS-5802) NameNode does not check for inode type before traversing down a path

2015-09-17 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-5802: Attachment: HDFS-5802.003.patch Thanks Yongjun for the additional comments! I have fixed it and uploaded

[jira] [Commented] (HDFS-7492) If multiple threads call FsVolumeList#checkDirs at the same time, we should only do checkDirs once and give the results to all waiting threads

2015-09-17 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14803419#comment-14803419 ] Colin Patrick McCabe commented on HDFS-7492: [~eclark], also check HDFS-8845 for another

[jira] [Commented] (HDFS-9063) Correctly handle snapshot path for getContentSummary

2015-09-17 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14803250#comment-14803250 ] Jing Zhao commented on HDFS-9063: - The test failures look suspicious. I just triggered the Jenkins again.

[jira] [Commented] (HDFS-8808) dfs.image.transfer.bandwidthPerSec should not apply to -bootstrapStandby

2015-09-17 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14803321#comment-14803321 ] Zhe Zhang commented on HDFS-8808: - Triggering Jenkins again. > dfs.image.transfer.bandwidthPerSec should

[jira] [Commented] (HDFS-9089) Balancer and Mover should use ".system" as reserved inode name instead of "system"

2015-09-17 Thread Surendra Singh Lilhore (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14803332#comment-14803332 ] Surendra Singh Lilhore commented on HDFS-9089: -- Thanks [~szetszwo] for comment. The main

[jira] [Commented] (HDFS-8855) Webhdfs client leaks active NameNode connections

2015-09-17 Thread Owen O'Malley (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14803351#comment-14803351 ] Owen O'Malley commented on HDFS-8855: - I'm looking at the patch, but you'll need to resolve the

[jira] [Commented] (HDFS-8968) New benchmark throughput tool for striping erasure coding

2015-09-17 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14803386#comment-14803386 ] Andrew Wang commented on HDFS-8968: --- In that case maybe we throw it in hadoop-tools? The only concern

[jira] [Commented] (HDFS-9097) Erasure coding: update EC command "-s" flag to "-p" when specifying policy

2015-09-17 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14803396#comment-14803396 ] Andrew Wang commented on HDFS-9097: --- +1 LGTM, thanks for the update Zhe > Erasure coding: update EC

[jira] [Commented] (HDFS-5802) NameNode does not check for inode type before traversing down a path

2015-09-17 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14803388#comment-14803388 ] Yongjun Zhang commented on HDFS-5802: - Hi [~xiaochen], Thanks for the new rev. Rev 2 looks good to me,

[jira] [Updated] (HDFS-9098) Erasure coding: emulate race conditions among striped streamers in write pipeline

2015-09-17 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-9098: Description: Apparently the interleaving of events among {{StripedDataStreamer}}'s is very tricky to

[jira] [Updated] (HDFS-9088) Cleanup erasure coding documentation

2015-09-17 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-9088: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-7285 Status: Resolved

[jira] [Updated] (HDFS-5802) NameNode does not check for inode type before traversing down a path

2015-09-17 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-5802: Attachment: HDFS-5802.002.patch Thanks Yongjun for the review! It makes sense, and I've addressed comment

[jira] [Updated] (HDFS-5802) NameNode does not check for inode type before traversing down a path

2015-09-17 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-5802: Status: Open (was: Patch Available) > NameNode does not check for inode type before traversing down a path

[jira] [Updated] (HDFS-5802) NameNode does not check for inode type before traversing down a path

2015-09-17 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-5802: Attachment: HDFS-5802.002.patch > NameNode does not check for inode type before traversing down a path >

[jira] [Commented] (HDFS-9004) Add upgrade domain to DatanodeInfo

2015-09-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14803271#comment-14803271 ] Hadoop QA commented on HDFS-9004: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Updated] (HDFS-5802) NameNode does not check for inode type before traversing down a path

2015-09-17 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-5802: Status: Open (was: Patch Available) > NameNode does not check for inode type before traversing down a path

[jira] [Commented] (HDFS-8808) dfs.image.transfer.bandwidthPerSec should not apply to -bootstrapStandby

2015-09-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14804288#comment-14804288 ] Hadoop QA commented on HDFS-8808: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Updated] (HDFS-5802) NameNode does not check for inode type before traversing down a path

2015-09-17 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-5802: Status: Patch Available (was: Open) > NameNode does not check for inode type before traversing down a path

[jira] [Commented] (HDFS-3107) HDFS truncate

2015-09-17 Thread Constantine Peresypkin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14803234#comment-14803234 ] Constantine Peresypkin commented on HDFS-3107: -- Hmm, fast patching nfs did not work.

[jira] [Updated] (HDFS-5802) NameNode does not check for inode type before traversing down a path

2015-09-17 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-5802: Status: Patch Available (was: Open) > NameNode does not check for inode type before traversing down a path

[jira] [Commented] (HDFS-9085) Show renewer information in DelegationTokenIdentifier#toString

2015-09-17 Thread zhihai xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14803307#comment-14803307 ] zhihai xu commented on HDFS-9085: - Thanks for the review [~cnauroth]! That is great information. Yes, it

[jira] [Commented] (HDFS-9022) Move NameNode.getAddress() and NameNode.getUri() to hadoop-hdfs-client

2015-09-17 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14803300#comment-14803300 ] Mingliang Liu commented on HDFS-9022: - Thank you [~wheat9] for reviewing the code. I filed jira

[jira] [Commented] (HDFS-8873) throttle directoryScanner

2015-09-17 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14804279#comment-14804279 ] Daniel Templeton commented on HDFS-8873: [sigh] Those tests pass for me locally, so can't say why

[jira] [Commented] (HDFS-9004) Add upgrade domain to DatanodeInfo

2015-09-17 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14804388#comment-14804388 ] Lei (Eddy) Xu commented on HDFS-9004: - Hi, [~mingma] You patch looks very good in general. It'd be

[jira] [Commented] (HDFS-7955) Improve naming of classes, methods, and variables related to block replication and recovery

2015-09-17 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14804298#comment-14804298 ] Andrew Wang commented on HDFS-7955: --- I noticed with HDFS-8899 the datanode config keys need some

[jira] [Commented] (HDFS-9047) deprecate libwebhdfs in branch-2; remove from trunk

2015-09-17 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14804319#comment-14804319 ] Colin Patrick McCabe commented on HDFS-9047: Like I said, I don't have any objection to

[jira] [Commented] (HDFS-9022) Move NameNode.getAddress() and NameNode.getUri() to hadoop-hdfs-client

2015-09-17 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14804459#comment-14804459 ] Haohui Mai commented on HDFS-9022: -- +1 > Move NameNode.getAddress() and NameNode.getUri() to

[jira] [Commented] (HDFS-9063) Correctly handle snapshot path for getContentSummary

2015-09-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14804482#comment-14804482 ] Hadoop QA commented on HDFS-9063: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Commented] (HDFS-8967) Create a BlockManagerLock class to represent the lock used in the BlockManager

2015-09-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14804485#comment-14804485 ] Hadoop QA commented on HDFS-8967: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Updated] (HDFS-9086) Rename dfs.datanode.stripedread.threshold.millis to dfs.datanode.stripedread.timeout.millis

2015-09-17 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-9086: -- Attachment: hdfs-9086-hdfs-7285.001.patch Patch attached doing this rename > Rename

[jira] [Updated] (HDFS-5802) NameNode does not check for inode type before traversing down a path

2015-09-17 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-5802: Status: Open (was: Patch Available) > NameNode does not check for inode type before traversing down a path

[jira] [Updated] (HDFS-9097) Erasure coding: update EC command "-s" flag to "-p" when specifying policy

2015-09-17 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-9097: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-7285 Status: Resolved

[jira] [Commented] (HDFS-7037) Using distcp to copy data from insecure to secure cluster via hftp doesn't work (branch-2 only)

2015-09-17 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14804594#comment-14804594 ] Haohui Mai commented on HDFS-7037: -- bq. adding this capability to HFTP does not change the security

[jira] [Commented] (HDFS-9086) Rename dfs.datanode.stripedread.threshold.millis to dfs.datanode.stripedread.timeout.millis

2015-09-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14804604#comment-14804604 ] Hadoop QA commented on HDFS-9086: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Updated] (HDFS-5802) NameNode does not check for inode type before traversing down a path

2015-09-17 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-5802: Attachment: HDFS-5802.004.patch Upload patch 004 to fix checkstyle warnings. The whitespace error is not

[jira] [Commented] (HDFS-7037) Using distcp to copy data from insecure to secure cluster via hftp doesn't work (branch-2 only)

2015-09-17 Thread Aaron T. Myers (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14804611#comment-14804611 ] Aaron T. Myers commented on HDFS-7037: -- bq. Please correct me if I misunderstood. (1) The current

[jira] [Updated] (HDFS-9086) Rename dfs.datanode.stripedread.threshold.millis to dfs.datanode.stripedread.timeout.millis

2015-09-17 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-9086: -- Attachment: HDFS-9086-HDFS-7285.001.patch Thanks for reviewing Zhe, attaching same patch with

[jira] [Commented] (HDFS-5802) NameNode does not check for inode type before traversing down a path

2015-09-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14804495#comment-14804495 ] Hadoop QA commented on HDFS-5802: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Created] (HDFS-9099) Move DistributedFileSystem to hadoop-hdfs-client

2015-09-17 Thread Mingliang Liu (JIRA)
Mingliang Liu created HDFS-9099: --- Summary: Move DistributedFileSystem to hadoop-hdfs-client Key: HDFS-9099 URL: https://issues.apache.org/jira/browse/HDFS-9099 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-8873) throttle directoryScanner

2015-09-17 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14804503#comment-14804503 ] Colin Patrick McCabe commented on HDFS-8873: The jenkins errors look like: {code}

[jira] [Updated] (HDFS-9080) update htrace version to 4.0

2015-09-17 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-9080: --- Attachment: HDFS-9080.004.patch > update htrace version to 4.0 >

[jira] [Commented] (HDFS-8696) Reduce the variances of latency of WebHDFS

2015-09-17 Thread Bob Hansen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14804558#comment-14804558 ] Bob Hansen commented on HDFS-8696: -- Testing the latest patch as part of a full Hadoop build (rather than

[jira] [Commented] (HDFS-8550) Erasure Coding: Fix FindBugs Multithreaded correctness Warning

2015-09-17 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14804556#comment-14804556 ] Zhe Zhang commented on HDFS-8550: - Thanks Rakesh! The fixes look good except for the following questions /

[jira] [Commented] (HDFS-9004) Add upgrade domain to DatanodeInfo

2015-09-17 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14804634#comment-14804634 ] Lei (Eddy) Xu commented on HDFS-9004: - +1 pending jenkins. > Add upgrade domain to DatanodeInfo >

[jira] [Commented] (HDFS-9022) Move NameNode.getAddress() and NameNode.getUri() to hadoop-hdfs-client

2015-09-17 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14804651#comment-14804651 ] Hudson commented on HDFS-9022: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2350 (See

[jira] [Updated] (HDFS-9022) Move NameNode.getAddress() and NameNode.getUri() to hadoop-hdfs-client

2015-09-17 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-9022: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved

[jira] [Updated] (HDFS-9022) Move NameNode.getAddress() and NameNode.getUri() to hadoop-hdfs-client

2015-09-17 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-9022: - Component/s: (was: namenode) > Move NameNode.getAddress() and NameNode.getUri() to hadoop-hdfs-client

[jira] [Updated] (HDFS-9086) Rename dfs.datanode.stripedread.threshold.millis to dfs.datanode.stripedread.timeout.millis

2015-09-17 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-9086: -- Status: Patch Available (was: Open) > Rename dfs.datanode.stripedread.threshold.millis to >

[jira] [Commented] (HDFS-9097) Erasure coding: update EC command "-s" flag to "-p" when specifying policy

2015-09-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14804539#comment-14804539 ] Hadoop QA commented on HDFS-9097: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Commented] (HDFS-8855) Webhdfs client leaks active NameNode connections

2015-09-17 Thread Owen O'Malley (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14804547#comment-14804547 ] Owen O'Malley commented on HDFS-8855: - A few points: * You need to use the Token.getKind(),

[jira] [Commented] (HDFS-9022) Move NameNode.getAddress() and NameNode.getUri() to hadoop-hdfs-client

2015-09-17 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14804564#comment-14804564 ] Hudson commented on HDFS-9022: -- FAILURE: Integrated in Hadoop-trunk-Commit #8472 (See

[jira] [Commented] (HDFS-9086) Rename dfs.datanode.stripedread.threshold.millis to dfs.datanode.stripedread.timeout.millis

2015-09-17 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14804608#comment-14804608 ] Zhe Zhang commented on HDFS-9086: - Thanks Andrew, the patch LGTM. I think Jenkins tried applying it on

[jira] [Updated] (HDFS-9004) Add upgrade domain to DatanodeInfo

2015-09-17 Thread Ming Ma (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated HDFS-9004: -- Attachment: HDFS-9004-3.patch Thanks [~eddyxu]! Here is the updated patch that addresses your comment. The

[jira] [Updated] (HDFS-5802) NameNode does not check for inode type before traversing down a path

2015-09-17 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-5802: Status: Patch Available (was: Open) > NameNode does not check for inode type before traversing down a path

[jira] [Commented] (HDFS-9092) Nfs silently drops overlapping write requests, thus data copying can't complete

2015-09-17 Thread Brandon Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14804667#comment-14804667 ] Brandon Li commented on HDFS-9092: -- Thank you, [~yzhangal] for the patch. Could you roughly describe the

[jira] [Commented] (HDFS-9040) Erasure coding: Refactor DFSStripedOutputStream (Move Namenode RPC Requests to Coordinator)

2015-09-17 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14803339#comment-14803339 ] Jing Zhao commented on HDFS-9040: - I think the key point here is to bump the GS, which is necessary to

  1   2   >