[jira] [Commented] (HDFS-12528) Short-circuit reads unnecessarily disabled for a long time

2018-01-28 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16343005#comment-16343005 ] John Zhuge commented on HDFS-12528: --- [~xiaochen], thanks for the great work! Patch 03 LG

[jira] [Commented] (HDFS-12528) Add an option to not disable short-circuit reads on failures

2018-01-29 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16344557#comment-16344557 ] John Zhuge commented on HDFS-12528: --- +1 LGTM > Add an option to not disable short-circu

[jira] [Assigned] (HDFS-240) Should HDFS restrict the names used for files?

2018-03-05 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge reassigned HDFS-240: --- Assignee: (was: John Zhuge) > Should HDFS restrict the names used for files? >

[jira] [Commented] (HDFS-240) Should HDFS restrict the names used for files?

2018-03-05 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16386758#comment-16386758 ] John Zhuge commented on HDFS-240: - Unfortunately don't have time to work on it. > Should HD

[jira] [Commented] (HDFS-13247) FileInputStream not closed when read the checksum header from the meta file

2018-03-07 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16390867#comment-16390867 ] John Zhuge commented on HDFS-13247: --- Please check out  [try-with-resources|https://docs.

[jira] [Commented] (HDFS-13984) getFileInfo of libhdfs call NameNode#getFileStatus twice

2018-11-02 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673339#comment-16673339 ] John Zhuge commented on HDFS-13984: --- [~yangjiandan]Thanks for the contribution! Could

[jira] [Work started] (HDFS-12528) Short-circuit reads getting disabled frequently in certain scenarios

2017-10-23 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-12528 started by John Zhuge. - > Short-circuit reads getting disabled frequently in certain scenarios > --

[jira] [Updated] (HDFS-12528) Short-circuit reads getting disabled frequently in certain scenarios

2017-10-23 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-12528: -- Attachment: HDFS-12528.000.patch Patch 000 * Unit test only Here is the test output. {{Unbuffer}} plac

[jira] [Comment Edited] (HDFS-12528) Short-circuit reads getting disabled frequently in certain scenarios

2017-10-23 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16216131#comment-16216131 ] John Zhuge edited comment on HDFS-12528 at 10/24/17 4:16 AM: -

[jira] [Commented] (HDFS-12528) Short-circuit reads getting disabled frequently in certain scenarios

2017-10-23 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16216277#comment-16216277 ] John Zhuge commented on HDFS-12528: --- The unit test is subtly different from the original

[jira] [Updated] (HDFS-3296) Running libhdfs tests in mac fails

2017-10-24 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-3296: - Attachment: HDFS-3296.004.patch I'd like to share a patch I have been using. Not totally polished or thorou

[jira] [Commented] (HDFS-3296) Running libhdfs tests in mac fails

2017-10-24 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16217817#comment-16217817 ] John Zhuge commented on HDFS-3296: -- TestDomainSocket#testAsyncCloseDuringWrite failure: {n

[jira] [Commented] (HDFS-12710) HTTPFS HTTP max header size env variable is not respected in branch-2

2017-10-25 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219130#comment-16219130 ] John Zhuge commented on HDFS-12710: --- [~pifta] Thanks for the contribution! Is this a bu

[jira] [Commented] (HDFS-12710) HTTPFS HTTP max header size env variable is not respected in branch-2

2017-10-25 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219380#comment-16219380 ] John Zhuge commented on HDFS-12710: --- Tested the patch in a pseudo distributed cluster. I

[jira] [Updated] (HDFS-12710) HTTPFS HTTP max header size env variable is not respected in branch-2

2017-10-26 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-12710: -- Resolution: Fixed Fix Version/s: 2.9.0 Status: Resolved (was: Patch Available) Commit

[jira] [Commented] (HDFS-12714) Hadoop 3 missing fix for HDFS-5169

2017-10-30 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16226037#comment-16226037 ] John Zhuge commented on HDFS-12714: --- [~joemcdonnell] Thanks for the contribution! How d

[jira] [Comment Edited] (HDFS-12714) Hadoop 3 missing fix for HDFS-5169

2017-10-30 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16226037#comment-16226037 ] John Zhuge edited comment on HDFS-12714 at 10/31/17 1:00 AM: -

[jira] [Updated] (HDFS-12744) More logs when short-circuit read is failed and disabled

2017-10-30 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-12744: -- Labels: supportability (was: ) > More logs when short-circuit read is failed and disabled > ---

[jira] [Commented] (HDFS-12744) More logs when short-circuit read is failed and disabled

2017-10-30 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16226256#comment-16226256 ] John Zhuge commented on HDFS-12744: --- +1 LGTM > More logs when short-circuit read is fai

[jira] [Commented] (HDFS-12714) Hadoop 3 missing fix for HDFS-5169

2017-10-31 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16227809#comment-16227809 ] John Zhuge commented on HDFS-12714: --- +1 LGTM I will commit once the testing done is cla

[jira] [Resolved] (HDFS-12714) Hadoop 3 missing fix for HDFS-5169

2017-10-31 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge resolved HDFS-12714. --- Resolution: Fixed Fix Version/s: 3.0.0-beta1 3.1.0 Committed to trunk and br

[jira] [Comment Edited] (HDFS-12714) Hadoop 3 missing fix for HDFS-5169

2017-10-31 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16233678#comment-16233678 ] John Zhuge edited comment on HDFS-12714 at 11/1/17 6:03 AM: Co

[jira] [Commented] (HDFS-12318) Fix IOException condition for openInfo in DFSInputStream

2017-11-09 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16246143#comment-16246143 ] John Zhuge commented on HDFS-12318: --- Backported to branch-2, branch-2.9, and branch-2.8.

[jira] [Updated] (HDFS-12318) Fix IOException condition for openInfo in DFSInputStream

2017-11-09 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-12318: -- Fix Version/s: 2.9.1 2.10.0 2.8.3 > Fix IOException condition for

[jira] [Updated] (HDFS-12821) Block invalid IOException causes the DFSClient domain socket being disabled

2017-11-15 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-12821: -- Description: We use HDFS2.4 & 2.6, and recently hit a issue that DFSClient domain socket is disabled wh

[jira] [Commented] (HDFS-12821) Block invalid IOException causes the DFSClient domain socket being disabled

2017-11-16 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16254904#comment-16254904 ] John Zhuge commented on HDFS-12821: --- Think so too. In HDFS-12528, the file was appended

[jira] [Resolved] (HDFS-12821) Block invalid IOException causes the DFSClient domain socket being disabled

2017-11-16 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge resolved HDFS-12821. --- Resolution: Duplicate > Block invalid IOException causes the DFSClient domain socket being disabled >

[jira] [Updated] (HDFS-12528) Short-circuit reads unnecessarily disabled for a long time

2017-11-16 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-12528: -- Summary: Short-circuit reads unnecessarily disabled for a long time (was: Short-circuit reads getting d

[jira] [Resolved] (HDFS-12815) Fix TestUnbuffer#testUnbufferException because of expected error message mismatch

2017-11-29 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge resolved HDFS-12815. --- Resolution: Duplicate Thanks [~msingh] for reporting the issue. I think it is a dup of HADOOP-15056 w

[jira] [Resolved] (HDFS-12604) StreamCapability enums are not displayed in javadoc

2017-11-29 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge resolved HDFS-12604. --- Resolution: Won't Fix StreamCapability enums are deprecated by HADOOP-15012. > StreamCapability enums

[jira] [Commented] (HDFS-12528) Short-circuit reads unnecessarily disabled for a long time

2017-12-20 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16299531#comment-16299531 ] John Zhuge commented on HDFS-12528: --- [~xiaochen] Puzzled. The hadoop-hdfs/pom.xml change

[jira] [Commented] (HDFS-12528) Short-circuit reads unnecessarily disabled for a long time

2017-12-20 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16299589#comment-16299589 ] John Zhuge commented on HDFS-12528: --- Yes, that would shorten the path greatly. > Short-

[jira] [Assigned] (HDFS-12528) Short-circuit reads unnecessarily disabled for a long time

2017-12-20 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge reassigned HDFS-12528: - Assignee: (was: John Zhuge) > Short-circuit reads unnecessarily disabled for a long time > --

[jira] [Commented] (HDFS-12528) Short-circuit reads unnecessarily disabled for a long time

2017-12-20 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16299614#comment-16299614 ] John Zhuge commented on HDFS-12528: --- Unfortunately I won't have time for a while. Unassi

[jira] [Commented] (HDFS-13752) fs.Path stores file path in java.net.URI causes big memory waste

2018-07-20 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16551109#comment-16551109 ] John Zhuge commented on HDFS-13752: --- cc [~mi...@cloudera.com] > fs.Path stores file pa

[jira] [Assigned] (HDFS-9841) FileDiff's skipped by hdfs snapshotDiff

2016-03-02 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge reassigned HDFS-9841: Assignee: John Zhuge > FileDiff's skipped by hdfs snapshotDiff > ---

[jira] [Assigned] (HDFS-9896) WebHDFS API may return invalid JSON

2016-03-03 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge reassigned HDFS-9896: Assignee: John Zhuge > WebHDFS API may return invalid JSON > --- > >

[jira] [Assigned] (HDFS-6520) Failed to run fsck -move

2016-03-03 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge reassigned HDFS-6520: Assignee: John Zhuge > Failed to run fsck -move > > > Key:

[jira] [Updated] (HDFS-6520) hdfs fsck -move not working

2016-03-03 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-6520: - Summary: hdfs fsck -move not working (was: Failed to run fsck -move) Was able to gain more insight after e

[jira] [Work started] (HDFS-6520) hdfs fsck -move not working

2016-03-03 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-6520 started by John Zhuge. > hdfs fsck -move not working > --- > > Key: HDFS-

[jira] [Commented] (HDFS-7639) Remove the limitation imposed by dfs.balancer.moverThreads

2016-03-03 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15179177#comment-15179177 ] John Zhuge commented on HDFS-7639: -- +1 Definitely wasteful to occupy a thread to send a re

[jira] [Updated] (HDFS-9913) DispCp doesn't use Trash with -delete option

2016-03-06 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9913: - Description: Documentation for DistCp -delete option says ([http://hadoop.apache.org/docs/current/hadoop-d

[jira] [Assigned] (HDFS-9913) DispCp doesn't use Trash with -delete option

2016-03-07 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge reassigned HDFS-9913: Assignee: John Zhuge > DispCp doesn't use Trash with -delete option > --

[jira] [Created] (HDFS-9915) Typo in DistCp.html

2016-03-07 Thread John Zhuge (JIRA)
John Zhuge created HDFS-9915: Summary: Typo in DistCp.html Key: HDFS-9915 URL: https://issues.apache.org/jira/browse/HDFS-9915 Project: Hadoop HDFS Issue Type: Improvement Components: d

[jira] [Commented] (HDFS-9913) DispCp doesn't use Trash with -delete option

2016-03-07 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15183607#comment-15183607 ] John Zhuge commented on HDFS-9913: -- The problem exists in CDH5.8.0 (2.6.0) based on my qui

[jira] [Assigned] (HDFS-9939) Possible performance improvement by increasing buf size in DecompressorStream in HDFS

2016-03-10 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge reassigned HDFS-9939: Assignee: John Zhuge (was: Yongjun Zhang) > Possible performance improvement by increasing buf siz

[jira] [Updated] (HDFS-9940) Rename dfs.balancer.max.concurrent.moves to avoid confusion

2016-03-10 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9940: - Summary: Rename dfs.balancer.max.concurrent.moves to avoid confusion (was: New property dfs.balancer.max.c

[jira] [Created] (HDFS-9940) New property dfs.balancer.max.concurrent.moves for Balancer

2016-03-10 Thread John Zhuge (JIRA)
John Zhuge created HDFS-9940: Summary: New property dfs.balancer.max.concurrent.moves for Balancer Key: HDFS-9940 URL: https://issues.apache.org/jira/browse/HDFS-9940 Project: Hadoop HDFS Issue

[jira] [Commented] (HDFS-9940) Rename dfs.balancer.max.concurrent.moves to avoid confusion

2016-03-15 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196363#comment-15196363 ] John Zhuge commented on HDFS-9940: -- +1 HDFS-7466 > Rename dfs.balancer.max.concurrent.mov

[jira] [Commented] (HDFS-9940) Rename dfs.balancer.max.concurrent.moves to avoid confusion

2016-03-15 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196482#comment-15196482 ] John Zhuge commented on HDFS-9940: -- Balancer uses config {{dfs.datanode.balance.max.concur

[jira] [Commented] (HDFS-9940) Rename dfs.balancer.max.concurrent.moves to avoid confusion

2016-03-15 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196705#comment-15196705 ] John Zhuge commented on HDFS-9940: -- [~yzhangal], great idea if we don't have to manually c

[jira] [Commented] (HDFS-9940) Rename dfs.balancer.max.concurrent.moves to avoid confusion

2016-03-15 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196712#comment-15196712 ] John Zhuge commented on HDFS-9940: -- I like 2) better because all rebalance configuration i

[jira] [Created] (HDFS-10171) Balancer should log config values and iteration termination reasons

2016-03-15 Thread John Zhuge (JIRA)
John Zhuge created HDFS-10171: - Summary: Balancer should log config values and iteration termination reasons Key: HDFS-10171 URL: https://issues.apache.org/jira/browse/HDFS-10171 Project: Hadoop HDFS

[jira] [Updated] (HDFS-10171) Balancer should log config values

2016-03-15 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-10171: -- Description: To improve supportability, Balancer should log config values. (was: To improve supportabil

[jira] [Resolved] (HDFS-10171) Balancer should log config values

2016-03-15 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge resolved HDFS-10171. --- Resolution: Duplicate Fix Version/s: 2.8.0 {noformat} 2016-03-15 22:42:31,618 [Thread-0] INFO

[jira] [Updated] (HDFS-9885) In distcp cmd ouput, Display name should be given for org.apache.hadoop.tools.mapred.CopyMapper$Counter.

2016-03-18 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9885: - Hadoop Flags: Reviewed +1 LGTM > In distcp cmd ouput, Display name should be given for > org.apache.hadoo

[jira] [Commented] (HDFS-10173) Typo in DataXceiverServer

2016-03-18 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15197765#comment-15197765 ] John Zhuge commented on HDFS-10173: --- +1 LGTM > Typo in DataXceiverServer >

[jira] [Commented] (HDFS-9885) In distcp cmd ouput, Display name should be given for org.apache.hadoop.tools.mapred.CopyMapper$Counter.

2016-03-19 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15200929#comment-15200929 ] John Zhuge commented on HDFS-9885: -- OK > In distcp cmd ouput, Display name should be give

[jira] [Commented] (HDFS-9940) Rename dfs.balancer.max.concurrent.moves to avoid confusion

2016-03-19 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15198864#comment-15198864 ] John Zhuge commented on HDFS-9940: -- Thank you all for the great discussion ! [~yzhangal]

[jira] [Commented] (HDFS-9885) In distcp cmd ouput, Display name should be given for org.apache.hadoop.tools.mapred.CopyMapper$Counter.

2016-03-19 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15200058#comment-15200058 ] John Zhuge commented on HDFS-9885: -- Thanks for the patch. Minor comments: * Replace "Distc

[jira] [Commented] (HDFS-9885) In distcp cmd ouput, Display name should be given for org.apache.hadoop.tools.mapred.CopyMapper$Counter.

2016-03-19 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15200092#comment-15200092 ] John Zhuge commented on HDFS-9885: -- My bad, it seems "_" is necessary because {{org.apach

[jira] [Commented] (HDFS-10185) TestHFlushInterrupted verifies interrupt state incorrectly

2016-03-20 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15203699#comment-15203699 ] John Zhuge commented on HDFS-10185: --- Thanks [~linyiqun] for working on the unit test iss

[jira] [Commented] (HDFS-10185) TestHFlushInterrupted verifies interrupt state incorrectly

2016-03-21 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15204427#comment-15204427 ] John Zhuge commented on HDFS-10185: --- bq. Yes, my new code is used to check if {{Thread.c

[jira] [Commented] (HDFS-10185) TestHFlushInterrupted verifies interrupt state incorrectly

2016-03-21 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15205713#comment-15205713 ] John Zhuge commented on HDFS-10185: --- Close it then. Thanks for attempting to fix this un

[jira] [Created] (HDFS-10190) Expose FSDatasetImpl lock metrics

2016-03-21 Thread John Zhuge (JIRA)
John Zhuge created HDFS-10190: - Summary: Expose FSDatasetImpl lock metrics Key: HDFS-10190 URL: https://issues.apache.org/jira/browse/HDFS-10190 Project: Hadoop HDFS Issue Type: Improvement A

[jira] [Updated] (HDFS-10190) Add DN FSDatasetImpl lock metrics

2016-03-22 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-10190: -- Description: Add DN FSDatasetImpl lock metrics: * Number of lock calls * Contention rate * Average wait

[jira] [Updated] (HDFS-9939) Increase DecompressorStream skip buffer size

2016-03-22 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9939: - Summary: Increase DecompressorStream skip buffer size (was: Possible performance improvement by increasin

[jira] [Updated] (HDFS-8496) Should not call stopWriter() with FSDatasetImpl lock held

2016-03-22 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-8496: - Summary: Should not call stopWriter() with FSDatasetImpl lock held (was: Calling stopWriter() with FSDatas

[jira] [Updated] (HDFS-8496) Calling stopWriter() with FSDatasetImpl lock held may block other threads

2016-03-23 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-8496: - Summary: Calling stopWriter() with FSDatasetImpl lock held may block other threads (was: Should not call s

[jira] [Commented] (HDFS-9633) Test SnapshotDiffReport file path corner cases

2016-03-23 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15209577#comment-15209577 ] John Zhuge commented on HDFS-9633: -- No, I am not trying fix any problem, just adding more

[jira] [Updated] (HDFS-9573) SnapshotDiffReport$DiffReportEntry$hashCode should use type field

2016-03-23 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9573: - Resolution: Won't Fix Status: Resolved (was: Patch Available) It is recommended for hashCode and e

[jira] [Updated] (HDFS-9939) Increase DecompressorStream skip buffer size

2016-03-23 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9939: - Fix Version/s: 2.8.0 Status: Patch Available (was: Open) > Increase DecompressorStream skip buf

[jira] [Updated] (HDFS-9939) Increase DecompressorStream skip buffer size

2016-03-23 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9939: - Attachment: HDFS-9939.001.patch Patch 001: * Set skip buffer size to 2048 (MAX_SKIP_BUFFER_SIZE) * Move Fak

[jira] [Created] (HDFS-10215) distcp throws vague exception without home directory

2016-03-25 Thread John Zhuge (JIRA)
John Zhuge created HDFS-10215: - Summary: distcp throws vague exception without home directory Key: HDFS-10215 URL: https://issues.apache.org/jira/browse/HDFS-10215 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-10216) distcp -diff relative path exception

2016-03-25 Thread John Zhuge (JIRA)
John Zhuge created HDFS-10216: - Summary: distcp -diff relative path exception Key: HDFS-10216 URL: https://issues.apache.org/jira/browse/HDFS-10216 Project: Hadoop HDFS Issue Type: Bug

[jira] [Commented] (HDFS-10216) distcp -diff relative path exception

2016-03-25 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15212151#comment-15212151 ] John Zhuge commented on HDFS-10216: --- Did hit the exception twice on 2 different nodes wh

[jira] [Updated] (HDFS-9633) Add more SnapshotDiffReport unit tests

2016-03-25 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9633: - Description: Add more behavior-based unit tests to verify and document SnapshotDiffReport API behaviors: *

[jira] [Commented] (HDFS-10215) distcp throws vague exception without home directory

2016-03-28 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15214228#comment-15214228 ] John Zhuge commented on HDFS-10215: --- Thanks [~steve_l]. Will do. > distcp throws vague

[jira] [Commented] (HDFS-6520) hdfs fsck -move not working

2016-03-28 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15214966#comment-15214966 ] John Zhuge commented on HDFS-6520: -- Wrote the unit test that reproduced the problem. Got c

[jira] [Commented] (HDFS-9939) Increase DecompressorStream skip buffer size

2016-03-28 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15215119#comment-15215119 ] John Zhuge commented on HDFS-9939: -- Thanks. That could have been caught by HADOOP-12701 si

[jira] [Commented] (HDFS-7060) Avoid taking locks when sending heartbeats from the DataNode

2016-03-28 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15215413#comment-15215413 ] John Zhuge commented on HDFS-7060: -- If we assume dataset lock protects {{bpSlices}}. The f

[jira] [Created] (HDFS-10230) HDFS Native Client build failed

2016-03-29 Thread John Zhuge (JIRA)
John Zhuge created HDFS-10230: - Summary: HDFS Native Client build failed Key: HDFS-10230 URL: https://issues.apache.org/jira/browse/HDFS-10230 Project: Hadoop HDFS Issue Type: Bug Compo

[jira] [Commented] (HDFS-10230) HDFS Native Client build failed

2016-03-29 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15216205#comment-15216205 ] John Zhuge commented on HDFS-10230: --- No problem with {{mvn clean install -DskipTests -Dm

[jira] [Assigned] (HDFS-10233) Ability to list inode number with -ls

2016-03-29 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge reassigned HDFS-10233: - Assignee: John Zhuge > Ability to list inode number with -ls > --

[jira] [Commented] (HDFS-6520) hdfs fsck -move not working

2016-04-01 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15222114#comment-15222114 ] John Zhuge commented on HDFS-6520: -- In this test case, {{echo CORRUPT >/path/to/block}} is

[jira] [Updated] (HDFS-6520) hdfs fsck -move not working

2016-04-01 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-6520: - Attachment: HDFS-6520-partial.001.patch Add a partial patch with a unit test that reproduces the problem. H

[jira] [Updated] (HDFS-10216) distcp -diff relative path exception

2016-04-08 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-10216: -- Assignee: (was: John Zhuge) > distcp -diff relative path exception > ---

[jira] [Commented] (HDFS-10216) distcp -diff relative path exception

2016-04-08 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15232890#comment-15232890 ] John Zhuge commented on HDFS-10216: --- Please go ahead. > distcp -diff relative path exce

[jira] [Commented] (HDFS-10216) distcp -diff relative path exception

2016-04-09 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15233621#comment-15233621 ] John Zhuge commented on HDFS-10216: --- Great job with the patch [~bwtakacy]. I assume the

[jira] [Commented] (HDFS-10216) distcp -diff relative path exception

2016-04-10 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15234354#comment-15234354 ] John Zhuge commented on HDFS-10216: --- Thanks for enhancing the unit tests which will defi

[jira] [Updated] (HDFS-7499) Add NFSv4 + Kerberos / client authentication support

2016-04-11 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-7499: - Assignee: (was: John Zhuge) > Add NFSv4 + Kerberos / client authentication support > --

[jira] [Commented] (HDFS-7499) Add NFSv4 + Kerberos / client authentication support

2016-04-11 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15235557#comment-15235557 ] John Zhuge commented on HDFS-7499: -- Do not have bandwidth to work on it for now. > Add NF

[jira] [Commented] (HDFS-10216) distcp -diff relative path exception

2016-04-13 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15239952#comment-15239952 ] John Zhuge commented on HDFS-10216: --- Great job. Just minor nitpicks: * {{TestDistCpSync.

[jira] [Commented] (HDFS-10216) distcp -diff relative path exception

2016-04-13 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15240461#comment-15240461 ] John Zhuge commented on HDFS-10216: --- Very nice. Consider reviewed after 2 tiny updates:

[jira] [Commented] (HDFS-9670) DistCp throws NPE when source is root "/"

2016-04-13 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15240486#comment-15240486 ] John Zhuge commented on HDFS-9670: -- Reproduced test in different ways: * The target dir mu

[jira] [Work started] (HDFS-9670) DistCp throws NPE when source is root "/"

2016-04-13 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-9670 started by John Zhuge. > DistCp throws NPE when source is root "/" > - >

[jira] [Commented] (HDFS-9670) DistCp throws NPE when source is root "/"

2016-04-13 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15240490#comment-15240490 ] John Zhuge commented on HDFS-9670: -- Source file system can also be different, e.g., {{s3a}

[jira] [Updated] (HDFS-9670) DistCp throws NPE when source is root and target exists

2016-04-13 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9670: - Summary: DistCp throws NPE when source is root and target exists (was: DistCp throws NPE when source is ro

[jira] [Created] (HDFS-10287) MiniDFSCluster should implement AutoCloseable

2016-04-13 Thread John Zhuge (JIRA)
John Zhuge created HDFS-10287: - Summary: MiniDFSCluster should implement AutoCloseable Key: HDFS-10287 URL: https://issues.apache.org/jira/browse/HDFS-10287 Project: Hadoop HDFS Issue Type: Impro

[jira] [Updated] (HDFS-9670) DistCp throws NPE when source is root

2016-04-13 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9670: - Summary: DistCp throws NPE when source is root (was: DistCp throws NPE when source is root and target exis

  1   2   3   4   5   6   7   8   9   10   >