[jira] [Comment Edited] (HDFS-13581) DN UI logs link is broken when https is enabled

2018-05-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16479149#comment-16479149 ] Kihwal Lee edited comment on HDFS-13581 at 5/17/18 2:43 PM: The integration

[jira] [Commented] (HDFS-13581) DN UI logs link is broken when https is enabled

2018-05-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16479149#comment-16479149 ] Kihwal Lee commented on HDFS-13581: --- The integration build ran on H19 failed due to this: {noformat}

[jira] [Updated] (HDFS-13581) DN UI logs link is broken when https is enabled

2018-05-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-13581: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.5 3.0.3

[jira] [Updated] (HDFS-13581) DN UI logs link is broken when https is enabled

2018-05-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-13581: -- Summary: DN UI logs link is broken when https is enabled (was: On clicking DN UI logs link it uses

[jira] [Commented] (HDFS-13581) On clicking DN UI logs link it uses http protocol for Wire encrypted cluster

2018-05-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16479076#comment-16479076 ] Kihwal Lee commented on HDFS-13581: --- +1 > On clicking DN UI logs link it uses http protocol for Wire

[jira] [Commented] (HDFS-13519) Namenode recovery does not replay the inProgress edit log

2018-05-01 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459976#comment-16459976 ] Kihwal Lee commented on HDFS-13519: --- You can always manually issue "dfsadmin -rollEdits". Also you can

[jira] [Commented] (HDFS-13433) webhdfs requests can be routed incorrectly in federated cluster

2018-05-01 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459694#comment-16459694 ] Kihwal Lee commented on HDFS-13433: --- This breaks the IP failover HA setup. In your example, nn3/nn4

[jira] [Comment Edited] (HDFS-13485) DataNode WebUI throws NPE

2018-04-19 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16444648#comment-16444648 ] Kihwal Lee edited comment on HDFS-13485 at 4/19/18 7:08 PM: That's not web ui.

[jira] [Commented] (HDFS-13485) DataNode WebUI throws NPE

2018-04-19 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16444648#comment-16444648 ] Kihwal Lee commented on HDFS-13485: --- That's not web ui. We could make it fail more gracefully when

[jira] [Commented] (HDFS-13272) DataNodeHttpServer to have configurable HttpServer2 threads

2018-04-18 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16442471#comment-16442471 ] Kihwal Lee commented on HDFS-13272: --- [~chris.douglas], makes sense. > DataNodeHttpServer to have

[jira] [Commented] (HDFS-13448) HDFS Block Placement - Ignore Locality for First Block Replica

2018-04-16 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16439676#comment-16439676 ] Kihwal Lee commented on HDFS-13448: --- bq. However, I think this should be a feature allowed by default. 

[jira] [Commented] (HDFS-13419) client can communicate to server even if hdfs delegation token expired

2018-04-10 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432402#comment-16432402 ] Kihwal Lee commented on HDFS-13419: --- Yes, a token is checked on connection establishment. If the

[jira] [Commented] (HDFS-12749) DN may not send block report to NN after NN restart

2018-03-30 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16420758#comment-16420758 ] Kihwal Lee commented on HDFS-12749: --- The {{register()}} method is called from two different contexts. BP

[jira] [Commented] (HDFS-12749) DN may not send block report to NN after NN restart

2018-03-27 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16416048#comment-16416048 ] Kihwal Lee commented on HDFS-12749: --- The latest patch is a step in the right direction. Some of the

[jira] [Assigned] (HDFS-12749) DN may not send block report to NN after NN restart

2018-03-27 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee reassigned HDFS-12749: - Assignee: He Xiaoqiao > DN may not send block report to NN after NN restart >

[jira] [Commented] (HDFS-12749) DN may not send block report to NN after NN restart

2018-03-23 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411417#comment-16411417 ] Kihwal Lee commented on HDFS-12749: --- Sorry for the delay. I will review it in a couple of days. > DN

[jira] [Updated] (HDFS-13195) DataNode conf page cannot display the current value after reconfig

2018-03-22 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-13195: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) >

[jira] [Commented] (HDFS-13195) DataNode conf page cannot display the current value after reconfig

2018-03-22 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409967#comment-16409967 ] Kihwal Lee commented on HDFS-13195: --- Just committed this to trunk, branch-3.1, branch-3.0, branch-2,

[jira] [Updated] (HDFS-13195) DataNode conf page cannot display the current value after reconfig

2018-03-22 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-13195: -- Fix Version/s: (was: 2.7.1) 3.1.1 3.2.0

[jira] [Commented] (HDFS-13195) DataNode conf page cannot display the current value after reconfig

2018-03-22 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409929#comment-16409929 ] Kihwal Lee commented on HDFS-13195: --- +1 looks good. > DataNode conf page cannot display the current

[jira] [Commented] (HDFS-13272) DataNodeHttpServer to have configurable HttpServer2 threads

2018-03-13 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16397433#comment-16397433 ] Kihwal Lee commented on HDFS-13272: --- bq. ... the possibility of this being too few threads ... The jetty

[jira] [Commented] (HDFS-12614) FSPermissionChecker#getINodeAttrs() throws NPE when INodeAttributesProvider configured

2018-03-08 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391888#comment-16391888 ] Kihwal Lee commented on HDFS-12614: --- Committed to branch-2, branch-2.9 and branch-2.8. >

[jira] [Updated] (HDFS-12614) FSPermissionChecker#getINodeAttrs() throws NPE when INodeAttributesProvider configured

2018-03-08 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12614: -- Fix Version/s: 2.8.4 2.9.1 2.10.0 >

[jira] [Commented] (HDFS-12614) FSPermissionChecker#getINodeAttrs() throws NPE when INodeAttributesProvider configured

2018-03-08 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391680#comment-16391680 ] Kihwal Lee commented on HDFS-12614: --- Attaching a branch-2 version. The only change is in the test.

[jira] [Updated] (HDFS-12614) FSPermissionChecker#getINodeAttrs() throws NPE when INodeAttributesProvider configured

2018-03-08 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12614: -- Attachment: HDFS-12614.branch-2.patch > FSPermissionChecker#getINodeAttrs() throws NPE when

[jira] [Commented] (HDFS-13236) Standby NN down with error encountered while tailing edits

2018-03-08 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391622#comment-16391622 ] Kihwal Lee commented on HDFS-13236: --- For block placement differences, {{DFSNetworkTopology}} is new in

[jira] [Commented] (HDFS-13234) Remove renew configuration instance in ConfiguredFailoverProxyProvider and reduce memory footprint for client

2018-03-08 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391519#comment-16391519 ] Kihwal Lee commented on HDFS-13234: --- [~jlowe] and I discussed a bit about the conf issue this morning.

[jira] [Commented] (HDFS-13234) Remove renew configuration instance in ConfiguredFailoverProxyProvider and reduce memory footprint for client

2018-03-07 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16390298#comment-16390298 ] Kihwal Lee commented on HDFS-13234: --- New conf objects are created to prevent unintended conf update

[jira] [Commented] (HDFS-13234) Remove renew configuration instance in ConfiguredFailoverProxyProvider and reduce memory footprint for client

2018-03-07 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16390147#comment-16390147 ] Kihwal Lee commented on HDFS-13234: --- bq. Configuration occupies over 600MB How big is a single instance

[jira] [Commented] (HDFS-13195) DataNode conf page cannot display the current value after reconfig

2018-03-06 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388627#comment-16388627 ] Kihwal Lee commented on HDFS-13195: --- The HttpServer2's conf servlet is supposed to show the http server

[jira] [Resolved] (HDFS-13206) IllegalStateException: Unable to finalize edits file

2018-03-01 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee resolved HDFS-13206. --- Resolution: Invalid > IllegalStateException: Unable to finalize edits file >

[jira] [Comment Edited] (HDFS-13206) IllegalStateException: Unable to finalize edits file

2018-02-28 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16380822#comment-16380822 ] Kihwal Lee edited comment on HDFS-13206 at 2/28/18 6:52 PM: It looks like the

[jira] [Commented] (HDFS-13206) IllegalStateException: Unable to finalize edits file

2018-02-28 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16380822#comment-16380822 ] Kihwal Lee commented on HDFS-13206: --- It looks like the NN went into safe mode because the disk ran out

[jira] [Updated] (HDFS-12070) Failed block recovery leaves files open indefinitely and at risk for data loss

2018-02-26 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12070: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.2.0 3.0.2

[jira] [Commented] (HDFS-13164) File not closed if streamer fail with DSQuotaExceededException

2018-02-26 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16376997#comment-16376997 ] Kihwal Lee commented on HDFS-13164: --- branch-2.8 is not building after this. > File not closed if

[jira] [Commented] (HDFS-13189) Standby NameNode should roll active edit log when checkpointing

2018-02-26 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16376939#comment-16376939 ] Kihwal Lee commented on HDFS-13189: --- We do configure auto rolling as you mentioned. A standby NN can be

[jira] [Commented] (HDFS-12749) DN may not send block report to NN after NN restart

2018-02-22 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373667#comment-16373667 ] Kihwal Lee commented on HDFS-12749: --- bq. About catching IOException: if the registration fails with a

[jira] [Commented] (HDFS-12070) Failed block recovery leaves files open indefinitely and at risk for data loss

2018-02-22 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373582#comment-16373582 ] Kihwal Lee commented on HDFS-12070: --- Attaching an updated patch that commits right away excluding

[jira] [Updated] (HDFS-12070) Failed block recovery leaves files open indefinitely and at risk for data loss

2018-02-22 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12070: -- Attachment: HDFS-12070.1.patch > Failed block recovery leaves files open indefinitely and at risk for

[jira] [Commented] (HDFS-12070) Failed block recovery leaves files open indefinitely and at risk for data loss

2018-02-22 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373276#comment-16373276 ] Kihwal Lee commented on HDFS-12070: --- It sure will be better if we can close the file right away. The

[jira] [Resolved] (HDFS-13169) Ambari UI deploy fails during startup of Ambari Metrics

2018-02-20 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee resolved HDFS-13169. --- Resolution: Invalid > Ambari UI deploy fails during startup of Ambari Metrics >

[jira] [Commented] (HDFS-13169) Ambari UI deploy fails during startup of Ambari Metrics

2018-02-19 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369193#comment-16369193 ] Kihwal Lee commented on HDFS-13169: --- Any reason why you filed it against HDFS instead of Ambari?  >

[jira] [Commented] (HDFS-13112) Token expiration edits may cause log corruption or deadlock

2018-02-15 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366307#comment-16366307 ] Kihwal Lee commented on HDFS-13112: --- Committed this to trunk, branch-3.1, branch-3.0, branch-3.0.1,

[jira] [Updated] (HDFS-13112) Token expiration edits may cause log corruption or deadlock

2018-02-15 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-13112: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.2.0 2.7.6

[jira] [Commented] (HDFS-13112) Token expiration edits may cause log corruption or deadlock

2018-02-15 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366274#comment-16366274 ] Kihwal Lee commented on HDFS-13112: --- Will commit it shortly. > Token expiration edits may cause log

[jira] [Commented] (HDFS-12749) DN may not send block report to NN after NN restart

2018-02-14 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364731#comment-16364731 ] Kihwal Lee commented on HDFS-12749: --- I see. The old registration looks identical to the new one, so NN

[jira] [Commented] (HDFS-12749) DN may not send block report to NN after NN restart

2018-02-14 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364508#comment-16364508 ] Kihwal Lee commented on HDFS-12749: --- What was the command that failed when you saw "Error processing

[jira] [Commented] (HDFS-12070) Failed block recovery leaves files open indefinitely and at risk for data loss

2018-02-13 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16363210#comment-16363210 ] Kihwal Lee commented on HDFS-12070: --- {noformat} [INFO]

[jira] [Commented] (HDFS-12070) Failed block recovery leaves files open indefinitely and at risk for data loss

2018-02-13 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16363160#comment-16363160 ] Kihwal Lee commented on HDFS-12070: --- The failed (long) tests in precommit are all passing on my machine.

[jira] [Commented] (HDFS-13111) Close recovery may incorrectly mark blocks corrupt

2018-02-13 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362677#comment-16362677 ] Kihwal Lee commented on HDFS-13111: --- It looks like "trying forever" might be actually part of the

[jira] [Commented] (HDFS-12070) Failed block recovery leaves files open indefinitely and at risk for data loss

2018-02-13 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362616#comment-16362616 ] Kihwal Lee commented on HDFS-12070: --- Finally got around to add a test case. From the test output, I see

[jira] [Updated] (HDFS-12070) Failed block recovery leaves files open indefinitely and at risk for data loss

2018-02-13 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12070: -- Attachment: HDFS-12070.0.patch > Failed block recovery leaves files open indefinitely and at risk for

[jira] [Comment Edited] (HDFS-13135) Lease not deleted when deleting INodeReference

2018-02-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361517#comment-16361517 ] Kihwal Lee edited comment on HDFS-13135 at 2/12/18 10:06 PM: - >From what I

[jira] [Comment Edited] (HDFS-13135) Lease not deleted when deleting INodeReference

2018-02-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361517#comment-16361517 ] Kihwal Lee edited comment on HDFS-13135 at 2/12/18 10:06 PM: - >From what I

[jira] [Commented] (HDFS-13135) Lease not deleted when deleting INodeReference

2018-02-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361517#comment-16361517 ] Kihwal Lee commented on HDFS-13135: --- >From what I remember, a change (bug fix?) intentionally made a

[jira] [Commented] (HDFS-13128) HDFS balancer in single node cluster fails with "Another Balancer is running.."

2018-02-09 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358483#comment-16358483 ] Kihwal Lee commented on HDFS-13128: --- What do you see in the balancer log? Look for something like this.

[jira] [Commented] (HDFS-13126) Re-enable HTTP Request Logging for WebHDFS

2018-02-08 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16357551#comment-16357551 ] Kihwal Lee commented on HDFS-13126: --- See HDFS-7959 > Re-enable HTTP Request Logging for WebHDFS >

[jira] [Updated] (HDFS-13112) Token expiration edits may cause log corruption or deadlock

2018-02-08 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-13112: -- Target Version/s: 3.0.1, 2.7.6 (was: 2.7.6) > Token expiration edits may cause log corruption or

[jira] [Commented] (HDFS-13112) Token expiration edits may cause log corruption or deadlock

2018-02-08 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16357180#comment-16357180 ] Kihwal Lee commented on HDFS-13112: --- The handler for {{enterSafeMode}} has acquired a write lock. The

[jira] [Commented] (HDFS-13112) Token expiration edits may cause log corruption or deadlock

2018-02-08 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16357050#comment-16357050 ] Kihwal Lee commented on HDFS-13112: --- When I ran the failed tests, the following is reproduced. It seems

[jira] [Commented] (HDFS-13112) Token expiration edits may cause log corruption or deadlock

2018-02-07 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355844#comment-16355844 ] Kihwal Lee commented on HDFS-13112: --- The patch looks good. - The addition of read locks ensures these

[jira] [Commented] (HDFS-13117) Proposal to support writing replications to HDFS asynchronously

2018-02-07 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355520#comment-16355520 ] Kihwal Lee commented on HDFS-13117: --- During writes, the data is written to multiple nodes concurrently.

[jira] [Commented] (HDFS-13111) Close recovery may incorrectly mark blocks corrupt

2018-02-06 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354076#comment-16354076 ] Kihwal Lee commented on HDFS-13111: --- Here is relevant log lines of the latest example. This node was

[jira] [Commented] (HDFS-13104) Make hadoop proxy user changes reconfigurable in Namenode

2018-02-05 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16352840#comment-16352840 ] Kihwal Lee commented on HDFS-13104: --- bq. Currently any changes to add/delete a new proxy user requires

[jira] [Commented] (HDFS-13100) Handle IllegalArgumentException in when GETSERVERDEFAULTS is not implemented in webhdfs.

2018-02-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350958#comment-16350958 ] Kihwal Lee commented on HDFS-13100: --- branch-3.0.1 is already cut. Be sure to commit it there too. >

[jira] [Updated] (HDFS-13100) Handle IllegalArgumentException in when GETSERVERDEFAULTS is not implemented in webhdfs.

2018-02-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-13100: -- Target Version/s: 3.1.0, 2.10.0, 2.9.1, 3.0.1, 2.8.4 > Handle IllegalArgumentException in when

[jira] [Commented] (HDFS-13100) Handle IllegalArgumentException in when GETSERVERDEFAULTS is not implemented in webhdfs.

2018-02-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350955#comment-16350955 ] Kihwal Lee commented on HDFS-13100: --- +1. Precommit successfully ran against the same code. The only diff

[jira] [Updated] (HDFS-13100) Handle IllegalArgumentException in when GETSERVERDEFAULTS is not implemented in webhdfs.

2018-02-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-13100: -- Priority: Critical (was: Major) > Handle IllegalArgumentException in when GETSERVERDEFAULTS is not

[jira] [Updated] (HDFS-13100) Handle IllegalArgumentException in when GETSERVERDEFAULTS is not implemented in webhdfs.

2018-02-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-13100: -- Issue Type: Bug (was: Improvement) > Handle IllegalArgumentException in when GETSERVERDEFAULTS is not

[jira] [Commented] (HDFS-12386) Add fsserver defaults call to WebhdfsFileSystem.

2018-02-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350454#comment-16350454 ] Kihwal Lee commented on HDFS-12386: --- The code added for backward-compatibility doesn't actually work. It

[jira] [Commented] (HDFS-13100) Handle IllegalArgumentException in when GETSERVERDEFAULTS is not implemented in webhdfs.

2018-02-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350445#comment-16350445 ] Kihwal Lee commented on HDFS-13100: --- I ran more tests (416 total) and they all passed. {noformat}$ mvn

[jira] [Commented] (HDFS-13100) Handle IllegalArgumentException in when GETSERVERDEFAULTS is not implemented in webhdfs.

2018-02-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350382#comment-16350382 ] Kihwal Lee commented on HDFS-13100: --- I know you just copied the comment from the previous section, but

[jira] [Commented] (HDFS-13100) Handle IllegalArgumentException in when GETSERVERDEFAULTS is not implemented in webhdfs.

2018-02-01 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349433#comment-16349433 ] Kihwal Lee commented on HDFS-13100: --- So, it's not the hadoop code explicitly doing {{valueOf()}}, but

[jira] [Comment Edited] (HDFS-13100) Handle IllegalArgumentException in when GETSERVERDEFAULTS is not implemented in webhdfs.

2018-02-01 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349424#comment-16349424 ] Kihwal Lee edited comment on HDFS-13100 at 2/1/18 10:42 PM: Got one at TRACE.

[jira] [Commented] (HDFS-13100) Handle IllegalArgumentException in when GETSERVERDEFAULTS is not implemented in webhdfs.

2018-02-01 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349424#comment-16349424 ] Kihwal Lee commented on HDFS-13100: --- Got one at TRACE. {noformat} 2018-02-01 22:38:55,161

[jira] [Commented] (HDFS-13100) Handle IllegalArgumentException in when GETSERVERDEFAULTS is not implemented in webhdfs.

2018-02-01 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349392#comment-16349392 ] Kihwal Lee commented on HDFS-13100: --- Reproduced the issue. Enabled DEBUG-level logging on NN, but

[jira] [Commented] (HDFS-12909) SSLConnectionConfigurator creation error should be printed only if security is enabled

2018-02-01 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16348954#comment-16348954 ] Kihwal Lee commented on HDFS-12909: --- The patch looks good in general. One concern is that it changes the

[jira] [Updated] (HDFS-13051) dead lock occurs when rolleditlog rpc call happen and editPendingQ is full

2018-02-01 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-13051: -- Target Version/s: 3.1.0, 2.10.0, 2.9.1, 3.0.1, 2.8.4, 2.7.6 (was: 2.8.1, 2.8.2, 2.7.5, 3.0.0) > dead

[jira] [Commented] (HDFS-11187) Optimize disk access for last partial chunk checksum of Finalized replica

2018-02-01 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16348782#comment-16348782 ] Kihwal Lee commented on HDFS-11187: --- The patch looks good to me. > Optimize disk access for last

[jira] [Commented] (HDFS-13100) Handle IllegalArgumentException in when GETSERVERDEFAULTS is not implemented in webhdfs.

2018-02-01 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16348777#comment-16348777 ] Kihwal Lee commented on HDFS-13100: --- The {{handle()}} method you are showing is from {{WebHdfsHandler}}

[jira] [Commented] (HDFS-11187) Optimize disk access for last partial chunk checksum of Finalized replica

2018-01-31 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346967#comment-16346967 ] Kihwal Lee commented on HDFS-11187: --- bq.  Not sure why yetus said it failed tests. Some tests failed to

[jira] [Updated] (HDFS-12574) Add CryptoInputStream to WebHdfsFileSystem read call.

2018-01-29 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12574: -- Fix Version/s: 2.10.0 > Add CryptoInputStream to WebHdfsFileSystem read call. >

[jira] [Updated] (HDFS-12574) Add CryptoInputStream to WebHdfsFileSystem read call.

2018-01-29 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12574: -- Attachment: HDFS-12574.012.branch-2.8.patch > Add CryptoInputStream to WebHdfsFileSystem read call. >

[jira] [Updated] (HDFS-12574) Add CryptoInputStream to WebHdfsFileSystem read call.

2018-01-29 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12574: -- Fix Version/s: 2.8.4 > Add CryptoInputStream to WebHdfsFileSystem read call. >

[jira] [Commented] (HDFS-12574) Add CryptoInputStream to WebHdfsFileSystem read call.

2018-01-29 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344239#comment-16344239 ] Kihwal Lee commented on HDFS-12574: --- Committed to branch-2.8.  > Add CryptoInputStream to

[jira] [Updated] (HDFS-12574) Add CryptoInputStream to WebHdfsFileSystem read call.

2018-01-29 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12574: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Also

[jira] [Updated] (HDFS-12574) Add CryptoInputStream to WebHdfsFileSystem read call.

2018-01-29 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12574: -- Attachment: HDFS-12574.012.branch-2.patch > Add CryptoInputStream to WebHdfsFileSystem read call. >

[jira] [Updated] (HDFS-12574) Add CryptoInputStream to WebHdfsFileSystem read call.

2018-01-29 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12574: -- Fix Version/s: 2.9.1 > Add CryptoInputStream to WebHdfsFileSystem read call. >

[jira] [Commented] (HDFS-12574) Add CryptoInputStream to WebHdfsFileSystem read call.

2018-01-29 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344189#comment-16344189 ] Kihwal Lee commented on HDFS-12574: --- Thanks, [~xiaochen]. I just committed it to trunk and branch-3.0. 

[jira] [Updated] (HDFS-12574) Add CryptoInputStream to WebHdfsFileSystem read call.

2018-01-29 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12574: -- Fix Version/s: 3.0.1 3.1.0 > Add CryptoInputStream to WebHdfsFileSystem read call. >

[jira] [Commented] (HDFS-12574) Add CryptoInputStream to WebHdfsFileSystem read call.

2018-01-29 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16343985#comment-16343985 ] Kihwal Lee commented on HDFS-12574: --- Since Rushabh is on vacation and the review comments are minor,

[jira] [Updated] (HDFS-12574) Add CryptoInputStream to WebHdfsFileSystem read call.

2018-01-29 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12574: -- Attachment: HDFS-12574.012.patch > Add CryptoInputStream to WebHdfsFileSystem read call. >

[jira] [Commented] (HDFS-13051) dead lock occurs when using async editlog while rolleditlog rpc call happen

2018-01-23 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16336181#comment-16336181 ] Kihwal Lee commented on HDFS-13051: --- [~daryn], please have a look. > dead lock occurs when using async

[jira] [Updated] (HDFS-12070) Failed block recovery leaves files open indefinitely and at risk for data loss

2018-01-10 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12070: -- Status: Patch Available (was: Open) > Failed block recovery leaves files open indefinitely and at risk

[jira] [Updated] (HDFS-12070) Failed block recovery leaves files open indefinitely and at risk for data loss

2018-01-10 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12070: -- Attachment: lease.patch Attaching a patch without any unit test to run through the qa bot. > Failed

[jira] [Commented] (HDFS-12993) Add an option to Balancer for excluding the paths

2018-01-08 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16316765#comment-16316765 ] Kihwal Lee commented on HDFS-12993: --- The requirement sounds fine at a high level, but {{getBlocks()}}

[jira] [Comment Edited] (HDFS-12070) Failed block recovery leaves files open indefinitely and at risk for data loss

2017-12-15 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16292994#comment-16292994 ] Kihwal Lee edited comment on HDFS-12070 at 12/15/17 7:20 PM: - To complete the

[jira] [Assigned] (HDFS-12070) Failed block recovery leaves files open indefinitely and at risk for data loss

2017-12-15 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee reassigned HDFS-12070: - Assignee: Kihwal Lee > Failed block recovery leaves files open indefinitely and at risk for data

[jira] [Commented] (HDFS-12070) Failed block recovery leaves files open indefinitely and at risk for data loss

2017-12-15 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16292994#comment-16292994 ] Kihwal Lee commented on HDFS-12070: --- To complete the history lesson, I traced down when {{closeFile}}

[jira] [Comment Edited] (HDFS-12070) Failed block recovery leaves files open indefinitely and at risk for data loss

2017-12-15 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16292929#comment-16292929 ] Kihwal Lee edited comment on HDFS-12070 at 12/15/17 5:59 PM: - bq. the PD needs

<    1   2   3   4   5   6   7   8   9   10   >