[jira] [Commented] (HDFS-11053) Unnecessary superuser check in versionRequest()

2016-10-26 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15608530#comment-15608530 ] Kihwal Lee commented on HDFS-11053: --- Ran three times. They all pass. {noformat}

[jira] [Updated] (HDFS-11050) Change log level to 'warn' when ssl initialization fails and defaults to DEFAULT_TIMEOUT_CONN_CONFIGURATOR

2016-10-26 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11050: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Commented] (HDFS-11050) Change log level to 'warn' when ssl initialization fails and defaults to DEFAULT_TIMEOUT_CONN_CONFIGURATOR

2016-10-26 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15608384#comment-15608384 ] Kihwal Lee commented on HDFS-11050: --- +1 the patch looks good. [~xiaobingo], I think the unit test is

[jira] [Comment Edited] (HDFS-11053) Unnecessary superuser check in versionRequest()

2016-10-25 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15606527#comment-15606527 ] Kihwal Lee edited comment on HDFS-11053 at 10/25/16 9:34 PM: - The check was

[jira] [Commented] (HDFS-11053) Unnecessary superuser check in versionRequest()

2016-10-25 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15606527#comment-15606527 ] Kihwal Lee commented on HDFS-11053: --- The check was added in the process of fixing HDFS-2264. >

[jira] [Updated] (HDFS-11053) Unnecessary superuser check in versionRequest()

2016-10-25 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11053: -- Component/s: security namenode > Unnecessary superuser check in versionRequest() >

[jira] [Updated] (HDFS-11053) Unnecessary superuser check in versionRequest()

2016-10-25 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11053: -- Attachment: HDFS-11053.patch > Unnecessary superuser check in versionRequest() >

[jira] [Updated] (HDFS-11053) Unnecessary superuser check in versionRequest()

2016-10-25 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11053: -- Status: Patch Available (was: Open) > Unnecessary superuser check in versionRequest() >

[jira] [Assigned] (HDFS-11053) Unnecessary superuser check in versionRequest()

2016-10-25 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee reassigned HDFS-11053: - Assignee: Kihwal Lee > Unnecessary superuser check in versionRequest() >

[jira] [Created] (HDFS-11053) Unnecessary superuser check in versionRequest()

2016-10-25 Thread Kihwal Lee (JIRA)
Kihwal Lee created HDFS-11053: - Summary: Unnecessary superuser check in versionRequest() Key: HDFS-11053 URL: https://issues.apache.org/jira/browse/HDFS-11053 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-11015) Enforce timeout in balancer

2016-10-25 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15606292#comment-15606292 ] Kihwal Lee commented on HDFS-11015: --- [~zhz], thanks for reviews and commits. > Enforce timeout in

[jira] [Commented] (HDFS-11050) Change log level to 'warn' when ssl initialization fails and defaults to DEFAULT_TIMEOUT_CONN_CONFIGURATOR

2016-10-25 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15605587#comment-15605587 ] Kihwal Lee commented on HDFS-11050: --- It was part of HDFS-5538. While loading ssl config is fine,

[jira] [Commented] (HDFS-11015) Enforce timeout in balancer

2016-10-25 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15605285#comment-15605285 ] Kihwal Lee commented on HDFS-11015: --- The failed tests all pass when run on my machine. > Enforce

[jira] [Updated] (HDFS-10997) Reduce number of path resolving methods

2016-10-24 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10997: -- Fix Version/s: 3.0.0-alpha2 > Reduce number of path resolving methods >

[jira] [Commented] (HDFS-10997) Reduce number of path resolving methods

2016-10-24 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15603416#comment-15603416 ] Kihwal Lee commented on HDFS-10997: --- Committed to trunk, but it doesn't apply to branch-2. Conflicts are

[jira] [Commented] (HDFS-10997) Reduce number of path resolving methods

2016-10-24 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15603385#comment-15603385 ] Kihwal Lee commented on HDFS-10997: --- Sorry, took a while to review. The change looks good and makes

[jira] [Commented] (HDFS-10857) Rolling upgrade can make data unavailable when the cluster has many failed volumes

2016-10-24 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15603358#comment-15603358 ] Kihwal Lee commented on HDFS-10857: --- I am not actively working on this. Please feel free to take over.

[jira] [Updated] (HDFS-11015) Enforce timeout in balancer

2016-10-24 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11015: -- Attachment: HDFS-11015-3.patch Silly me. I put {{isIterationOver()}} instead of {{!isIterationOver()}}

[jira] [Commented] (HDFS-11043) TestWebHdfsTimeouts fails

2016-10-24 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15603011#comment-15603011 ] Kihwal Lee commented on HDFS-11043: --- The 10 second timeout is probably HDFS-11042. We had it failing

[jira] [Commented] (HDFS-11042) Add missing cleanupSSLConfig() call for tests that use setupSSLConfig()

2016-10-24 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15602297#comment-15602297 ] Kihwal Lee commented on HDFS-11042: --- bq. E.g. {{TestWebHdfsTimeouts}} would randomly fail, because the

[jira] [Updated] (HDFS-11042) Add missing cleanupSSLConfig() call for tests that use setupSSLConfig()

2016-10-24 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11042: -- Description: Tests that setup SSLConfigs can leave conf-files lingering unless they are cleaned up via

[jira] [Updated] (HDFS-11042) Add missing cleanupSSLConfig() call for tests that use setupSSLConfig()

2016-10-24 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11042: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Commented] (HDFS-11042) Add missing cleanupSSLConfig() call for tests that use setupSSLConfig()

2016-10-24 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15601972#comment-15601972 ] Kihwal Lee commented on HDFS-11042: --- +1 looks good. > Add missing cleanupSSLConfig() call for tests

[jira] [Commented] (HDFS-11015) Enforce timeout in balancer

2016-10-21 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15595313#comment-15595313 ] Kihwal Lee commented on HDFS-11015: --- Heh. Looks like the unit tests are doing their job. > Enforce

[jira] [Updated] (HDFS-11015) Enforce timeout in balancer

2016-10-21 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11015: -- Attachment: HDFS-11015-2.patch Attaching the updated patch. > Enforce timeout in balancer >

[jira] [Updated] (HDFS-10627) Volume Scanner marks a block as "suspect" even if the exception is network-related

2016-10-20 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10627: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Updated] (HDFS-10627) Volume Scanner marks a block as "suspect" even if the exception is network-related

2016-10-20 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10627: -- Summary: Volume Scanner marks a block as "suspect" even if the exception is network-related (was:

[jira] [Commented] (HDFS-10627) Volume Scanner marks a block as "suspect" even if the block sender encounters 'Broken pipe' or 'Connection reset by peer' exception

2016-10-20 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15592168#comment-15592168 ] Kihwal Lee commented on HDFS-10627: --- bq. +1 Will check in tomorrow afternoon unless there are further

[jira] [Comment Edited] (HDFS-9096) Issue in Rollback (after rolling upgrade) from hadoop 2.7.1 to 2.4.0

2016-10-19 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588854#comment-15588854 ] Kihwal Lee edited comment on HDFS-9096 at 10/19/16 2:05 PM: Were you issuing

[jira] [Commented] (HDFS-9096) Issue in Rollback (after rolling upgrade) from hadoop 2.7.1 to 2.4.0

2016-10-19 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588854#comment-15588854 ] Kihwal Lee commented on HDFS-9096: -- Were you issuing rollback with 2.7.1? > Issue in Rollback (after

[jira] [Commented] (HDFS-9096) Issue in Rollback (after rolling upgrade) from hadoop 2.7.1 to 2.4.0

2016-10-18 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15586242#comment-15586242 ] Kihwal Lee commented on HDFS-9096: -- If it is indeed the VERSION file issue, you can workaround it by

[jira] [Commented] (HDFS-11026) Convert BlockTokenIdentifier to use Protobuf

2016-10-18 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15586236#comment-15586236 ] Kihwal Lee commented on HDFS-11026: --- [~daryn], would you care to comment on this? > Convert

[jira] [Commented] (HDFS-11015) Enforce timeout in balancer

2016-10-18 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15585860#comment-15585860 ] Kihwal Lee commented on HDFS-11015: --- Here is the graph that shows the improvement from an expanding

[jira] [Updated] (HDFS-11015) Enforce timeout in balancer

2016-10-18 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11015: -- Attachment: balancer.png > Enforce timeout in balancer > --- > >

[jira] [Updated] (HDFS-11015) Enforce timeout in balancer

2016-10-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11015: -- Attachment: HDFS-11015-1.patch > Enforce timeout in balancer > --- > >

[jira] [Updated] (HDFS-11015) Enforce timeout in balancer

2016-10-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11015: -- Status: Patch Available (was: Open) > Enforce timeout in balancer > --- > >

[jira] [Commented] (HDFS-11019) Inconsistent number of corrupt replicas if a corrupt replica is reported multiple times

2016-10-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15582321#comment-15582321 ] Kihwal Lee commented on HDFS-11019: --- [~kshukla] has debugged and worked on this quite a bit. She might

[jira] [Commented] (HDFS-10960) TestDataNodeHotSwapVolumes#testRemoveVolumeBeingWritten fails at disk error verification after volume remove

2016-10-14 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15576572#comment-15576572 ] Kihwal Lee commented on HDFS-10960: --- Reverted from branch-2 and branch-2.8. Please rework the patch for

[jira] [Reopened] (HDFS-10960) TestDataNodeHotSwapVolumes#testRemoveVolumeBeingWritten fails at disk error verification after volume remove

2016-10-14 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee reopened HDFS-10960: --- > TestDataNodeHotSwapVolumes#testRemoveVolumeBeingWritten fails at disk error > verification after

[jira] [Commented] (HDFS-10960) TestDataNodeHotSwapVolumes#testRemoveVolumeBeingWritten fails at disk error verification after volume remove

2016-10-14 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15576561#comment-15576561 ] Kihwal Lee commented on HDFS-10960: --- branch-2 build fails. {noformat} [ERROR]

[jira] [Updated] (HDFS-11015) Enforce timeout in balancer

2016-10-14 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11015: -- Description: 1) Hung node detection: HDFS-6247 has removed the socket read timeout while adding the

[jira] [Assigned] (HDFS-11015) Enforce timeout in balancer

2016-10-14 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee reassigned HDFS-11015: - Assignee: Kihwal Lee > Enforce timeout in balancer > --- > >

[jira] [Created] (HDFS-11015) Enforce timeout in balancer

2016-10-14 Thread Kihwal Lee (JIRA)
Kihwal Lee created HDFS-11015: - Summary: Enforce timeout in balancer Key: HDFS-11015 URL: https://issues.apache.org/jira/browse/HDFS-11015 Project: Hadoop HDFS Issue Type: Bug

[jira] [Commented] (HDFS-9096) Issue in Rollback (after rolling upgrade) from hadoop 2.7.1 to 2.4.0

2016-10-14 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15576095#comment-15576095 ] Kihwal Lee commented on HDFS-9096: -- [~andreina], have you seen this kind of issue in rollback? >From the

[jira] [Updated] (HDFS-10990) TestPendingInvalidateBlock should wait for IBRs

2016-10-14 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10990: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Commented] (HDFS-10990) TestPendingInvalidateBlock should wait for IBRs

2016-10-14 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15575807#comment-15575807 ] Kihwal Lee commented on HDFS-10990: --- +1 > TestPendingInvalidateBlock should wait for IBRs >

[jira] [Updated] (HDFS-10987) Make Decommission less expensive when lot of blocks present.

2016-10-13 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10987: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved

[jira] [Updated] (HDFS-10987) Make Decommission less expensive when lot of blocks present.

2016-10-13 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10987: -- Fix Version/s: 3.0.0-alpha2 > Make Decommission less expensive when lot of blocks present. >

[jira] [Commented] (HDFS-10990) TestPendingInvalidateBlock should wait for IBRs

2016-10-13 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15572980#comment-15572980 ] Kihwal Lee commented on HDFS-10990: --- It looks like branch-2 build is failing after this. >

[jira] [Updated] (HDFS-11000) webhdfs PUT does not work if requests are routed to call queue.

2016-10-13 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11000: -- Description: If webhdfs calls are routed to ipc call queue, PUT does not work. I think this is what we

[jira] [Updated] (HDFS-11000) webhdfs PUT does not work if requests are routed to call queue.

2016-10-13 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11000: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Commented] (HDFS-10987) Make Decommission less expensive when lot of blocks present.

2016-10-13 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15571920#comment-15571920 ] Kihwal Lee commented on HDFS-10987: --- Ok. Not committing. Let's wait for Andrew. > Make Decommission

[jira] [Commented] (HDFS-10987) Make Decommission less expensive when lot of blocks present.

2016-10-13 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15571904#comment-15571904 ] Kihwal Lee commented on HDFS-10987: --- +1 from me too. Will commit it shortly. > Make Decommission less

[jira] [Commented] (HDFS-10789) Route webhdfs through the RPC call queue

2016-10-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569954#comment-15569954 ] Kihwal Lee commented on HDFS-10789: --- I remember we had to fix one bug after this feature. Filed

[jira] [Updated] (HDFS-11000) webhdfs PUT does not work if requests are routed to call queue.

2016-10-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11000: -- Target Version/s: 2.8.0 > webhdfs PUT does not work if requests are routed to call queue. >

[jira] [Commented] (HDFS-11000) webhdfs PUT does not work if requests are routed to call queue.

2016-10-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569951#comment-15569951 ] Kihwal Lee commented on HDFS-11000: --- We have been using this fix since last year in production. >

[jira] [Updated] (HDFS-11000) webhdfs PUT does not work if requests are routed to call queue.

2016-10-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11000: -- Description: If webhdfs calls are routed to ipc call queue, PUT does not work. I think this is what we

[jira] [Commented] (HDFS-11000) webhdfs PUT does not work if requests are routed to call queue.

2016-10-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569937#comment-15569937 ] Kihwal Lee commented on HDFS-11000: --- [~daryn], you know what it is. > webhdfs PUT does not work if

[jira] [Updated] (HDFS-11000) webhdfs PUT does not work if requests are routed to call queue.

2016-10-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11000: -- Status: Patch Available (was: Open) > webhdfs PUT does not work if requests are routed to call queue.

[jira] [Updated] (HDFS-11000) webhdfs PUT does not work if requests are routed to call queue.

2016-10-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11000: -- Attachment: HDFS-11000.patch > webhdfs PUT does not work if requests are routed to call queue. >

[jira] [Assigned] (HDFS-11000) webhdfs PUT does not work if requests are routed to call queue.

2016-10-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee reassigned HDFS-11000: - Assignee: Kihwal Lee > webhdfs PUT does not work if requests are routed to call queue. >

[jira] [Created] (HDFS-11000) webhdfs PUT does not work if requests are routed to call queue.

2016-10-12 Thread Kihwal Lee (JIRA)
Kihwal Lee created HDFS-11000: - Summary: webhdfs PUT does not work if requests are routed to call queue. Key: HDFS-11000 URL: https://issues.apache.org/jira/browse/HDFS-11000 Project: Hadoop HDFS

[jira] [Updated] (HDFS-10789) Route webhdfs through the RPC call queue

2016-10-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10789: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Commented] (HDFS-10789) Route webhdfs through the RPC call queue

2016-10-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569729#comment-15569729 ] Kihwal Lee commented on HDFS-10789: --- +1 looks good. We have been running this internally for quite some

[jira] [Commented] (HDFS-10987) Make Decommission less expensive when lot of blocks present.

2016-10-11 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15565805#comment-15565805 ] Kihwal Lee commented on HDFS-10987: --- bq. IIUC,Daryn Sharp was doing finegrained locking which is big

[jira] [Resolved] (HDFS-8041) Consider remaining space during block blockplacement if dfs space is highly utilized

2016-10-11 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee resolved HDFS-8041. -- Resolution: Won't Fix While this jira makes balancing more aggressive when free space is low, there are

[jira] [Updated] (HDFS-8041) Consider remaining space during block blockplacement if dfs space is highly utilized

2016-10-11 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-8041: - Status: Open (was: Patch Available) > Consider remaining space during block blockplacement if dfs space

[jira] [Commented] (HDFS-10987) Make Decommission less expensive when lot of blocks present.

2016-10-10 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562642#comment-15562642 ] Kihwal Lee commented on HDFS-10987: --- We've seen this also. We don't have that many blocks per node, but

[jira] [Updated] (HDFS-10980) Optimize check for existence of parent directory

2016-10-07 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10980: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Commented] (HDFS-10980) Optimize check for existence of parent directory

2016-10-07 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556402#comment-15556402 ] Kihwal Lee commented on HDFS-10980: --- +1 looks simple and straightforward. > Optimize check for

[jira] [Updated] (HDFS-10979) Pass IIP for FSDirDeleteOp methods

2016-10-07 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10979: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Commented] (HDFS-10979) Pass IIP for FSDirDeleteOp methods

2016-10-07 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556010#comment-15556010 ] Kihwal Lee commented on HDFS-10979: --- +1 Looks good. > Pass IIP for FSDirDeleteOp methods >

[jira] [Updated] (HDFS-10939) Reduce performance penalty of encryption zones

2016-10-07 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10939: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved

[jira] [Commented] (HDFS-10925) Cache symlinkString in INodeSymlink

2016-10-06 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553377#comment-15553377 ] Kihwal Lee commented on HDFS-10925: --- [~daryn] Please have a look. > Cache symlinkString in INodeSymlink

[jira] [Updated] (HDFS-10955) Pass IIP for FSDirAttr methods

2016-10-06 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10955: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Commented] (HDFS-10955) Pass IIP for FSDirAttr methods

2016-10-06 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553241#comment-15553241 ] Kihwal Lee commented on HDFS-10955: --- +1 > Pass IIP for FSDirAttr methods >

[jira] [Issue Comment Deleted] (HDFS-10955) Pass IIP for FSDirAttr methods

2016-10-06 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10955: -- Comment: was deleted (was: +1) > Pass IIP for FSDirAttr methods > -- > >

[jira] [Commented] (HDFS-10955) Pass IIP for FSDirAttr methods

2016-10-06 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553239#comment-15553239 ] Kihwal Lee commented on HDFS-10955: --- +1 > Pass IIP for FSDirAttr methods >

[jira] [Updated] (HDFS-10939) Reduce performance penalty of encryption zones

2016-10-06 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10939: -- Fix Version/s: 3.0.0-alpha2 > Reduce performance penalty of encryption zones >

[jira] [Commented] (HDFS-10939) Reduce performance penalty of encryption zones

2016-10-06 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553090#comment-15553090 ] Kihwal Lee commented on HDFS-10939: --- Committed it to trunk, but branch-2 merge is not clean. The

[jira] [Commented] (HDFS-10939) Reduce performance penalty of encryption zones

2016-10-06 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553049#comment-15553049 ] Kihwal Lee commented on HDFS-10939: --- +1 the new patch looks good. > Reduce performance penalty of

[jira] [Commented] (HDFS-10925) Cache symlinkString in INodeSymlink

2016-10-04 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15546797#comment-15546797 ] Kihwal Lee commented on HDFS-10925: --- I think what Daryn meant by bq. The latter is the dominate use

[jira] [Comment Edited] (HDFS-10955) Pass IIP for FSDirAttr methods

2016-10-04 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15546781#comment-15546781 ] Kihwal Lee edited comment on HDFS-10955 at 10/4/16 9:54 PM: It won't compile

[jira] [Commented] (HDFS-10955) Pass IIP for FSDirAttr methods

2016-10-04 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15546781#comment-15546781 ] Kihwal Lee commented on HDFS-10955: --- It won't compile with the patch. It looks like the {{setTime()}}

[jira] [Commented] (HDFS-10939) Reduce performance penalty of encryption zones

2016-10-04 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15546748#comment-15546748 ] Kihwal Lee commented on HDFS-10939: --- In {{startFileInt()}}, {{unlock()}} is done in

[jira] [Updated] (HDFS-10956) Remove rename/delete performance penalty when not using snapshots

2016-10-04 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10956: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Commented] (HDFS-10956) Remove rename/delete performance penalty when not using snapshots

2016-10-04 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15546493#comment-15546493 ] Kihwal Lee commented on HDFS-10956: --- +1 looks straight-forward. > Remove rename/delete performance

[jira] [Commented] (HDFS-10957) Retire BKJM from trunk

2016-10-04 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15546475#comment-15546475 ] Kihwal Lee commented on HDFS-10957: --- {{hadoop-hdfs-project/pom.xml}} needs to be updated. > Retire BKJM

[jira] [Updated] (HDFS-10955) Pass IIP for FSDirAttr methods

2016-10-04 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10955: -- Status: Patch Available (was: Open) > Pass IIP for FSDirAttr methods > --

[jira] [Comment Edited] (HDFS-10956) Remove rename/delete performance penalty when not using snapshots

2016-10-04 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15546016#comment-15546016 ] Kihwal Lee edited comment on HDFS-10956 at 10/4/16 5:17 PM: Actually I've been

[jira] [Commented] (HDFS-10956) Remove rename/delete performance penalty when not using snapshots

2016-10-04 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15546016#comment-15546016 ] Kihwal Lee commented on HDFS-10956: --- Actually I've been thinking about doing this myself. Finally we are

[jira] [Updated] (HDFS-10878) TestDFSClientRetries#testIdempotentAllocateBlockAndClose throws ConcurrentModificationException

2016-10-04 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10878: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Commented] (HDFS-10878) TestDFSClientRetries#testIdempotentAllocateBlockAndClose throws ConcurrentModificationException

2016-10-04 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15545827#comment-15545827 ] Kihwal Lee commented on HDFS-10878: --- +1 looks good to me. >

[jira] [Commented] (HDFS-10946) basic hadoop commands should not hang irrespective of any cluster issue

2016-10-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15543495#comment-15543495 ] Kihwal Lee commented on HDFS-10946: --- It should. I think the default rpc timeout is 60 seconds. > basic

[jira] [Commented] (HDFS-10789) Route webhdfs through the RPC call queue

2016-10-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15543312#comment-15543312 ] Kihwal Lee commented on HDFS-10789: --- The RPC ACL controls at protocol level, bit it looks like what you

[jira] [Commented] (HDFS-10939) Reduce performance penalty of encryption zones

2016-10-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15542838#comment-15542838 ] Kihwal Lee commented on HDFS-10939: --- Please rebase your patch. > Reduce performance penalty of

[jira] [Updated] (HDFS-10940) Reduce performance penalty of block caching when not used

2016-10-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10940: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Commented] (HDFS-10940) Reduce performance penalty of block caching when not used

2016-10-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15542788#comment-15542788 ] Kihwal Lee commented on HDFS-10940: --- +1 a straight-forward optimization for some of the most frequently

[jira] [Updated] (HDFS-10945) Fix the Findbugwaring FSNamesystem#renameTo(String, String, boolean) in branch-2

2016-10-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10945: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved

[jira] [Commented] (HDFS-10945) Fix the Findbugwaring FSNamesystem#renameTo(String, String, boolean) in branch-2

2016-10-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15542710#comment-15542710 ] Kihwal Lee commented on HDFS-10945: --- +1 > Fix the Findbugwaring FSNamesystem#renameTo(String, String,

<    4   5   6   7   8   9   10   11   12   13   >