[jira] [Updated] (HDFS-15638) Make Hive tables directory permission check flat

[ https://issues.apache.org/jira/browse/HDFS-15638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang updated HDFS-15638: --- Description: Problem: Currently, when a user tries to accesses a file he/she needs the permissions

[jira] [Created] (HDFS-15638) Make Hive tables directory permission check flat

Xinli Shang created HDFS-15638: -- Summary: Make Hive tables directory permission check flat Key: HDFS-15638 URL: https://issues.apache.org/jira/browse/HDFS-15638 Project: Hadoop HDFS Issue

[jira] [Commented] (HDFS-15618) Improve datanode shutdown latency

[ https://issues.apache.org/jira/browse/HDFS-15618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17215697#comment-17215697 ] Ahmed Hussein commented on HDFS-15618: -- I checked the failing Junit tests. They are unrelated to the

[jira] [Commented] (HDFS-15618) Improve datanode shutdown latency

[ https://issues.apache.org/jira/browse/HDFS-15618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17215691#comment-17215691 ] Hadoop QA commented on HDFS-15618: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-15634) Invalidate block on decommissioning DataNode after replication

[ https://issues.apache.org/jira/browse/HDFS-15634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17215676#comment-17215676 ] Stephen O'Donnell commented on HDFS-15634: -- The issue you have seen is a fairly extreme example

[jira] [Commented] (HDFS-15634) Invalidate block on decommissioning DataNode after replication

[ https://issues.apache.org/jira/browse/HDFS-15634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17215654#comment-17215654 ] Fengnan Li commented on HDFS-15634: --- Thanks for the comment [~sodonnell]. More context here: We were

[jira] [Updated] (HDFS-15634) Invalidate block on decommissioning DataNode after replication

[ https://issues.apache.org/jira/browse/HDFS-15634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fengnan Li updated HDFS-15634: -- Attachment: write lock.png > Invalidate block on decommissioning DataNode after replication >

[jira] [Commented] (HDFS-15630) RBF: Fix wrong client IP info in CallerContext when requests mount points with multi-destinations.

[ https://issues.apache.org/jira/browse/HDFS-15630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17215650#comment-17215650 ] Íñigo Goiri commented on HDFS-15630: In addition to [~ferhui] comments, a couple minor ones. Let's

[jira] [Commented] (HDFS-14383) Compute datanode load based on StoragePolicy

[ https://issues.apache.org/jira/browse/HDFS-14383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17215648#comment-17215648 ] Íñigo Goiri commented on HDFS-14383: [^HDFS-14383-02.patch] LGTM. +1 [~LiJinglun], can you verify

[jira] [Commented] (HDFS-15618) Improve datanode shutdown latency

[ https://issues.apache.org/jira/browse/HDFS-15618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17215646#comment-17215646 ] Ahmed Hussein commented on HDFS-15618: -- Thanks [~kihwal]! I thinks that's a good idea to have a

[jira] [Updated] (HDFS-15618) Improve datanode shutdown latency

[ https://issues.apache.org/jira/browse/HDFS-15618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmed Hussein updated HDFS-15618: - Attachment: HDFS-15618.004.patch > Improve datanode shutdown latency >

[jira] [Commented] (HDFS-15628) HttpFS server throws NPE if a file is a symlink

[ https://issues.apache.org/jira/browse/HDFS-15628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17215497#comment-17215497 ] Kihwal Lee commented on HDFS-15628: --- I've committed this to trunk, branch-3.3, branch-3.2 and

[jira] [Updated] (HDFS-15628) HttpFS server throws NPE if a file is a symlink

[ https://issues.apache.org/jira/browse/HDFS-15628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-15628: -- Fix Version/s: 3.1.5 > HttpFS server throws NPE if a file is a symlink >

[jira] [Updated] (HDFS-15627) Audit log deletes before collecting blocks

[ https://issues.apache.org/jira/browse/HDFS-15627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-15627: -- Fix Version/s: 3.2.3 3.1.5 3.4.0 3.3.1

[jira] [Updated] (HDFS-15627) Audit log deletes before collecting blocks

[ https://issues.apache.org/jira/browse/HDFS-15627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-15627: -- Summary: Audit log deletes before collecting blocks (was: Audit log deletes after edit is written) >

[jira] [Commented] (HDFS-15627) Audit log deletes after edit is written

[ https://issues.apache.org/jira/browse/HDFS-15627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17215476#comment-17215476 ] Kihwal Lee commented on HDFS-15627: --- +1 lgtm > Audit log deletes after edit is written >

[jira] [Commented] (HDFS-15618) Improve datanode shutdown latency

[ https://issues.apache.org/jira/browse/HDFS-15618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17215454#comment-17215454 ] Kihwal Lee commented on HDFS-15618: --- It can be as fancy as adding a builder method for setting it (for

[jira] [Commented] (HDFS-15618) Improve datanode shutdown latency

[ https://issues.apache.org/jira/browse/HDFS-15618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17215452#comment-17215452 ] Kihwal Lee commented on HDFS-15618: --- In production, there is no harm in exiting after waiting for 5

[jira] [Work logged] (HDFS-15635) ViewFileSystemOverloadScheme support specifying mount table loader imp through conf

[ https://issues.apache.org/jira/browse/HDFS-15635?focusedWorklogId=501492=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-501492 ] ASF GitHub Bot logged work on HDFS-15635: - Author: ASF GitHub Bot

[jira] [Updated] (HDFS-15637) Viewfs should make mount-table to read from central place

[ https://issues.apache.org/jira/browse/HDFS-15637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junfan Zhang updated HDFS-15637: Component/s: viewfs > Viewfs should make mount-table to read from central place >

[jira] [Comment Edited] (HDFS-15637) Viewfs should make mount-table to read from central place

[ https://issues.apache.org/jira/browse/HDFS-15637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17215296#comment-17215296 ] Junfan Zhang edited comment on HDFS-15637 at 10/16/20, 9:42 AM: Hi

[jira] [Commented] (HDFS-15637) Viewfs should make mount-table to read from central place

[ https://issues.apache.org/jira/browse/HDFS-15637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17215296#comment-17215296 ] Junfan Zhang commented on HDFS-15637: - Hi [~umamaheswararao] Can you please take a look at the ISSUE,

[jira] [Updated] (HDFS-15637) Viewfs should make mount-table to read from central place

[ https://issues.apache.org/jira/browse/HDFS-15637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junfan Zhang updated HDFS-15637: Description: Like [HDFS-15637|https://issues.apache.org/jira/browse/HDFS-15637]. Viewfs should

[jira] [Created] (HDFS-15637) Viewfs should make mount-table to read from central place

Junfan Zhang created HDFS-15637: --- Summary: Viewfs should make mount-table to read from central place Key: HDFS-15637 URL: https://issues.apache.org/jira/browse/HDFS-15637 Project: Hadoop HDFS

[jira] [Comment Edited] (HDFS-15634) Invalidate block on decommissioning DataNode after replication

[ https://issues.apache.org/jira/browse/HDFS-15634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17215288#comment-17215288 ] Stephen O'Donnell edited comment on HDFS-15634 at 10/16/20, 9:27 AM: -

[jira] [Commented] (HDFS-15634) Invalidate block on decommissioning DataNode after replication

[ https://issues.apache.org/jira/browse/HDFS-15634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17215288#comment-17215288 ] Stephen O'Donnell commented on HDFS-15634: -- {quote} Proposal: Invalidate these blocks once they

[jira] [Updated] (HDFS-15635) ViewFileSystemOverloadScheme support specifying mount table loader imp through conf

[ https://issues.apache.org/jira/browse/HDFS-15635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junfan Zhang updated HDFS-15635: Description: According to HDFS-15289, the default mountable loader is

[jira] [Comment Edited] (HDFS-15635) ViewFileSystemOverloadScheme support specifying mount table loader imp through conf

[ https://issues.apache.org/jira/browse/HDFS-15635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17215223#comment-17215223 ] Junfan Zhang edited comment on HDFS-15635 at 10/16/20, 8:26 AM: Hi

[jira] [Updated] (HDFS-15635) ViewFileSystemOverloadScheme support specifying mount table loader imp through conf

[ https://issues.apache.org/jira/browse/HDFS-15635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junfan Zhang updated HDFS-15635: Description: According to HDFS-15289, the default mountable loader is

[jira] [Resolved] (HDFS-15636) NameNode computes load by group when choosing datanodes.

[ https://issues.apache.org/jira/browse/HDFS-15636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jinglun resolved HDFS-15636. Resolution: Duplicate Duplicate with HDFS-14383 > NameNode computes load by group when choosing

[jira] [Commented] (HDFS-14383) Compute datanode load based on StoragePolicy

[ https://issues.apache.org/jira/browse/HDFS-14383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17215242#comment-17215242 ] Jinglun commented on HDFS-14383: I met the same problem recently. This patch makes sense to me. Thanks

[jira] [Commented] (HDFS-15636) NameNode computes load by group when choosing datanodes.

[ https://issues.apache.org/jira/browse/HDFS-15636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17215232#comment-17215232 ] Jinglun commented on HDFS-15636: Hi [~ayushtkn], seems related, I'll give it a check. > NameNode

[jira] [Commented] (HDFS-15636) NameNode computes load by group when choosing datanodes.

[ https://issues.apache.org/jira/browse/HDFS-15636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17215231#comment-17215231 ] Jinglun commented on HDFS-15636: We can implement this by 2 steps: # Let NameNode support computing load

[jira] [Commented] (HDFS-15636) NameNode computes load by group when choosing datanodes.

[ https://issues.apache.org/jira/browse/HDFS-15636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17215228#comment-17215228 ] Ayush Saxena commented on HDFS-15636: - Similar to HDFS-14383? > NameNode computes load by group when

[jira] [Created] (HDFS-15636) NameNode computes load by group when choosing datanodes.

Jinglun created HDFS-15636: -- Summary: NameNode computes load by group when choosing datanodes. Key: HDFS-15636 URL: https://issues.apache.org/jira/browse/HDFS-15636 Project: Hadoop HDFS Issue Type:

[jira] [Assigned] (HDFS-15636) NameNode computes load by group when choosing datanodes.

[ https://issues.apache.org/jira/browse/HDFS-15636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jinglun reassigned HDFS-15636: -- Assignee: Jinglun > NameNode computes load by group when choosing datanodes. >

[jira] [Commented] (HDFS-15635) ViewFileSystemOverloadScheme support specifying mount table loader imp through conf

[ https://issues.apache.org/jira/browse/HDFS-15635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17215223#comment-17215223 ] Junfan Zhang commented on HDFS-15635: - Hi [~umamaheswararao] Please review it.Thanks~ >

[jira] [Updated] (HDFS-15635) ViewFileSystemOverloadScheme support specifying mount table loader imp through conf

[ https://issues.apache.org/jira/browse/HDFS-15635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junfan Zhang updated HDFS-15635: External issue ID: https://github.com/apache/hadoop/pull/2389 > ViewFileSystemOverloadScheme

[jira] [Work logged] (HDFS-15635) ViewFileSystemOverloadScheme support specifying mount table loader imp through conf

[ https://issues.apache.org/jira/browse/HDFS-15635?focusedWorklogId=501439=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-501439 ] ASF GitHub Bot logged work on HDFS-15635: - Author: ASF GitHub Bot

[jira] [Updated] (HDFS-15635) ViewFileSystemOverloadScheme support specifying mount table loader imp through conf

[ https://issues.apache.org/jira/browse/HDFS-15635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDFS-15635: -- Labels: pull-request-available (was: ) > ViewFileSystemOverloadScheme support specifying

[jira] [Created] (HDFS-15635) ViewFileSystemOverloadScheme support specifying mount table loader imp through conf

Junfan Zhang created HDFS-15635: --- Summary: ViewFileSystemOverloadScheme support specifying mount table loader imp through conf Key: HDFS-15635 URL: https://issues.apache.org/jira/browse/HDFS-15635

[jira] [Work logged] (HDFS-15634) Invalidate block on decommissioning DataNode after replication

[ https://issues.apache.org/jira/browse/HDFS-15634?focusedWorklogId=501431=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-501431 ] ASF GitHub Bot logged work on HDFS-15634: - Author: ASF GitHub Bot

[jira] [Work logged] (HDFS-15634) Invalidate block on decommissioning DataNode after replication

[ https://issues.apache.org/jira/browse/HDFS-15634?focusedWorklogId=501429=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-501429 ] ASF GitHub Bot logged work on HDFS-15634: - Author: ASF GitHub Bot

[jira] [Work logged] (HDFS-15634) Invalidate block on decommissioning DataNode after replication

[ https://issues.apache.org/jira/browse/HDFS-15634?focusedWorklogId=501430=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-501430 ] ASF GitHub Bot logged work on HDFS-15634: - Author: ASF GitHub Bot