[GitHub] [hadoop-ozone] codecov-io commented on pull request #1520: Merge master branch into HDDS-1880-Decom

2020-10-23 Thread GitBox
codecov-io commented on pull request #1520: URL: https://github.com/apache/hadoop-ozone/pull/1520#issuecomment-715660815 # [Codecov](https://codecov.io/gh/apache/hadoop-ozone/pull/1520?src=pr=h1) Report > :exclamation: No coverage uploaded for pull request base

[GitHub] [hadoop-ozone] GlenGeng commented on pull request #1518: HDDS-4386: Each EndpointStateMachine uses its own thread pool to talk with SCM/Recon

2020-10-23 Thread GitBox
GlenGeng commented on pull request #1518: URL: https://github.com/apache/hadoop-ozone/pull/1518#issuecomment-715656741 @avijayanhwx Thanks for looking at this PR. @ChenSammi Could you please create a Jira for slow Recon ? > Also, irrespective of whether there is a slow SCM or

[jira] [Commented] (HDDS-4342) Add DataNode state and transitions for a node going through upgrade

2020-10-23 Thread Prashant Pogde (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219968#comment-17219968 ] Prashant Pogde commented on HDDS-4342: -- A typical SCM sequence for driving datanodes through upgrade

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1497: HDDS-4345. Replace the deprecated Lock method

2020-10-23 Thread GitBox
xiaoyuyao commented on a change in pull request #1497: URL: https://github.com/apache/hadoop-ozone/pull/1497#discussion_r511191225 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/PrefixManagerImpl.java ## @@ -253,18 +253,18 @@ public boolean

[jira] [Created] (HDDS-4392) Add Recon architecture to docs

2020-10-23 Thread Vivek Ratnavel Subramanian (Jira)
Vivek Ratnavel Subramanian created HDDS-4392: Summary: Add Recon architecture to docs Key: HDDS-4392 URL: https://issues.apache.org/jira/browse/HDDS-4392 Project: Hadoop Distributed Data

[GitHub] [hadoop-ozone] avijayanhwx commented on pull request #1518: HDDS-4386: Each EndpointStateMachine uses its own thread pool to talk with SCM/Recon

2020-10-23 Thread GitBox
avijayanhwx commented on pull request #1518: URL: https://github.com/apache/hadoop-ozone/pull/1518#issuecomment-715582860 @GlenGeng Thanks for fixing this issue. Is there a JIRA created for the Recon slowness issue? Also, irrespective of whether there is a slow SCM or Recon,

[GitHub] [hadoop-ozone] avijayanhwx commented on a change in pull request #1486: HDDS-4296. SCM changes to process Layout Info in heartbeat request/response

2020-10-23 Thread GitBox
avijayanhwx commented on a change in pull request #1486: URL: https://github.com/apache/hadoop-ozone/pull/1486#discussion_r511140146 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/SCMNodeManager.java ## @@ -400,6 +405,66 @@ public void

[jira] [Updated] (HDDS-4391) UnixPath.toUri() is expensive

2020-10-23 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDDS-4391: -- Target Version/s: 1.1.0 > UnixPath.toUri() is expensive > - > >

[jira] [Created] (HDDS-4391) UnixPath.toUri() is expensive

2020-10-23 Thread Wei-Chiu Chuang (Jira)
Wei-Chiu Chuang created HDDS-4391: - Summary: UnixPath.toUri() is expensive Key: HDDS-4391 URL: https://issues.apache.org/jira/browse/HDDS-4391 Project: Hadoop Distributed Data Store Issue

[jira] [Comment Edited] (HDDS-4337) Implement RocksDB options cache for new datanode DB utilities

2020-10-23 Thread Ethan Rose (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219807#comment-17219807 ] Ethan Rose edited comment on HDDS-4337 at 10/23/20, 4:59 PM: - >From the

[jira] [Comment Edited] (HDDS-4337) Implement RocksDB options cache for new datanode DB utilities

2020-10-23 Thread Ethan Rose (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219807#comment-17219807 ] Ethan Rose edited comment on HDDS-4337 at 10/23/20, 4:58 PM: - >From the

[jira] [Comment Edited] (HDDS-4337) Implement RocksDB options cache for new datanode DB utilities

2020-10-23 Thread Ethan Rose (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219807#comment-17219807 ] Ethan Rose edited comment on HDDS-4337 at 10/23/20, 4:58 PM: - >From the

[jira] [Created] (HDDS-4390) Change ozone.om.db.dirs may fail OM start when security is enabled.

2020-10-23 Thread Xiaoyu Yao (Jira)
Xiaoyu Yao created HDDS-4390: Summary: Change ozone.om.db.dirs may fail OM start when security is enabled. Key: HDDS-4390 URL: https://issues.apache.org/jira/browse/HDDS-4390 Project: Hadoop

[jira] [Comment Edited] (HDDS-4337) Implement RocksDB options cache for new datanode DB utilities

2020-10-23 Thread Ethan Rose (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219807#comment-17219807 ] Ethan Rose edited comment on HDDS-4337 at 10/23/20, 4:57 PM: - >From the

[jira] [Comment Edited] (HDDS-4337) Implement RocksDB options cache for new datanode DB utilities

2020-10-23 Thread Ethan Rose (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219807#comment-17219807 ] Ethan Rose edited comment on HDDS-4337 at 10/23/20, 4:56 PM: - >From the

[jira] [Commented] (HDDS-4337) Implement RocksDB options cache for new datanode DB utilities

2020-10-23 Thread Ethan Rose (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219807#comment-17219807 ] Ethan Rose commented on HDDS-4337: -- >From the[ RocksDB DBOptions java documentation|

[jira] [Comment Edited] (HDDS-4337) Implement RocksDB options cache for new datanode DB utilities

2020-10-23 Thread Ethan Rose (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219807#comment-17219807 ] Ethan Rose edited comment on HDDS-4337 at 10/23/20, 4:55 PM: - >From the

[jira] [Created] (HDDS-4389) Disable caching when performing Trash specific operations

2020-10-23 Thread Sadanand Shenoy (Jira)
Sadanand Shenoy created HDDS-4389: - Summary: Disable caching when performing Trash specific operations Key: HDDS-4389 URL: https://issues.apache.org/jira/browse/HDDS-4389 Project: Hadoop Distributed

[jira] [Updated] (HDDS-4389) Disable caching when performing Trash specific operations

2020-10-23 Thread Sadanand Shenoy (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sadanand Shenoy updated HDDS-4389: -- Parent: HDDS-3367 Issue Type: Sub-task (was: Improvement) > Disable caching when

[jira] [Updated] (HDDS-4387) Create a Trash File System that will be used by Trash Emptier

2020-10-23 Thread Sadanand Shenoy (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sadanand Shenoy updated HDDS-4387: -- Parent: HDDS-3367 Issue Type: Sub-task (was: Improvement) > Create a Trash File System

[jira] [Updated] (HDDS-4307) Start Trash Emptier in Ozone Manager

2020-10-23 Thread Sadanand Shenoy (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sadanand Shenoy updated HDDS-4307: -- Summary: Start Trash Emptier in Ozone Manager (was: Start Background Service for Trash

[GitHub] [hadoop-ozone] sodonnel opened a new pull request #1520: Merge master branch into HDDS-1880-Decom

2020-10-23 Thread GitBox
sodonnel opened a new pull request #1520: URL: https://github.com/apache/hadoop-ozone/pull/1520 ## What changes were proposed in this pull request? Merge master into the decommission branch. This was a clean merge with no conflicts at all. ## What is the link to the Apache

[GitHub] [hadoop-ozone] lokeshj1703 commented on a change in pull request #1519: HDDS-4388. Make writeStateMachineTimeout retry count proportional to node failure timeout

2020-10-23 Thread GitBox
lokeshj1703 commented on a change in pull request #1519: URL: https://github.com/apache/hadoop-ozone/pull/1519#discussion_r510844138 ## File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/XceiverServerRatis.java

[GitHub] [hadoop-ozone] lokeshj1703 commented on pull request #1513: HDDS-4366. SCM deletion service should delete configured number of blocks every interval.

2020-10-23 Thread GitBox
lokeshj1703 commented on pull request #1513: URL: https://github.com/apache/hadoop-ozone/pull/1513#issuecomment-715293590 @bshashikant Thanks for reviewing the PR! I have added a commit which addresses the comments. This is

[GitHub] [hadoop-ozone] lokeshj1703 commented on a change in pull request #1513: HDDS-4366. SCM deletion service should delete configured number of blocks every interval.

2020-10-23 Thread GitBox
lokeshj1703 commented on a change in pull request #1513: URL: https://github.com/apache/hadoop-ozone/pull/1513#discussion_r510827330 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/block/DeletedBlockLogImpl.java ## @@ -323,31 +322,54 @@ public

[GitHub] [hadoop-ozone] timmylicheng commented on a change in pull request #1514: HDDS-4191 Add failover proxy for SCM container location.

2020-10-23 Thread GitBox
timmylicheng commented on a change in pull request #1514: URL: https://github.com/apache/hadoop-ozone/pull/1514#discussion_r510791922 ## File path: hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/proxy/SCMContainerLocationFailoverProxyProvider.java ## @@ -0,0

[jira] [Updated] (HDDS-4388) Make writeStateMachineTimeout retry count proportional to node failure timeout

2020-10-23 Thread Shashikant Banerjee (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDDS-4388: -- Description: Currently, in ratis "writeStateMachinecall" gets retried indefinitely in

[jira] [Updated] (HDDS-4388) Make writeStateMachineTimeout retry count proportional to node failure timeout

2020-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-4388: - Labels: pull-request-available (was: ) > Make writeStateMachineTimeout retry count proportional

[GitHub] [hadoop-ozone] bshashikant opened a new pull request #1519: HDDS-4388. Make writeStateMachineTimeout retry count proportional to node failure timeout

2020-10-23 Thread GitBox
bshashikant opened a new pull request #1519: URL: https://github.com/apache/hadoop-ozone/pull/1519 ## What changes were proposed in this pull request? Currently, in ratis "writeStateMachinecall" gets retried indefinitely in event of a timeout. In case, where disks are

[GitHub] [hadoop-ozone] bshashikant commented on a change in pull request #1513: HDDS-4366. SCM deletion service should delete configured number of blocks every interval.

2020-10-23 Thread GitBox
bshashikant commented on a change in pull request #1513: URL: https://github.com/apache/hadoop-ozone/pull/1513#discussion_r510761184 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/block/SCMBlockDeletingService.java ## @@ -127,62 +112,65 @@

[jira] [Created] (HDDS-4388) Make writeStateMachineTimeout retry count proportional to node failure timeout

2020-10-23 Thread Shashikant Banerjee (Jira)
Shashikant Banerjee created HDDS-4388: - Summary: Make writeStateMachineTimeout retry count proportional to node failure timeout Key: HDDS-4388 URL: https://issues.apache.org/jira/browse/HDDS-4388

[jira] [Updated] (HDDS-4386) Each EndpointStateMachine uses its own thread pool to talk with SCM/Recon

2020-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-4386: - Labels: pull-request-available (was: ) > Each EndpointStateMachine uses its own thread pool to

[GitHub] [hadoop-ozone] GlenGeng opened a new pull request #1518: HDDS-4386: Each EndpointStateMachine uses its own thread pool to talk with SCM/Recon

2020-10-23 Thread GitBox
GlenGeng opened a new pull request #1518: URL: https://github.com/apache/hadoop-ozone/pull/1518 ## What changes were proposed in this pull request? In Tencent production environment, after start Recon for a while, we got warnings that all DNs become stale/dead at SCM side. After

[jira] [Created] (HDDS-4387) Create a Trash File System that will be used by Trash Emptier

2020-10-23 Thread Sadanand Shenoy (Jira)
Sadanand Shenoy created HDDS-4387: - Summary: Create a Trash File System that will be used by Trash Emptier Key: HDDS-4387 URL: https://issues.apache.org/jira/browse/HDDS-4387 Project: Hadoop