[jira] [Updated] (HDDS-7315) SCM order of close of dbstore and other services

2022-10-12 Thread Sammi Chen (Jira)
[ https://issues.apache.org/jira/browse/HDDS-7315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sammi Chen updated HDDS-7315: - Fix Version/s: 1.3.0 > SCM order of close of dbstore and other services >

[GitHub] [ozone] navinko commented on a diff in pull request #3819: HDDS-7308. Updated Doc for OM HA Setup

2022-10-12 Thread GitBox
navinko commented on code in PR #3819: URL: https://github.com/apache/ozone/pull/3819#discussion_r994223827 ## hadoop-hdds/docs/content/feature/OM-HA.zh.md: ## @@ -58,7 +58,7 @@ Ozone Manager 和 Storage Container Manager 都支持 HA。在这种模式下, ``` Review Comment: Thanks @ChenSammi

[GitHub] [ozone] navinko commented on a diff in pull request #3819: HDDS-7308. Updated Doc for OM HA Setup

2022-10-12 Thread GitBox
navinko commented on code in PR #3819: URL: https://github.com/apache/ozone/pull/3819#discussion_r994222895 ## hadoop-hdds/docs/content/feature/OM-HA.md: ## @@ -56,7 +56,7 @@ This logical name is called `serviceId` and can be configured in the `ozone-site ``` Review Comment:

[GitHub] [ozone] ChenSammi commented on a diff in pull request #3819: HDDS-7308. Updated Doc for OM HA Setup

2022-10-12 Thread GitBox
ChenSammi commented on code in PR #3819: URL: https://github.com/apache/ozone/pull/3819#discussion_r994218467 ## hadoop-hdds/docs/content/feature/OM-HA.zh.md: ## @@ -58,7 +58,7 @@ Ozone Manager 和 Storage Container Manager 都支持 HA。在这种模式下, ``` Review Comment: @adoroszlai , so

[GitHub] [ozone] ChenSammi commented on a diff in pull request #3819: HDDS-7308. Updated Doc for OM HA Setup

2022-10-12 Thread GitBox
ChenSammi commented on code in PR #3819: URL: https://github.com/apache/ozone/pull/3819#discussion_r994218467 ## hadoop-hdds/docs/content/feature/OM-HA.zh.md: ## @@ -58,7 +58,7 @@ Ozone Manager 和 Storage Container Manager 都支持 HA。在这种模式下, ``` Review Comment: @adoroszlai , so

[GitHub] [ozone] navinko commented on a diff in pull request #3819: HDDS-7308. Updated Doc for OM HA Setup

2022-10-12 Thread GitBox
navinko commented on code in PR #3819: URL: https://github.com/apache/ozone/pull/3819#discussion_r994218400 ## hadoop-hdds/docs/content/feature/OM-HA.md: ## @@ -56,7 +56,7 @@ This logical name is called `serviceId` and can be configured in the `ozone-site ``` Review Comment:

[GitHub] [ozone] ChenSammi merged pull request #3819: HDDS-7308. Updated Doc for OM HA Setup

2022-10-12 Thread GitBox
ChenSammi merged PR #3819: URL: https://github.com/apache/ozone/pull/3819 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ozone.apach

[jira] [Commented] (HDDS-7318) Publish Ozone 1.2.1 documentation

2022-10-12 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-7318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616798#comment-17616798 ] Attila Doroszlai commented on HDDS-7318: bq. Apache Ozone is moved from a Hadoop s

[GitHub] [ozone] siddhantsangwan commented on a diff in pull request #3831: HDDS-6893. EC: ReplicationManager - move the empty container handling into RM from Legacy

2022-10-12 Thread GitBox
siddhantsangwan commented on code in PR #3831: URL: https://github.com/apache/ozone/pull/3831#discussion_r994188204 ## hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/ReplicationManager.java: ## @@ -362,6 +364,35 @@ public void sendCloseCont

[jira] [Updated] (HDDS-6893) EC: ReplicationManager - move the empty container handling into RM from Legacy

2022-10-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-6893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-6893: - Labels: pull-request-available (was: ) > EC: ReplicationManager - move the empty container handli

[GitHub] [ozone] siddhantsangwan opened a new pull request, #3831: HDDS-6893. EC: ReplicationManager - move the empty container handling into RM from Legacy

2022-10-12 Thread GitBox
siddhantsangwan opened a new pull request, #3831: URL: https://github.com/apache/ozone/pull/3831 ## What changes were proposed in this pull request? Created a handler for empty containers. This mostly contains the same logic as the legacy RM for checking if a container is closed and e

[GitHub] [ozone] adoroszlai commented on a diff in pull request #3819: HDDS-7308. Updated Doc for OM HA Setup

2022-10-12 Thread GitBox
adoroszlai commented on code in PR #3819: URL: https://github.com/apache/ozone/pull/3819#discussion_r994176894 ## hadoop-hdds/docs/content/feature/OM-HA.zh.md: ## @@ -58,7 +58,7 @@ Ozone Manager 和 Storage Container Manager 都支持 HA。在这种模式下, ``` Review Comment: Same here:

[GitHub] [ozone] navinko commented on pull request #3819: HDDS-7308. Updated Doc for OM HA Setup

2022-10-12 Thread GitBox
navinko commented on PR #3819: URL: https://github.com/apache/ozone/pull/3819#issuecomment-1277035327 Thanks @ChenSammi for reviewing . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [ozone] ChenSammi commented on pull request #3819: HDDS-7308. Updated Doc for OM HA Setup

2022-10-12 Thread GitBox
ChenSammi commented on PR #3819: URL: https://github.com/apache/ozone/pull/3819#issuecomment-1277030758 LGTM + 1. Thanks @navinko . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[jira] [Resolved] (HDDS-7315) SCM order of close of dbstore and other services

2022-10-12 Thread Sammi Chen (Jira)
[ https://issues.apache.org/jira/browse/HDDS-7315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sammi Chen resolved HDDS-7315. -- Resolution: Fixed > SCM order of close of dbstore and other services > -

[GitHub] [ozone] ChenSammi merged pull request #3830: HDDS-7315. SCM order of close of dbstore and other services

2022-10-12 Thread GitBox
ChenSammi merged PR #3830: URL: https://github.com/apache/ozone/pull/3830 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ozone.apach

[GitHub] [ozone] ChenSammi commented on pull request #3830: HDDS-7315. SCM order of close of dbstore and other services

2022-10-12 Thread GitBox
ChenSammi commented on PR #3830: URL: https://github.com/apache/ozone/pull/3830#issuecomment-1277028634 LGTM + 1. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

[GitHub] [ozone] sumitagrawl commented on pull request #3830: HDDS-7315. SCM order of close of dbstore and other services

2022-10-12 Thread GitBox
sumitagrawl commented on PR #3830: URL: https://github.com/apache/ozone/pull/3830#issuecomment-1277005808 @ChenSammi @nandakumar131 Please review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Updated] (HDDS-6492) Add metric for failed container moves

2022-10-12 Thread Sumit Agrawal (Jira)
[ https://issues.apache.org/jira/browse/HDDS-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Agrawal updated HDDS-6492: Fix Version/s: 1.3.0 > Add metric for failed container moves > -

[jira] [Resolved] (HDDS-6492) Add metric for failed container moves

2022-10-12 Thread Sumit Agrawal (Jira)
[ https://issues.apache.org/jira/browse/HDDS-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Agrawal resolved HDDS-6492. - Resolution: Fixed > Add metric for failed container moves > -

[jira] [Updated] (HDDS-7315) SCM order of close of dbstore and other services

2022-10-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-7315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-7315: - Labels: pull-request-available (was: ) > SCM order of close of dbstore and other services > -

[GitHub] [ozone] sumitagrawl opened a new pull request, #3830: HDDS-7315 SCM order of close of dbstore and other services

2022-10-12 Thread GitBox
sumitagrawl opened a new pull request, #3830: URL: https://github.com/apache/ozone/pull/3830 ## What changes were proposed in this pull request? scmMetaStore to be close at last in sequence, as other service like metric may be using this to access tables. ## What is the link to

[GitHub] [ozone] Xushaohong commented on a diff in pull request #3811: HDDS-7300. Race condition between full data scan and block deletion

2022-10-12 Thread GitBox
Xushaohong commented on code in PR #3811: URL: https://github.com/apache/ozone/pull/3811#discussion_r994046535 ## hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainerCheck.java: ## @@ -230,44 +232,98 @@ private void scanData(Da

[GitHub] [ozone] GeorgeJahad commented on a diff in pull request #3800: HDDS-7231. Integrate the GetKeyInfo API to key read flows

2022-10-12 Thread GitBox
GeorgeJahad commented on code in PR #3800: URL: https://github.com/apache/ozone/pull/3800#discussion_r994020811 ## hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java: ## @@ -1437,6 +1421,34 @@ public OzoneKeyDetails getKeyDetails( getInp

[GitHub] [ozone] GeorgeJahad commented on pull request #3800: HDDS-7231. Integrate the GetKeyInfo API to key read flows

2022-10-12 Thread GitBox
GeorgeJahad commented on PR #3800: URL: https://github.com/apache/ozone/pull/3800#issuecomment-1276898359 There is a call to lookupFile() here: https://github.com/apache/ozone/blob/1ea4c2adc640a02508696bf346c7f1f7b6e0afd9/hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rp

[GitHub] [ozone] GeorgeJahad commented on pull request #3800: HDDS-7231. Integrate the GetKeyInfo API to key read flows

2022-10-12 Thread GitBox
GeorgeJahad commented on PR #3800: URL: https://github.com/apache/ozone/pull/3800#issuecomment-1276898159 There is a call to looupKey here: https://github.com/apache/ozone/blob/1ea4c2adc640a02508696bf346c7f1f7b6e0afd9/hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/Rp

[jira] [Commented] (HDDS-7319) To support s3g usernames with Fair Call Queue from hadoop-commons

2022-10-12 Thread Neil Joshi (Jira)
[ https://issues.apache.org/jira/browse/HDDS-7319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616737#comment-17616737 ] Neil Joshi commented on HDDS-7319: -- cc [~smeng]  > To support s3g usernames with Fair Ca

[jira] [Created] (HDDS-7319) To support s3g usernames with Fair Call Queue from hadoop-commons

2022-10-12 Thread Neil Joshi (Jira)
Neil Joshi created HDDS-7319: Summary: To support s3g usernames with Fair Call Queue from hadoop-commons Key: HDDS-7319 URL: https://issues.apache.org/jira/browse/HDDS-7319 Project: Apache Ozone

[GitHub] [ozone] duongkame commented on a diff in pull request #3800: HDDS-7231. Integrate the GetKeyInfo API to key read flows

2022-10-12 Thread GitBox
duongkame commented on code in PR #3800: URL: https://github.com/apache/ozone/pull/3800#discussion_r993964476 ## hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockInputStream.java: ## @@ -302,20 +310,24 @@ protected synchronized int readWithStrategy(ByteR

[GitHub] [ozone] smengcl commented on pull request #3824: HDDS-7281. [Snapshot] Handle RocksDB compaction DAG persistence and reconstruction

2022-10-12 Thread GitBox
smengcl commented on PR #3824: URL: https://github.com/apache/ozone/pull/3824#issuecomment-1276821184 > This is in response to the discussion here: [#3786 (comment)](https://github.com/apache/ozone/pull/3786#discussion_r992935645) > > To me there is value in the conceptual simplicity

[jira] [Resolved] (HDDS-5650) Wrong mock assumption in TestOmMetrics#testBucketOps

2022-10-12 Thread Aswin Shakil Balasubramanian (Jira)
[ https://issues.apache.org/jira/browse/HDDS-5650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aswin Shakil Balasubramanian resolved HDDS-5650. Resolution: Fixed This is a part of the test to inject failure to te

[GitHub] [ozone] GeorgeJahad commented on a diff in pull request #3824: HDDS-7281. [Snapshot] Handle RocksDB compaction DAG persistence and reconstruction

2022-10-12 Thread GitBox
GeorgeJahad commented on code in PR #3824: URL: https://github.com/apache/ozone/pull/3824#discussion_r993956506 ## hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/ozone/rocksdiff/RocksDBCheckpointDiffer.java: ## @@ -129,6 +201,24 @@ public RocksDBCheckpointDiffer(

[GitHub] [ozone] GeorgeJahad commented on a diff in pull request #3824: HDDS-7281. [Snapshot] Handle RocksDB compaction DAG persistence and reconstruction

2022-10-12 Thread GitBox
GeorgeJahad commented on code in PR #3824: URL: https://github.com/apache/ozone/pull/3824#discussion_r993954747 ## hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/ozone/rocksdiff/RocksDBCheckpointDiffer.java: ## @@ -145,16 +235,29 @@ public RocksDBCheckpointDiffer

[GitHub] [ozone] GeorgeJahad commented on a diff in pull request #3824: HDDS-7281. [Snapshot] Handle RocksDB compaction DAG persistence and reconstruction

2022-10-12 Thread GitBox
GeorgeJahad commented on code in PR #3824: URL: https://github.com/apache/ozone/pull/3824#discussion_r993951742 ## hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/ozone/rocksdiff/RocksDBCheckpointDiffer.java: ## @@ -114,9 +113,82 @@ public class RocksDBCheckpointD

[GitHub] [ozone] GeorgeJahad commented on a diff in pull request #3824: HDDS-7281. [Snapshot] Handle RocksDB compaction DAG persistence and reconstruction

2022-10-12 Thread GitBox
GeorgeJahad commented on code in PR #3824: URL: https://github.com/apache/ozone/pull/3824#discussion_r993959701 ## hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/ozone/rocksdiff/RocksDBCheckpointDiffer.java: ## @@ -145,16 +235,29 @@ public RocksDBCheckpointDiffer

[GitHub] [ozone] GeorgeJahad commented on a diff in pull request #3824: HDDS-7281. [Snapshot] Handle RocksDB compaction DAG persistence and reconstruction

2022-10-12 Thread GitBox
GeorgeJahad commented on code in PR #3824: URL: https://github.com/apache/ozone/pull/3824#discussion_r993958846 ## hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmSnapshotManager.java: ## @@ -131,6 +132,22 @@ public static DBCheckpoint createOmSnapshotCheck

[GitHub] [ozone] GeorgeJahad commented on a diff in pull request #3824: HDDS-7281. [Snapshot] Handle RocksDB compaction DAG persistence and reconstruction

2022-10-12 Thread GitBox
GeorgeJahad commented on code in PR #3824: URL: https://github.com/apache/ozone/pull/3824#discussion_r993958198 ## hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/ozone/rocksdiff/RocksDBCheckpointDiffer.java: ## @@ -746,19 +950,71 @@ public MutableGraph getCompac

[GitHub] [ozone] GeorgeJahad commented on a diff in pull request #3824: HDDS-7281. [Snapshot] Handle RocksDB compaction DAG persistence and reconstruction

2022-10-12 Thread GitBox
GeorgeJahad commented on code in PR #3824: URL: https://github.com/apache/ozone/pull/3824#discussion_r993957587 ## hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/ozone/rocksdiff/RocksDBCheckpointDiffer.java: ## @@ -230,196 +330,154 @@ public void createCheckPoint

[GitHub] [ozone] GeorgeJahad commented on a diff in pull request #3824: HDDS-7281. [Snapshot] Handle RocksDB compaction DAG persistence and reconstruction

2022-10-12 Thread GitBox
GeorgeJahad commented on code in PR #3824: URL: https://github.com/apache/ozone/pull/3824#discussion_r993956506 ## hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/ozone/rocksdiff/RocksDBCheckpointDiffer.java: ## @@ -129,6 +201,24 @@ public RocksDBCheckpointDiffer(

[GitHub] [ozone] GeorgeJahad commented on a diff in pull request #3824: HDDS-7281. [Snapshot] Handle RocksDB compaction DAG persistence and reconstruction

2022-10-12 Thread GitBox
GeorgeJahad commented on code in PR #3824: URL: https://github.com/apache/ozone/pull/3824#discussion_r993956025 ## hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/ozone/rocksdiff/RocksDBCheckpointDiffer.java: ## @@ -230,196 +330,154 @@ public void createCheckPoint

[GitHub] [ozone] GeorgeJahad commented on a diff in pull request #3824: HDDS-7281. [Snapshot] Handle RocksDB compaction DAG persistence and reconstruction

2022-10-12 Thread GitBox
GeorgeJahad commented on code in PR #3824: URL: https://github.com/apache/ozone/pull/3824#discussion_r993955658 ## hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/ozone/rocksdiff/RocksDBCheckpointDiffer.java: ## @@ -230,196 +330,154 @@ public void createCheckPoint

[GitHub] [ozone] GeorgeJahad commented on a diff in pull request #3824: HDDS-7281. [Snapshot] Handle RocksDB compaction DAG persistence and reconstruction

2022-10-12 Thread GitBox
GeorgeJahad commented on code in PR #3824: URL: https://github.com/apache/ozone/pull/3824#discussion_r993954747 ## hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/ozone/rocksdiff/RocksDBCheckpointDiffer.java: ## @@ -145,16 +235,29 @@ public RocksDBCheckpointDiffer

[GitHub] [ozone] GeorgeJahad commented on a diff in pull request #3824: HDDS-7281. [Snapshot] Handle RocksDB compaction DAG persistence and reconstruction

2022-10-12 Thread GitBox
GeorgeJahad commented on code in PR #3824: URL: https://github.com/apache/ozone/pull/3824#discussion_r993951742 ## hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/ozone/rocksdiff/RocksDBCheckpointDiffer.java: ## @@ -114,9 +113,82 @@ public class RocksDBCheckpointD

[GitHub] [ozone] GeorgeJahad commented on a diff in pull request #3824: HDDS-7281. [Snapshot] Handle RocksDB compaction DAG persistence and reconstruction

2022-10-12 Thread GitBox
GeorgeJahad commented on code in PR #3824: URL: https://github.com/apache/ozone/pull/3824#discussion_r993951742 ## hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/ozone/rocksdiff/RocksDBCheckpointDiffer.java: ## @@ -114,9 +113,82 @@ public class RocksDBCheckpointD

[GitHub] [ozone] GeorgeJahad commented on a diff in pull request #3824: HDDS-7281. [Snapshot] Handle RocksDB compaction DAG persistence and reconstruction

2022-10-12 Thread GitBox
GeorgeJahad commented on code in PR #3824: URL: https://github.com/apache/ozone/pull/3824#discussion_r993951315 ## hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/ozone/rocksdiff/RocksDBCheckpointDiffer.java: ## @@ -114,9 +113,82 @@ public class RocksDBCheckpointD

[GitHub] [ozone] GeorgeJahad commented on a diff in pull request #3824: HDDS-7281. [Snapshot] Handle RocksDB compaction DAG persistence and reconstruction

2022-10-12 Thread GitBox
GeorgeJahad commented on code in PR #3824: URL: https://github.com/apache/ozone/pull/3824#discussion_r993950723 ## hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/ozone/rocksdiff/RocksDBCheckpointDiffer.java: ## @@ -114,9 +113,82 @@ public class RocksDBCheckpointD

[GitHub] [ozone] GeorgeJahad commented on pull request #3824: HDDS-7281. [Snapshot] Handle RocksDB compaction DAG persistence and reconstruction

2022-10-12 Thread GitBox
GeorgeJahad commented on PR #3824: URL: https://github.com/apache/ozone/pull/3824#issuecomment-1276799800 This is in response to the discussion here: https://github.com/apache/ozone/pull/3786#discussion_r992935645 To me there is value in the conceptual simplicity of having all the sta

[GitHub] [ozone] GeorgeJahad commented on pull request #3824: HDDS-7281. [Snapshot] Handle RocksDB compaction DAG persistence and reconstruction

2022-10-12 Thread GitBox
GeorgeJahad commented on PR #3824: URL: https://github.com/apache/ozone/pull/3824#issuecomment-1276774483 These lines are confusing to me: https://github.com/apache/ozone/blob/c7e922fb0c4d4c6c312bd717246c9fd8fde27e6f/hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/ozone/rocks

[jira] [Updated] (HDDS-6946) Bump Hadoop to 3.3.4

2022-10-12 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-6946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-6946: --- Fix Version/s: 1.3.0 Resolution: Done Status: Resolved (was: Patch Available

[GitHub] [ozone] adoroszlai commented on pull request #3557: HDDS-6946. Bump Hadoop to 3.3.4

2022-10-12 Thread GitBox
adoroszlai commented on PR #3557: URL: https://github.com/apache/ozone/pull/3557#issuecomment-1276702777 Thanks @ayushtkn, @jojochuang, @smengcl for the review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [ozone] adoroszlai merged pull request #3557: HDDS-6946. Bump Hadoop to 3.3.4

2022-10-12 Thread GitBox
adoroszlai merged PR #3557: URL: https://github.com/apache/ozone/pull/3557 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ozone.apac

[jira] [Assigned] (HDDS-3201) SafeMode exit rule for all SCMs

2022-10-12 Thread Siddharth Wagle (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Wagle reassigned HDDS-3201: - Assignee: Swaminathan Balachandran > SafeMode exit rule for all SCMs > --

[jira] [Created] (HDDS-7318) Publish Ozone 1.2.1 documentation

2022-10-12 Thread Siyao Meng (Jira)
Siyao Meng created HDDS-7318: Summary: Publish Ozone 1.2.1 documentation Key: HDDS-7318 URL: https://issues.apache.org/jira/browse/HDDS-7318 Project: Apache Ozone Issue Type: Task Rep

[jira] [Commented] (HDDS-7208) Erasure coding and encryption are not flagged on FileStatus

2022-10-12 Thread Uma Maheswara Rao G (Jira)
[ https://issues.apache.org/jira/browse/HDDS-7208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616655#comment-17616655 ] Uma Maheswara Rao G commented on HDDS-7208: --- Thanks for the details [~MikaelSmit

[jira] [Resolved] (HDDS-7271) Ozone Integration test shows memory leak (graceful shutdown cleanup)

2022-10-12 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-7271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai resolved HDDS-7271. Fix Version/s: 1.3.0 Resolution: Done > Ozone Integration test shows memory leak (gra

[GitHub] [ozone] adoroszlai merged pull request #3826: HDDS-7271. Ozone Integration test shows memory leak (graceful shutdown cleanup)

2022-10-12 Thread GitBox
adoroszlai merged PR #3826: URL: https://github.com/apache/ozone/pull/3826 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ozone.apac

[GitHub] [ozone] smengcl commented on pull request #3725: HDDS-7178. [Multi-Tenant] Use optimistic read in Ranger background sync

2022-10-12 Thread GitBox
smengcl commented on PR #3725: URL: https://github.com/apache/ozone/pull/3725#issuecomment-1276592065 @errose28 Would you resolve the conflict and address the comments? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[jira] [Commented] (HDDS-7089) Intermittent failure in freon data validation after datanode restart in Kubernetes test

2022-10-12 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-7089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616643#comment-17616643 ] Attila Doroszlai commented on HDDS-7089: The problem is that the test library look

[jira] [Updated] (HDDS-7317) CI: kubernetes fails intermittently in freon data validation: Failed to execute command GetBlock on the pipeline

2022-10-12 Thread Ethan Rose (Jira)
[ https://issues.apache.org/jira/browse/HDDS-7317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Rose updated HDDS-7317: - Parent: HDDS-5626 Issue Type: Sub-task (was: Bug) > CI: kubernetes fails intermittently in freon

[jira] [Resolved] (HDDS-7317) CI: kubernetes fails intermittently in freon data validation: Failed to execute command GetBlock on the pipeline

2022-10-12 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-7317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai resolved HDDS-7317. Resolution: Duplicate > CI: kubernetes fails intermittently in freon data validation: Failed

[jira] [Updated] (HDDS-7317) CI: kubernetes fails intermittently in freon data validation: Failed to execute command GetBlock on the pipeline

2022-10-12 Thread Siyao Meng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-7317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated HDDS-7317: - Description: The following is from [~duongn] 's PR CI. I recall seeing this somewhere else recently (and

[jira] [Updated] (HDDS-7317) CI: kubernetes fails intermittently in freon data validation: Failed to execute command GetBlock on the pipeline

2022-10-12 Thread Siyao Meng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-7317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated HDDS-7317: - Summary: CI: kubernetes fails intermittently in freon data validation: Failed to execute command GetBlock

[GitHub] [ozone] sumitagrawl commented on pull request #3826: HDDS-7271. Ozone Integration test shows memory leak (graceful shutdown cleanup)

2022-10-12 Thread GitBox
sumitagrawl commented on PR #3826: URL: https://github.com/apache/ozone/pull/3826#issuecomment-1276570208 @ChenSammi Please merge -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

[jira] [Updated] (HDDS-7317) CI: kubernetes is failing in freon data validation intermittently

2022-10-12 Thread Siyao Meng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-7317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated HDDS-7317: - Description: {code:title=https://github.com/apache/ozone/actions/runs/3230990324/jobs/5290112904#step:5:83

[jira] [Updated] (HDDS-7317) CI: kubernetes fails intermittently in freon data validation due to Failed to execute command GetBlock on the pipeline

2022-10-12 Thread Siyao Meng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-7317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated HDDS-7317: - Summary: CI: kubernetes fails intermittently in freon data validation due to Failed to execute command Get

[jira] [Created] (HDDS-7317) CI: kubernetes is failing in freon data validation intermittently

2022-10-12 Thread Siyao Meng (Jira)
Siyao Meng created HDDS-7317: Summary: CI: kubernetes is failing in freon data validation intermittently Key: HDDS-7317 URL: https://issues.apache.org/jira/browse/HDDS-7317 Project: Apache Ozone

[GitHub] [ozone] adoroszlai commented on pull request #3748: HDDS-7213. extract ReplicationManagerConfiguration to a separate class

2022-10-12 Thread GitBox
adoroszlai commented on PR #3748: URL: https://github.com/apache/ozone/pull/3748#issuecomment-1276547892 @myskov are you OK with closing this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [ozone] adoroszlai commented on a diff in pull request #3827: HDDS-7316. Print stacktrace to identify the location of RocksObject leaks.

2022-10-12 Thread GitBox
adoroszlai commented on code in PR #3827: URL: https://github.com/apache/ozone/pull/3827#discussion_r993756288 ## hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/managed/ManagedRocksObjectUtils.java: ## @@ -41,6 +41,20 @@ static void assertClosed(RocksObject

[GitHub] [ozone] duongkame commented on a diff in pull request #3800: HDDS-7231. Integrate the GetKeyInfo API to key read flows

2022-10-12 Thread GitBox
duongkame commented on code in PR #3800: URL: https://github.com/apache/ozone/pull/3800#discussion_r993755980 ## hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockInputStream.java: ## @@ -302,20 +310,24 @@ protected synchronized int readWithStrategy(ByteR

[jira] [Updated] (HDDS-7261) Add container location cache metrics.

2022-10-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-7261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-7261: - Labels: pull-request-available (was: ) > Add container location cache metrics. >

[GitHub] [ozone] duongkame opened a new pull request, #3829: HDDS-7261. Add container location cache metrics.

2022-10-12 Thread GitBox
duongkame opened a new pull request, #3829: URL: https://github.com/apache/ozone/pull/3829 ## What changes were proposed in this pull request? Add container location cache metrics. ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-7261

[jira] [Updated] (HDDS-7261) Add container location cache metrics.

2022-10-12 Thread Duong (Jira)
[ https://issues.apache.org/jira/browse/HDDS-7261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duong updated HDDS-7261: Summary: Add container location cache metrics. (was: Add cache related metrics) > Add container location cache met

[GitHub] [ozone] errose28 commented on a diff in pull request #3788: HDDS-7095. allow on demand scanning for containers

2022-10-12 Thread GitBox
errose28 commented on code in PR #3788: URL: https://github.com/apache/ozone/pull/3788#discussion_r993639556 ## hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/ozoneimpl/TestOnDemandContainerScanner.java: ## @@ -0,0 +1,147 @@ +/* + * Licensed to the

[jira] [Updated] (HDDS-7270) Fix bug in checking healthy replica placement policy check in EC underReplication handler

2022-10-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-7270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-7270: - Labels: pull-request-available (was: ) > Fix bug in checking healthy replica placement policy che

[GitHub] [ozone] swamirishi opened a new pull request, #3828: HDDS-7270. Fix bug in checking healthy replica placement policy check in EC underReplication handler

2022-10-12 Thread GitBox
swamirishi opened a new pull request, #3828: URL: https://github.com/apache/ozone/pull/3828 ## What changes were proposed in this pull request? Fixing bug on existing healthy node selection in ECUnderReplicationHandler. ## What is the link to the Apache JIRA https://issues.apac

[jira] [Commented] (HDDS-7312) CI: acceptance (MR) fails due to Unsupported transport protocol

2022-10-12 Thread Siyao Meng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-7312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616590#comment-17616590 ] Siyao Meng commented on HDDS-7312: -- Sounds good. Thanks [~adoroszlai] > CI: acceptance (

[GitHub] [ozone] sodonnel commented on a diff in pull request #3788: HDDS-7095. allow on demand scanning for containers

2022-10-12 Thread GitBox
sodonnel commented on code in PR #3788: URL: https://github.com/apache/ozone/pull/3788#discussion_r993672113 ## hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/ozoneimpl/OnDemandContainerScanner.java: ## @@ -0,0 +1,164 @@ +/* + * Licensed to the Apa

[jira] [Updated] (HDDS-7095) Allow containers to be prioritized for health scanning

2022-10-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-7095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-7095: - Labels: pull-request-available (was: ) > Allow containers to be prioritized for health scanning >

[GitHub] [ozone] sodonnel commented on a diff in pull request #3788: HDDS-7095. allow on demand scanning for containers

2022-10-12 Thread GitBox
sodonnel commented on code in PR #3788: URL: https://github.com/apache/ozone/pull/3788#discussion_r993664378 ## hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/ozoneimpl/OnDemandContainerScanner.java: ## @@ -0,0 +1,164 @@ +/* + * Licensed to the Apa

[GitHub] [ozone] sodonnel commented on pull request #3806: HDDS-7291. Fixing exception handling in case of non positive replica index

2022-10-12 Thread GitBox
sodonnel commented on PR #3806: URL: https://github.com/apache/ozone/pull/3806#issuecomment-1276407540 Also I see a failure on ` TestOzoneHARatisLogParser.testRatisLogParsing` I believe this is not related to this change. The problem causing it has been reverted, so you may need to rebase y

[GitHub] [ozone] michelsumbul commented on pull request #3820: HDDS-7309. Enable by default GRPC between S3G and OM

2022-10-12 Thread GitBox
michelsumbul commented on PR #3820: URL: https://github.com/apache/ozone/pull/3820#issuecomment-1276405186 @kerneltime As we said at the apacheCon, we will look at having this number when we have a bit more time, however this should not be a prerequisite to enable that feature by default. W

[GitHub] [ozone] adoroszlai commented on pull request #3826: HDDS-7271. Ozone Integration test shows memory leak (graceful shutdown cleanup)

2022-10-12 Thread GitBox
adoroszlai commented on PR #3826: URL: https://github.com/apache/ozone/pull/3826#issuecomment-1276374377 With previous commit repeated test run in CI shows 60% failure rate. https://github.com/adoroszlai/hadoop-ozone/actions/runs/3235026012/jobs/5298892266#step:6:12 I'll check

[GitHub] [ozone] sodonnel commented on a diff in pull request #3806: HDDS-7291. Fixing exception handling in case of non positive replica index

2022-10-12 Thread GitBox
sodonnel commented on code in PR #3806: URL: https://github.com/apache/ozone/pull/3806#discussion_r993617132 ## hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerReportHandler.java: ## @@ -187,7 +188,11 @@ public void onMessage(final ContainerRep

[GitHub] [ozone] sumitagrawl commented on pull request #3826: HDDS-7271. Ozone Integration test shows memory leak (graceful shutdown cleanup)

2022-10-12 Thread GitBox
sumitagrawl commented on PR #3826: URL: https://github.com/apache/ozone/pull/3826#issuecomment-1276362571 > Thanks @sumitagrawl for checking. I have never seen this problem before (we collect [test results](https://github.com/adoroszlai/ozone-build-results) from `master` branch), so until w

[jira] [Updated] (HDDS-7316) Print stacktrace to identify the location of RocksObject leaks.

2022-10-12 Thread Sadanand Shenoy (Jira)
[ https://issues.apache.org/jira/browse/HDDS-7316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sadanand Shenoy updated HDDS-7316: -- Description: HDDS-7087 adds a tracker to detect if the Rocksdb objects are not closed properly

[jira] [Updated] (HDDS-7316) Print stacktrace to identify the location of RocksObject leaks.

2022-10-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-7316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-7316: - Labels: pull-request-available (was: ) > Print stacktrace to identify the location of RocksObject

[GitHub] [ozone] sadanand48 opened a new pull request, #3827: HDDS-7316. Print stacktrace to identify the location of RocksObject leaks.

2022-10-12 Thread GitBox
sadanand48 opened a new pull request, #3827: URL: https://github.com/apache/ozone/pull/3827 ## What changes were proposed in this pull request? Add logging to identify location/trace of unclosed RocksObjects . ## What is the link to the Apache JIRA https://issues.apache.org/jira/

[jira] [Created] (HDDS-7316) Print stacktrace to identify the location of RocksObject leaks.

2022-10-12 Thread Sadanand Shenoy (Jira)
Sadanand Shenoy created HDDS-7316: - Summary: Print stacktrace to identify the location of RocksObject leaks. Key: HDDS-7316 URL: https://issues.apache.org/jira/browse/HDDS-7316 Project: Apache Ozone

[GitHub] [ozone] sumitagrawl commented on pull request #3801: HDDS-7284. JVM crash for rocksdb for read/write after close

2022-10-12 Thread GitBox
sumitagrawl commented on PR #3801: URL: https://github.com/apache/ozone/pull/3801#issuecomment-1276179704 > > @sumitagrawl , thanks for reporting and fixing this issue. Since JVM is crashed for there is still read/write after close, which means there is some issue with current Recon code, t

[GitHub] [ozone] adoroszlai commented on a diff in pull request #3811: HDDS-7300. Race condition between full data scan and block deletion

2022-10-12 Thread GitBox
adoroszlai commented on code in PR #3811: URL: https://github.com/apache/ozone/pull/3811#discussion_r993465720 ## hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainerCheck.java: ## @@ -230,44 +232,98 @@ private void scanData(Da

[GitHub] [ozone] adoroszlai commented on pull request #3826: HDDS-7271. Ozone Integration test shows memory leak (graceful shutdown cleanup)

2022-10-12 Thread GitBox
adoroszlai commented on PR #3826: URL: https://github.com/apache/ozone/pull/3826#issuecomment-1276170454 Thanks @sumitagrawl for checking. I have never seen this problem before (we collect [test results](https://github.com/adoroszlai/ozone-build-results) from `master` branch), so until we

[GitHub] [ozone] sumitagrawl commented on pull request #3826: HDDS-7271. Ozone Integration test shows memory leak (graceful shutdown cleanup)

2022-10-12 Thread GitBox
sumitagrawl commented on PR #3826: URL: https://github.com/apache/ozone/pull/3826#issuecomment-1276152886 > Thanks @sumitagrawl for the new PR. Can you please check the following failure? > > ``` > Error: org.apache.hadoop.ozone.client.rpc.TestCommitWatcher.testReleaseBuffersOnEx

[GitHub] [ozone] adoroszlai commented on pull request #3815: HDDS-7269. Remove fair lock configuration for container state management flows.

2022-10-12 Thread GitBox
adoroszlai commented on PR #3815: URL: https://github.com/apache/ozone/pull/3815#issuecomment-1276149684 @nandakumar131 @sumitagrawl I think this implements the [suggestion](https://github.com/apache/ozone/pull/3766#discussion_r981309325) from review of HDDS-7232, please take a look if you

[jira] [Resolved] (HDDS-7312) CI: acceptance (MR) fails due to Unsupported transport protocol

2022-10-12 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-7312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai resolved HDDS-7312. Resolution: Works for Me Thanks [~smeng] for reporting this. I think it was a temporary net

[GitHub] [ozone] kaijchen commented on pull request #3825: HDDS-7313. Update github actions for Node16

2022-10-12 Thread GitBox
kaijchen commented on PR #3825: URL: https://github.com/apache/ozone/pull/3825#issuecomment-1276024117 Thanks @adoroszlai for the review and merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Updated] (HDDS-7305) Fix Hadoop imports

2022-10-12 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-7305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-7305: --- Status: Patch Available (was: In Progress) > Fix Hadoop imports > -- > >

[jira] [Updated] (HDDS-7313) Update github actions for Node16

2022-10-12 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-7313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-7313: --- Issue Type: Task (was: Improvement) > Update github actions for Node16 >

[jira] [Resolved] (HDDS-7313) Update github actions for Node16

2022-10-12 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-7313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai resolved HDDS-7313. Fix Version/s: 1.3.0 Resolution: Done > Update github actions for Node16 > --

[GitHub] [ozone] adoroszlai merged pull request #3825: HDDS-7313. Update github actions for Node16

2022-10-12 Thread GitBox
adoroszlai merged PR #3825: URL: https://github.com/apache/ozone/pull/3825 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ozone.apac

[GitHub] [ozone] sadanand48 commented on a diff in pull request #3824: HDDS-7281. [Snapshot] Handle RocksDB compaction DAG persistence and reconstruction

2022-10-12 Thread GitBox
sadanand48 commented on code in PR #3824: URL: https://github.com/apache/ozone/pull/3824#discussion_r993317466 ## hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/ozone/rocksdiff/RocksDBCheckpointDiffer.java: ## @@ -471,13 +529,151 @@ public HashSet readRocksDBLive

  1   2   >