Re: [PR] [DO NOT MERGE] Backport some fixes, performance optimizations from master to ozone-1.4 #6929 [ozone]

2024-07-22 Thread via GitHub
rohit-kb commented on PR #6964: URL: https://github.com/apache/ozone/pull/6964#issuecomment-2244322726 Thanks for the backport. Mine are good too -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] [DO NOT MERGE] Backport some fixes, performance optimizations from master to ozone-1.4 #6929 [ozone]

2024-07-22 Thread via GitHub
hemantk-12 commented on PR #6964: URL: https://github.com/apache/ozone/pull/6964#issuecomment-2244310569 Thanks @xichen01 for the patch. Backport of my changes look good. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] HDDS-11068. Move SstFiltered flag to a file in the snapshot directory [ozone]

2024-07-22 Thread via GitHub
hemantk-12 commented on code in PR #6965: URL: https://github.com/apache/ozone/pull/6965#discussion_r1687224493 ## hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmSnapshotManager.java: ## @@ -441,6 +441,13 @@ public void invalidateCacheEntry(UUID key) {

Re: [PR] HDDS-10917. Refactoring more tests from TestContainerBalancerTask [ozone]

2024-07-22 Thread via GitHub
Tejaskriya commented on code in PR #6734: URL: https://github.com/apache/ozone/pull/6734#discussion_r1687425398 ## hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/balancer/TestContainerBalancerDatanodeNodeLimit.java: ## @@ -323,6 +333,259 @@ public void

[jira] [Created] (HDDS-11220) [HBase Replication] RS down due to "ManagedChannelOrphanWrapper: Previous channel was not shutdown properly"

2024-07-22 Thread Pratyush Bhatt (Jira)
Pratyush Bhatt created HDDS-11220: - Summary: [HBase Replication] RS down due to "ManagedChannelOrphanWrapper: Previous channel was not shutdown properly" Key: HDDS-11220 URL: https://issues.apache.org/jira/browse/

[jira] [Created] (HDDS-11219) [HBase Replication] RS and Master Nodes down with "Waiting for one of pipelines to be OPEN failed"

2024-07-22 Thread Pratyush Bhatt (Jira)
Pratyush Bhatt created HDDS-11219: - Summary: [HBase Replication] RS and Master Nodes down with "Waiting for one of pipelines to be OPEN failed" Key: HDDS-11219 URL: https://issues.apache.org/jira/browse/HDDS-11219

Re: [PR] HDDS-10055. [hsync] Optimize FilePerBlockStrategy.writeChunk(). [ozone]

2024-07-22 Thread via GitHub
jojochuang commented on PR #6923: URL: https://github.com/apache/ozone/pull/6923#issuecomment-2243964899 @swamirishi I'd like to have your thoughts on this one. After this PR, the failure recovery behavior changed and I had to make a change in the test TestContainerReplication#testECConta

[PR] HDDS-11216. Replace HAUtils#buildCAX509List usages with other direct usages [ozone]

2024-07-22 Thread via GitHub
Galsza opened a new pull request, #6981: URL: https://github.com/apache/ozone/pull/6981 ## To simplify the CertificateClient the HAUtils#buildCAX509List can be refactored There are 3 methods on DefaultCertificateClient that solely exist to serve the HAUtils method, preferably these s

[jira] [Updated] (HDDS-11216) Replace HAUtils#buildCAX509List usages with other direct usages

2024-07-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-11216: -- Labels: pull-request-available (was: ) > Replace HAUtils#buildCAX509List usages with other dir

Re: [PR] HDDS-10376. Add a Datanode API to supply a merkle tree for a given container. [ozone]

2024-07-22 Thread via GitHub
errose28 commented on code in PR #6945: URL: https://github.com/apache/ozone/pull/6945#discussion_r1687006365 ## hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/checksum/ContainerChecksumTreeManager.java: ## @@ -157,6 +159,28 @@ private void write(K

Re: [PR] HDDS-11193. KeyOutputStream flakiness when running write and hsync concurrently [ozone]

2024-07-22 Thread via GitHub
duongkame commented on code in PR #6968: URL: https://github.com/apache/ozone/pull/6968#discussion_r1686953813 ## hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java: ## @@ -682,7 +700,7 @@ private synchronized CompletableFuture handleFlus

[jira] [Created] (HDDS-11218) Combine datanode clients for reconciliation and EC reconstruction

2024-07-22 Thread Ethan Rose (Jira)
Ethan Rose created HDDS-11218: - Summary: Combine datanode clients for reconciliation and EC reconstruction Key: HDDS-11218 URL: https://issues.apache.org/jira/browse/HDDS-11218 Project: Apache Ozone

Re: [PR] HDDS-11193. KeyOutputStream flakiness when running write and hsync concurrently [ozone]

2024-07-22 Thread via GitHub
duongkame commented on code in PR #6968: URL: https://github.com/apache/ozone/pull/6968#discussion_r1686936091 ## hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java: ## @@ -150,6 +153,7 @@ public class BlockOutputStream extends OutputStrea

Re: [PR] HDDS-11136. Some containers affected by HDDS-8129 may still be in the DELETING state incorrectly [ozone]

2024-07-22 Thread via GitHub
siddhantsangwan commented on PR #6967: URL: https://github.com/apache/ozone/pull/6967#issuecomment-2243512098 > How a container that was incorrectly in the DELETING state and moved back to CLOSED will eventually get cleared from the system. @errose28 can you elaborate on this please?

[jira] [Resolved] (HDDS-9237) Enable lock order fairness for striped locks.

2024-07-22 Thread Nandakumar (Jira)
[ https://issues.apache.org/jira/browse/HDDS-9237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandakumar resolved HDDS-9237. -- Resolution: Fixed The PR is merged! > Enable lock order fairness for striped locks. > -

[jira] [Updated] (HDDS-11217) Use zero-copy for readMerkleTree API

2024-07-22 Thread Ethan Rose (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Rose updated HDDS-11217: -- Description: In HDDS-10376, the API to read a container's merkle tree was implemented without zero-copy

[jira] [Created] (HDDS-11217) Use zero-copy for readMerkleTree API

2024-07-22 Thread Ethan Rose (Jira)
Ethan Rose created HDDS-11217: - Summary: Use zero-copy for readMerkleTree API Key: HDDS-11217 URL: https://issues.apache.org/jira/browse/HDDS-11217 Project: Apache Ozone Issue Type: Sub-task

Re: [PR] HDDS-11136. Some containers affected by HDDS-8129 may still be in the DELETING state incorrectly [ozone]

2024-07-22 Thread via GitHub
nandakumar131 commented on code in PR #6967: URL: https://github.com/apache/ozone/pull/6967#discussion_r1686915966 ## hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/AbstractContainerReportHandler.java: ## @@ -349,8 +349,13 @@ private boolean updateCont

[PR] Bump dnsjava:dnsjava from 2.1.9 to 3.6.0 [ozone]

2024-07-22 Thread via GitHub
dependabot[bot] opened a new pull request, #6979: URL: https://github.com/apache/ozone/pull/6979 Bumps [dnsjava:dnsjava](https://github.com/dnsjava/dnsjava) from 2.1.9 to 3.6.0. Release notes Sourced from https://github.com/dnsjava/dnsjava/releases";>dnsjava:dnsjava's releases.

[jira] [Updated] (HDDS-11215) Quota count can go wrong when double buffer flush takes time

2024-07-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-11215: -- Labels: pull-request-available (was: ) > Quota count can go wrong when double buffer flush tak

[PR] HDDS-11215. Quota count can go wrong when double buffer flush takes time [ozone]

2024-07-22 Thread via GitHub
sumitagrawl opened a new pull request, #6978: URL: https://github.com/apache/ozone/pull/6978 ## What changes were proposed in this pull request? bucketInfo can have incorrect count, and even can go -ve value. This is as DirectoryDeletingService keeps sending purgeDirectory request til

Re: [PR] HDDS-11140. Recon Disk Usage Metadata Details are not working for du api [ozone]

2024-07-22 Thread via GitHub
smitajoshi12 commented on PR #6966: URL: https://github.com/apache/ozone/pull/6966#issuecomment-2242991614 > @smitajoshi12, thank you for working on this. > > Could you please show us the error message that appeared earlier and how it is displayed in the UI after the fix? @Araf

Re: [PR] HDDS-11023. Recon Disk Usage null conditions not handled properly for null response [ozone]

2024-07-22 Thread via GitHub
smitajoshi12 commented on code in PR #6894: URL: https://github.com/apache/ozone/pull/6894#discussion_r1686556567 ## hadoop-ozone/recon/src/main/resources/webapps/recon/ozone-recon-web/src/views/diskUsage/diskUsage.tsx: ## @@ -314,165 +314,175 @@ export class DiskUsage extends

[jira] [Created] (HDDS-11216) Replace HAUtils#buildCAX509List usages with other direct usages

2024-07-22 Thread Jira
Szabolcs Gál created HDDS-11216: --- Summary: Replace HAUtils#buildCAX509List usages with other direct usages Key: HDDS-11216 URL: https://issues.apache.org/jira/browse/HDDS-11216 Project: Apache Ozone

Re: [PR] HDDS-11023. Recon Disk Usage null conditions not handled properly for null response [ozone]

2024-07-22 Thread via GitHub
smitajoshi12 commented on code in PR #6894: URL: https://github.com/apache/ozone/pull/6894#discussion_r1686554676 ## hadoop-ozone/recon/src/main/resources/webapps/recon/ozone-recon-web/src/views/diskUsage/diskUsage.tsx: ## @@ -37,7 +37,7 @@ import { import { DetailPanel } fro

Re: [PR] HDDS-11023. Recon Disk Usage null conditions not handled properly for null response [ozone]

2024-07-22 Thread via GitHub
smitajoshi12 commented on code in PR #6894: URL: https://github.com/apache/ozone/pull/6894#discussion_r1686553381 ## hadoop-ozone/recon/src/main/resources/webapps/recon/ozone-recon-web/src/utils/common.tsx: ## @@ -33,11 +33,26 @@ const showErrorNotification = (title: string, des

Re: [PR] HDDS-11206. Statistics storage usage indicators include min, max, median, stdev [ozone]

2024-07-22 Thread via GitHub
jianghuazhu commented on PR #6977: URL: https://github.com/apache/ozone/pull/6977#issuecomment-224293 Can you help review this PR, @dombizita @xBis7. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[jira] [Updated] (HDDS-11206) Statistics storage usage indicators include min, max, median, stdev

2024-07-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-11206: -- Labels: pull-request-available (was: ) > Statistics storage usage indicators include min, max,

[PR] HDDS-11206. Statistics storage usage indicators include min, max, median, stdev [ozone]

2024-07-22 Thread via GitHub
jianghuazhu opened a new pull request, #6977: URL: https://github.com/apache/ozone/pull/6977 ## What changes were proposed in this pull request? For SCM, it is very useful to count the min, max, median, and stdev of node storage. After adding the patch: jmx: ui: ##

[jira] [Resolved] (HDDS-11120) Rich rebalancing status info

2024-07-22 Thread Ivan Andika (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Andika resolved HDDS-11120. Fix Version/s: 1.5.0 Resolution: Implemented > Rich rebalancing status info > --

Re: [PR] HDDS-11120. Rich rebalancing status info [ozone]

2024-07-22 Thread via GitHub
ivandika3 commented on PR #6911: URL: https://github.com/apache/ozone/pull/6911#issuecomment-2242659299 Thanks @juncevich for the patch, @adoroszlai for the review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

[jira] [Updated] (HDDS-11120) Rich rebalancing status info

2024-07-22 Thread Ivan Andika (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Andika updated HDDS-11120: --- Component/s: Balancer > Rich rebalancing status info > > >

Re: [PR] HDDS-11120. Rich rebalancing status info [ozone]

2024-07-22 Thread via GitHub
ivandika3 merged PR #6911: URL: https://github.com/apache/ozone/pull/6911 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ozone.apach

[jira] [Updated] (HDDS-11215) Quota count can go wrong when double buffer flush takes time

2024-07-22 Thread Sumit Agrawal (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Agrawal updated HDDS-11215: - Description: Quota count can go wrong in below scenario, # DirectoryDeletingService submitted ta

[jira] [Updated] (HDDS-11215) Quota count can go wrong when double buffer flush takes time

2024-07-22 Thread Sumit Agrawal (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Agrawal updated HDDS-11215: - Priority: Critical (was: Major) > Quota count can go wrong when double buffer flush takes time >

[jira] [Created] (HDDS-11215) Quota count can go wrong when double buffer flush takes time

2024-07-22 Thread Sumit Agrawal (Jira)
Sumit Agrawal created HDDS-11215: Summary: Quota count can go wrong when double buffer flush takes time Key: HDDS-11215 URL: https://issues.apache.org/jira/browse/HDDS-11215 Project: Apache Ozone

Re: [PR] HDDS-11188. Initial setup for new UI layout and enable users to switch to new UI [ozone]

2024-07-22 Thread via GitHub
devabhishekpal commented on PR #6953: URL: https://github.com/apache/ozone/pull/6953#issuecomment-2242417303 Hi @adoroszlai , thanks for pointing out the build issues. I have fixed those, removed the files which are currently not required for this PR, and tested out the patch locally.

[jira] [Resolved] (HDDS-11187) Fix Event Handling Corruption in OMDBUpdatesHandler to Prevent ClassCastException in Recon Server

2024-07-22 Thread Sadanand Shenoy (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sadanand Shenoy resolved HDDS-11187. Resolution: Fixed > Fix Event Handling Corruption in OMDBUpdatesHandler to Prevent > Class

Re: [PR] HDDS-11187. Fix Event Handling in Recon OMDBUpdatesHandler to Prevent ClassCastException [ozone]

2024-07-22 Thread via GitHub
sadanand48 merged PR #6950: URL: https://github.com/apache/ozone/pull/6950 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ozone.apac

Re: [PR] HDDS-11187. Fix Event Handling in Recon OMDBUpdatesHandler to Prevent ClassCastException [ozone]

2024-07-22 Thread via GitHub
sadanand48 commented on PR #6950: URL: https://github.com/apache/ozone/pull/6950#issuecomment-2242368071 Thanks @ArafatKhan2198 for the detailed explanation. I tried to reproduce the class cast exception and found another place where we could add this check for OMKeyInfo. I wrote this

[jira] [Created] (HDDS-11214) LOG files for RocksDB ops should be cleaned more frequently for heavy load

2024-07-22 Thread Jyotirmoy Sinha (Jira)
Jyotirmoy Sinha created HDDS-11214: -- Summary: LOG files for RocksDB ops should be cleaned more frequently for heavy load Key: HDDS-11214 URL: https://issues.apache.org/jira/browse/HDDS-11214 Project:

Re: [PR] HDDS-11136. Some containers affected by HDDS-8129 may still be in the DELETING state incorrectly [ozone]

2024-07-22 Thread via GitHub
siddhantsangwan commented on code in PR #6967: URL: https://github.com/apache/ozone/pull/6967#discussion_r1686114996 ## hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/TestContainerReportHandler.java: ## @@ -100,7 +101,6 @@ void setup() throws IOExcepti