Re: [PR] HDDS-13612. Track SST file ranges in sst file metadata [ozone]

2025-09-06 Thread via GitHub
swamirishi commented on PR #8981: URL: https://github.com/apache/ozone/pull/8981#issuecomment-3261943045 @smengcl @SaketaChalamchala @jojochuang Can you take a look at this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[PR] Bump software.amazon.awssdk:bom from 2.33.0 to 2.33.4 [ozone]

2025-09-06 Thread via GitHub
dependabot[bot] opened a new pull request, #9008: URL: https://github.com/apache/ozone/pull/9008 Bumps software.amazon.awssdk:bom from 2.33.0 to 2.33.4. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=software.a

Re: [PR] HDDS-7544. ContainerStateMap uses HashSet to store container replica lists [ozone]

2025-09-05 Thread via GitHub
cchung100m commented on PR #8990: URL: https://github.com/apache/ozone/pull/8990#issuecomment-3261192015 Hi @jojochuang My username is cchung100m and I also leave a comment at jira ticket [HDDS-7544](https://issues.apache.org/jira/browse/HDDS-7544). > @cchung100m do you have a

Re: [PR] HDDS-12998. Bring real container size in pb message when exporting/importing containers [ozone]

2025-09-05 Thread via GitHub
peterxcli commented on code in PR #8915: URL: https://github.com/apache/ozone/pull/8915#discussion_r2326399824 ## hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/ECMisReplicationHandler.java: ## @@ -83,6 +83,7 @@ protected int sendReplicateC

Re: [PR] [Don't merge] Merge HDDS-5713 disk balancer feature branch to master [ozone]

2025-09-05 Thread via GitHub
Gargi-jais11 commented on code in PR #8932: URL: https://github.com/apache/ozone/pull/8932#discussion_r2321461987 ## hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/diskbalancer/policy/DefaultVolumeChoosingPolicy.java: ## @@ -0,0 +1,110 @@ +/* + * L

Re: [PR] HDDS-13005. MinHeap based SstFileSetReader to return unique results in sorted order [ozone]

2025-09-05 Thread via GitHub
Copilot commented on code in PR #8912: URL: https://github.com/apache/ozone/pull/8912#discussion_r2325882538 ## hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/ozone/rocksdb/util/SstFileSetReader.java: ## @@ -233,70 +236,149 @@ public String next() { } }

Re: [PR] HDDS-7566. Refactor TestRocksDBCheckpointDiffer tests. [ozone]

2025-09-05 Thread via GitHub
SaketaChalamchala commented on PR #8785: URL: https://github.com/apache/ozone/pull/8785#issuecomment-3259944003 Thanks for the patch @jojochuang . We can also move `testPruneOlderSnapshotsWithCompactionHistory` to the `TestCompactionDag` class as it tests removing nodes (SST files) from the

Re: [PR] HDDS-13005. MinHeap based SstFileSetReader to return unique results in sorted order [ozone]

2025-09-05 Thread via GitHub
jojochuang commented on code in PR #8912: URL: https://github.com/apache/ozone/pull/8912#discussion_r2326024846 ## hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/ozone/rocksdb/util/SstFileSetReader.java: ## @@ -233,70 +236,149 @@ public String next() { }

[PR] HDDS-13652. Enable Atomic Rewrite Key feature by default for Ozone Manager. [ozone]

2025-09-05 Thread via GitHub
devmadhuu opened a new pull request, #9006: URL: https://github.com/apache/ozone/pull/9006 ## What changes were proposed in this pull request? This PR change is just to enable the Atomic Rewrite Key feature by default for Ozone Manager. ## What is the link to the Apache JIRA htt

Re: [PR] HDDS-13652. Enable Atomic Rewrite Key feature by default for Ozone Manager. [ozone]

2025-09-05 Thread via GitHub
devmadhuu commented on PR #9006: URL: https://github.com/apache/ozone/pull/9006#issuecomment-3257833712 No upstream change is needed cc: @sodonnel -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] HDDS-13576. Recon reprocess of all tasks should be non-blocking. [ozone]

2025-09-05 Thread via GitHub
sumitagrawl merged PR #8962: URL: https://github.com/apache/ozone/pull/8962 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ozone.apa

Re: [PR] HDDS-13649. [DiskBalancer] Refine Disk Balancer CLI, Documentation, and Naming for Improved Usability [ozone]

2025-09-05 Thread via GitHub
ChenSammi commented on code in PR #9005: URL: https://github.com/apache/ozone/pull/9005#discussion_r2324335479 ## hadoop-hdds/docs/content/design/diskbalancer.md: ## @@ -129,6 +129,11 @@ The feature can be enabled by setting the following property to `true` in the `o Developer

Re: [PR] [Don't merge] Merge HDDS-5713 disk balancer feature branch to master [ozone]

2025-09-05 Thread via GitHub
Gargi-jais11 commented on code in PR #8932: URL: https://github.com/apache/ozone/pull/8932#discussion_r2324334653 ## hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/storage/DiskBalancerConfiguration.java: ## @@ -0,0 +1,296 @@ +/* + * Licensed to the Apache Software F

Re: [PR] HDDS-13649. [DiskBalancer] Refine Disk Balancer CLI, Documentation, and Naming for Improved Usability [ozone]

2025-09-05 Thread via GitHub
ChenSammi commented on code in PR #9005: URL: https://github.com/apache/ozone/pull/9005#discussion_r2324341001 ## hadoop-hdds/docs/content/feature/DiskBalancer.md: ## @@ -41,9 +41,22 @@ A disk is considered a candidate for balancing if its ![Data spread across disks](diskBala

Re: [PR] HDDS-13649. [DiskBalancer] Refine Disk Balancer CLI, Documentation, and Naming for Improved Usability [ozone]

2025-09-05 Thread via GitHub
Gargi-jais11 commented on code in PR #9005: URL: https://github.com/apache/ozone/pull/9005#discussion_r2324323911 ## hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/storage/DiskBalancerConfiguration.java: ## @@ -53,7 +53,7 @@ public final class DiskBalancerConfigurat

Re: [PR] HDDS-13649. [DiskBalancer] Refine Disk Balancer CLI, Documentation, and Naming for Improved Usability [ozone]

2025-09-05 Thread via GitHub
ChenSammi commented on code in PR #9005: URL: https://github.com/apache/ozone/pull/9005#discussion_r2324339725 ## hadoop-hdds/docs/content/feature/DiskBalancer.md: ## @@ -41,9 +41,22 @@ A disk is considered a candidate for balancing if its ![Data spread across disks](diskBala

Re: [PR] HDDS-13649. [DiskBalancer] Refine Disk Balancer CLI, Documentation, and Naming for Improved Usability [ozone]

2025-09-05 Thread via GitHub
ChenSammi commented on code in PR #9005: URL: https://github.com/apache/ozone/pull/9005#discussion_r2324335479 ## hadoop-hdds/docs/content/design/diskbalancer.md: ## @@ -129,6 +129,11 @@ The feature can be enabled by setting the following property to `true` in the `o Developer

Re: [PR] HDDS-13649. [DiskBalancer] Refine Disk Balancer CLI, Documentation, and Naming for Improved Usability [ozone]

2025-09-05 Thread via GitHub
ChenSammi commented on code in PR #9005: URL: https://github.com/apache/ozone/pull/9005#discussion_r2324316201 ## hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/storage/DiskBalancerConfiguration.java: ## @@ -53,7 +53,7 @@ public final class DiskBalancerConfiguration

[PR] HDDS-13649. [DiskBalancer] Refine Disk Balancer CLI, Documentation, and Naming for Improved Usability [ozone]

2025-09-05 Thread via GitHub
Gargi-jais11 opened a new pull request, #9005: URL: https://github.com/apache/ozone/pull/9005 ## What changes were proposed in this pull request? 1. naming of variable DatanodeDiskBalancerInfoRequest should start with small letter, same for DatanodeDiskBalancerInfoResponse 2. U

Re: [PR] HDDS-7544. ContainerStateMap uses HashSet to store container replica lists [ozone]

2025-09-04 Thread via GitHub
jojochuang commented on PR #8990: URL: https://github.com/apache/ozone/pull/8990#issuecomment-3257117187 @cchung100m do you have a jira account? I'd like to assigne the jira to you. thx -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

Re: [PR] HDDS-13414. Build a basic deb package for Ozone [ozone]

2025-09-04 Thread via GitHub
jojochuang commented on PR #: URL: https://github.com/apache/ozone/pull/#issuecomment-3257095929 Merged. Thanks @echonesis @chungen0126 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] HDDS-12379. Fix spotbugs warnings in TestRadixTree [ozone]

2025-09-04 Thread via GitHub
jojochuang merged PR #8968: URL: https://github.com/apache/ozone/pull/8968 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ozone.apac

Re: [PR] HDDS-13414. Build a basic deb package for Ozone [ozone]

2025-09-04 Thread via GitHub
jojochuang merged PR #: URL: https://github.com/apache/ozone/pull/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ozone.apac

[PR] HDDS-13623. Recon UI: Generalize themeIcon to support multiple replication types [ozone]

2025-09-04 Thread via GitHub
laiyousin opened a new pull request, #9003: URL: https://github.com/apache/ozone/pull/9003 ## What changes were proposed in this pull request? This PR enhances Recon’s replication “theme icon” system and styles so we can properly visualize EC buckets (alongside existing RATIS/Standalone),

Re: [PR] HDDS-13604. Add volume and bucket to checkpoint existence/creation logs. [ozone]

2025-09-04 Thread via GitHub
jojochuang merged PR #8976: URL: https://github.com/apache/ozone/pull/8976 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ozone.apac

[PR] HDDS-13641. [Docs] Include configurations and metrics related to DataNode decommission [ozone]

2025-09-04 Thread via GitHub
jojochuang opened a new pull request, #9004: URL: https://github.com/apache/ozone/pull/9004 ## What changes were proposed in this pull request? HDDS-13641. [Docs] Include configurations and metrics related to DataNode decommission Please describe your PR in detail: * Similar to

Re: [PR] HDDS-13612. Track SST file ranges in sst file metadata [ozone]

2025-09-04 Thread via GitHub
swamirishi commented on code in PR #8981: URL: https://github.com/apache/ozone/pull/8981#discussion_r2323772740 ## hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmSnapshotLocalData.java: ## @@ -45,32 +51,33 @@ public abstract class OmSnapshotLocalData {

Re: [PR] HDDS-13633. Snapshot Cache loader should check if the double buffer has flushed the create transaction [ozone]

2025-09-04 Thread via GitHub
jojochuang commented on PR #8999: URL: https://github.com/apache/ozone/pull/8999#issuecomment-3255619486 Merged. Thanks @swamirishi -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] HDDS-13576. Recon reprocess of all tasks should be non-blocking. [ozone]

2025-09-04 Thread via GitHub
sumitagrawl commented on code in PR #8962: URL: https://github.com/apache/ozone/pull/8962#discussion_r2322180470 ## hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/recovery/ReconOmMetadataManagerImpl.java: ## @@ -72,6 +74,31 @@ public ReconOmMetadataManagerImpl(Oz

Re: [PR] HDDS-13638. Update the default backup SST pruning interval to 10min. [ozone]

2025-09-04 Thread via GitHub
SaketaChalamchala commented on PR #9002: URL: https://github.com/apache/ozone/pull/9002#issuecomment-3255423998 @jojochuang and @swamirishi could you take a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] HDDS-13604. Add volume and bucket to checkpoint existence/creation logs. [ozone]

2025-09-04 Thread via GitHub
swamirishi commented on code in PR #8976: URL: https://github.com/apache/ozone/pull/8976#discussion_r2321413049 ## hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmSnapshotManager.java: ## @@ -513,12 +513,18 @@ public static DBCheckpoint createOmSnapshotChec

Re: [PR] [Don't merge] Merge HDDS-5713 disk balancer feature branch to master [ozone]

2025-09-04 Thread via GitHub
Gargi-jais11 commented on code in PR #8932: URL: https://github.com/apache/ozone/pull/8932#discussion_r2321672991 ## hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/DiskBalancerManager.java: ## @@ -0,0 +1,399 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] HDDS-13633. Snapshot Cache loader should check if the double buffer has flushed the create transaction [ozone]

2025-09-04 Thread via GitHub
jojochuang commented on PR #8999: URL: https://github.com/apache/ozone/pull/8999#issuecomment-3250597271 ### What does the PR do? **Current behavior:** When loading a snapshot in Ozone, the Snapshot Cache checks if the directory for the snapshot exists. That’s its main criterion t

Re: [PR] [Don't merge] Merge HDDS-5713 disk balancer feature branch to master [ozone]

2025-09-04 Thread via GitHub
Gargi-jais11 commented on code in PR #8932: URL: https://github.com/apache/ozone/pull/8932#discussion_r2321672991 ## hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/DiskBalancerManager.java: ## @@ -0,0 +1,399 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] HDDS-13633. Snapshot Cache loader should check if the double buffer has flushed the create transaction [ozone]

2025-09-04 Thread via GitHub
jojochuang commented on code in PR #8999: URL: https://github.com/apache/ozone/pull/8999#discussion_r2320276975 ## hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmSnapshotManager.java: ## @@ -397,9 +394,29 @@ public OmSnapshot load(@Nonnull UUID snapshotId)

Re: [PR] HDDS-12998. Bring real container size in pb message when exporting/importing containers [ozone]

2025-09-04 Thread via GitHub
peterxcli commented on PR #8915: URL: https://github.com/apache/ozone/pull/8915#issuecomment-3251944392 please request my review whenever you think this is ready :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [PR] HDDS-5195. Support uploading objects with presigned URL [ozone]

2025-09-04 Thread via GitHub
hevinhsu commented on PR #9000: URL: https://github.com/apache/ozone/pull/9000#issuecomment-3251895061 Correct me if I'm wrong. From what I see, the current implementation does not actually use `x-amz-content-sha256` to verify the payload. Because of that, the current version still fails th

Re: [PR] [Don't merge] Merge HDDS-5713 disk balancer feature branch to master [ozone]

2025-09-04 Thread via GitHub
sumitagrawl commented on code in PR #8932: URL: https://github.com/apache/ozone/pull/8932#discussion_r2321624312 ## hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/diskbalancer/policy/DefaultVolumeChoosingPolicy.java: ## @@ -0,0 +1,110 @@ +/* + * Li

Re: [PR] [Don't merge] Merge HDDS-5713 disk balancer feature branch to master [ozone]

2025-09-04 Thread via GitHub
sumitagrawl commented on code in PR #8932: URL: https://github.com/apache/ozone/pull/8932#discussion_r2321596512 ## hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/DiskBalancerManager.java: ## @@ -0,0 +1,399 @@ +/* + * Licensed to the Apache Software Foundat

[PR] HDDS-13592. Make InfoSubcommand validate all container IDs before proceeding [ozone]

2025-09-04 Thread via GitHub
Gargi-jais11 opened a new pull request, #9001: URL: https://github.com/apache/ozone/pull/9001 ## What changes were proposed in this pull request? [HDDS-12078](https://issues.apache.org/jira/browse/HDDS-12078) added a method ContainerIDParameters#getValidatedIDs to validate and convert con

[PR] HDDS-13638. Update the default backup SST pruning interval to 10min. [ozone]

2025-09-04 Thread via GitHub
SaketaChalamchala opened a new pull request, #9002: URL: https://github.com/apache/ozone/pull/9002 ## What changes were proposed in this pull request? Updated default value of `ozone.om.snapshot.compaction.dag.prune.daemon.run.interval = 10min (1h previous value)` to keep up with AOS

Re: [PR] HDDS-12715. Add integration tests for debug-replicas-verify-checksums tool [ozone]

2025-09-04 Thread via GitHub
Tejaskriya commented on PR #8209: URL: https://github.com/apache/ozone/pull/8209#issuecomment-3253993430 Thanks for the reviews @dombizita @sarvekshayr @adoroszlai , and the contribution @ptlrs -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] HDDS-13518. Move ozone debug ratislogparser to ozone debug ratis parse [ozone]

2025-09-04 Thread via GitHub
Tejaskriya merged PR #8902: URL: https://github.com/apache/ozone/pull/8902 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ozone.apac

Re: [PR] HDDS-13546. Exit code should be non-zero if balancer commands fail [ozone]

2025-09-04 Thread via GitHub
Tejaskriya commented on PR #8910: URL: https://github.com/apache/ozone/pull/8910#issuecomment-3253793386 Thanks for the reviews @siddhantsangwan @sadanand48 @adoroszlai -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] HDDS-13546. Exit code should be non-zero if balancer commands fail [ozone]

2025-09-04 Thread via GitHub
Tejaskriya merged PR #8910: URL: https://github.com/apache/ozone/pull/8910 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ozone.apac

Re: [PR] [Don't merge] Merge HDDS-5713 disk balancer feature branch to master [ozone]

2025-09-04 Thread via GitHub
Gargi-jais11 commented on code in PR #8932: URL: https://github.com/apache/ozone/pull/8932#discussion_r2321610400 ## hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/ContainerSet.java: ## @@ -218,6 +218,29 @@ private void updateContainerI

Re: [PR] [Don't merge] Merge HDDS-5713 disk balancer feature branch to master [ozone]

2025-09-04 Thread via GitHub
Gargi-jais11 commented on code in PR #8932: URL: https://github.com/apache/ozone/pull/8932#discussion_r2321646260 ## hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/diskbalancer/policy/DefaultVolumeChoosingPolicy.java: ## @@ -0,0 +1,110 @@ +/* + * L

Re: [PR] [Don't merge] Merge HDDS-5713 disk balancer feature branch to master [ozone]

2025-09-04 Thread via GitHub
sumitagrawl commented on code in PR #8932: URL: https://github.com/apache/ozone/pull/8932#discussion_r2321612734 ## hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/report/DiskBalancerReportPublisher.java: ## @@ -0,0 +1,64 @@ +/* + * Licensed

Re: [PR] [Don't merge] Merge HDDS-5713 disk balancer feature branch to master [ozone]

2025-09-04 Thread via GitHub
Gargi-jais11 commented on code in PR #8932: URL: https://github.com/apache/ozone/pull/8932#discussion_r2321610400 ## hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/ContainerSet.java: ## @@ -218,6 +218,29 @@ private void updateContainerI

Re: [PR] [Don't merge] Merge HDDS-5713 disk balancer feature branch to master [ozone]

2025-09-04 Thread via GitHub
sumitagrawl commented on code in PR #8932: URL: https://github.com/apache/ozone/pull/8932#discussion_r2321604562 ## hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainer.java: ## @@ -897,6 +901,84 @@ public DataScanResult scanDa

Re: [PR] [Don't merge] Merge HDDS-5713 disk balancer feature branch to master [ozone]

2025-09-04 Thread via GitHub
sumitagrawl commented on code in PR #8932: URL: https://github.com/apache/ozone/pull/8932#discussion_r2321604562 ## hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainer.java: ## @@ -897,6 +901,84 @@ public DataScanResult scanDa

Re: [PR] [Don't merge] Merge HDDS-5713 disk balancer feature branch to master [ozone]

2025-09-04 Thread via GitHub
sumitagrawl commented on code in PR #8932: URL: https://github.com/apache/ozone/pull/8932#discussion_r2321598654 ## hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/DiskBalancerManager.java: ## @@ -0,0 +1,399 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] [Don't merge] Merge HDDS-5713 disk balancer feature branch to master [ozone]

2025-09-04 Thread via GitHub
Gargi-jais11 commented on code in PR #8932: URL: https://github.com/apache/ozone/pull/8932#discussion_r2321567904 ## hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/DiskBalancerManager.java: ## @@ -0,0 +1,399 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] [Don't merge] Merge HDDS-5713 disk balancer feature branch to master [ozone]

2025-09-04 Thread via GitHub
Gargi-jais11 commented on code in PR #8932: URL: https://github.com/apache/ozone/pull/8932#discussion_r2321473061 ## hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/diskbalancer/policy/DefaultVolumeChoosingPolicy.java: ## @@ -0,0 +1,110 @@ +/* + * L

Re: [PR] HDDS-13633. Snapshot Cache loader should check if the double buffer has flushed the create transaction [ozone]

2025-09-04 Thread via GitHub
swamirishi commented on code in PR #8999: URL: https://github.com/apache/ozone/pull/8999#discussion_r2321435272 ## hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmSnapshotManager.java: ## @@ -397,9 +394,29 @@ public OmSnapshot load(@Nonnull UUID snapshotId)

Re: [PR] HDDS-13604. Add volume and bucket to checkpoint existence/creation logs. [ozone]

2025-09-04 Thread via GitHub
swamirishi commented on PR #8976: URL: https://github.com/apache/ozone/pull/8976#issuecomment-3252757633 @ermahesh TestOmSnapshotManager.testCreateSnapshotIdempotent:778 is failing can you fix this and then we can merge this -- This is an automated message from the Apache Git Service. To

Re: [PR] HDDS-13576. Recon reprocess of all tasks should be non-blocking. [ozone]

2025-09-03 Thread via GitHub
sumitagrawl commented on code in PR #8962: URL: https://github.com/apache/ozone/pull/8962#discussion_r2320913596 ## hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/ReconTaskControllerImpl.java: ## @@ -416,19 +459,358 @@ private void processEventBatchDirectl

Re: [PR] [Don't merge] Merge HDDS-5713 disk balancer feature branch to master [ozone]

2025-09-03 Thread via GitHub
Gargi-jais11 commented on code in PR #8932: URL: https://github.com/apache/ozone/pull/8932#discussion_r2320982405 ## hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/report/DiskBalancerReportPublisher.java: ## @@ -0,0 +1,64 @@ +/* + * Licensed

Re: [PR] HDDS-13628. LocalSnapshotMetadata create should track snapshot from checkpoint instead of active object store [ozone]

2025-09-03 Thread via GitHub
jojochuang commented on PR #8982: URL: https://github.com/apache/ozone/pull/8982#issuecomment-3252084758 This PR requires #8981 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [PR] HDDS-13374. [DOCS] Mount Ozone as a Local File System [ozone]

2025-09-03 Thread via GitHub
jojochuang closed pull request #8731: HDDS-13374. [DOCS] Mount Ozone as a Local File System URL: https://github.com/apache/ozone/pull/8731 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] HDDS-5195. Support uploading objects with presigned URL [ozone]

2025-09-03 Thread via GitHub
ivandika3 commented on PR #9000: URL: https://github.com/apache/ozone/pull/9000#issuecomment-3251910708 > Correct me if I'm wrong. From what I see, the current implementation does not actually use x-amz-content-sha256 to verify the payload. Because of that, the current version still fails t

Re: [PR] HDDS-5195. Support uploading objects with presigned URL [ozone]

2025-09-03 Thread via GitHub
hevinhsu commented on code in PR #9000: URL: https://github.com/apache/ozone/pull/9000#discussion_r2320847347 ## hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/util/S3Utils.java: ## @@ -164,9 +164,13 @@ public static void validateMultiChunksUpload(HttpHeaders h

Re: [PR] HDDS-12998. Bring real container size in pb message when exporting/importing containers [ozone]

2025-09-03 Thread via GitHub
Gargi-jais11 commented on code in PR #8915: URL: https://github.com/apache/ozone/pull/8915#discussion_r2320766924 ## hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/ECMisReplicationHandler.java: ## @@ -83,6 +83,7 @@ protected int sendReplica

Re: [PR] HDDS-12998. Bring real container size in pb message when exporting/importing containers [ozone]

2025-09-03 Thread via GitHub
Gargi-jais11 commented on code in PR #8915: URL: https://github.com/apache/ozone/pull/8915#discussion_r2320760681 ## hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/SendContainerRequestHandler.java: ## @@ -84,7 +86,15 @@ public void onNe

Re: [PR] HDDS-12998. Bring real container size in pb message when exporting/importing containers [ozone]

2025-09-03 Thread via GitHub
Gargi-jais11 commented on code in PR #8915: URL: https://github.com/apache/ozone/pull/8915#discussion_r2320759050 ## hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/DownloadAndImportReplicator.java: ## @@ -71,9 +71,20 @@ public void repl

Re: [PR] HDDS-13382. Add RocksDB documentation page [ozone]

2025-09-03 Thread via GitHub
jojochuang commented on code in PR #8740: URL: https://github.com/apache/ozone/pull/8740#discussion_r2320023927 ## hadoop-hdds/docs/content/concept/RocksDB.md: ## @@ -0,0 +1,161 @@ +--- +title: "RocksDB in Apache Ozone" +menu: + main: +parent: Architecture +--- + + + +> Not

[PR] HDDS-13633. Snapshot Cache loader should check if the double buffer has flushed the create transaction [ozone]

2025-09-03 Thread via GitHub
swamirishi opened a new pull request, #8999: URL: https://github.com/apache/ozone/pull/8999 ## What changes were proposed in this pull request? Right now Snapshot Cache while trying to load snapshot checks if the directory merely exists however since we are going to create another snapsho

Re: [PR] HDDS-13536. [Design doc] Add support for Listener OM. [ozone]

2025-09-03 Thread via GitHub
jojochuang commented on PR #8935: URL: https://github.com/apache/ozone/pull/8935#issuecomment-3250330599 Merged. Thanks for review @peterxcli @YutaLin -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] HDDS-13536. [Design doc] Add support for Listener OM. [ozone]

2025-09-03 Thread via GitHub
jojochuang merged PR #8935: URL: https://github.com/apache/ozone/pull/8935 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ozone.apac

Re: [PR] HDDS-13536. [Design doc] Add support for Listener OM. [ozone]

2025-09-03 Thread via GitHub
peterxcli commented on code in PR #8935: URL: https://github.com/apache/ozone/pull/8935#discussion_r2319504436 ## hadoop-hdds/docs/content/design/listener-om.md: ## @@ -0,0 +1,102 @@ +--- +title: "Listener Ozone Manager" +summary: Read-only Ozone Manager to scale out read perfor

Re: [PR] HDDS-13612. Track SST file ranges in sst file metadata [ozone]

2025-09-03 Thread via GitHub
Copilot commented on code in PR #8981: URL: https://github.com/apache/ozone/pull/8981#discussion_r2319490193 ## hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmSnapshotLocalDataYaml.java: ## @@ -91,21 +106,77 @@ public static boolean verifyChecksum(OmSnapsh

Re: [PR] HDDS-12998. Bring real container size in pb message when exporting/importing containers [ozone]

2025-09-03 Thread via GitHub
peterxcli commented on code in PR #8915: URL: https://github.com/apache/ozone/pull/8915#discussion_r2319437109 ## hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/SendContainerRequestHandler.java: ## @@ -84,7 +86,15 @@ public void onNext(

Re: [PR] HDDS-13518. Move ozone debug ratislogparser to ozone debug ratis parse [ozone]

2025-09-03 Thread via GitHub
Tejaskriya commented on code in PR #8902: URL: https://github.com/apache/ozone/pull/8902#discussion_r2319419811 ## hadoop-hdds/docs/content/tools/debug/RatisLogParser.md: ## @@ -22,18 +22,16 @@ weight: 5 limitations under the License. --> -The Ratis log parser tool takes a

Re: [PR] HDDS-10338. Implement a Client Datanode API to stream a block [ozone]

2025-09-03 Thread via GitHub
sodonnel commented on PR #6613: URL: https://github.com/apache/ozone/pull/6613#issuecomment-3249780357 @chungen0126 Could you explain more detail how this change works, and why it should perform better than the current approach? Has this code been run on any cluster to prove it works and al

Re: [PR] HDDS-13576. Recon reprocess of all tasks should be non-blocking. [ozone]

2025-09-03 Thread via GitHub
devmadhuu commented on code in PR #8962: URL: https://github.com/apache/ozone/pull/8962#discussion_r2318998626 ## hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/ReconTaskControllerImpl.java: ## @@ -423,12 +463,348 @@ public void resetEventBufferOverflowFlag

Re: [PR] HDDS-7544. ContainerStateMap uses HashSet to store container replica lists [ozone]

2025-09-03 Thread via GitHub
cchung100m commented on PR #8990: URL: https://github.com/apache/ozone/pull/8990#issuecomment-3249561506 Hi @sodonnel Thanks for the prompt reply and suggestions. I can update the part you mentioned. Hi @peterxcli Any thoughts would be appreciated if you are available.

Re: [PR] [Don't merge] Merge HDDS-5713 disk balancer feature branch to master [ozone]

2025-09-03 Thread via GitHub
Gargi-jais11 commented on code in PR #8932: URL: https://github.com/apache/ozone/pull/8932#discussion_r2319107485 ## hadoop-hdds/interface-admin/src/main/proto/ScmAdminProtocol.proto: ## @@ -86,6 +86,8 @@ message ScmContainerLocationRequest { optional GetMetricsRequestProto g

Re: [PR] [Don't merge] Merge HDDS-5713 disk balancer feature branch to master [ozone]

2025-09-03 Thread via GitHub
Gargi-jais11 commented on code in PR #8932: URL: https://github.com/apache/ozone/pull/8932#discussion_r2319096767 ## hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/DiskBalancerManager.java: ## @@ -0,0 +1,399 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] [Don't merge] Merge HDDS-5713 disk balancer feature branch to master [ozone]

2025-09-03 Thread via GitHub
sumitagrawl commented on code in PR #8932: URL: https://github.com/apache/ozone/pull/8932#discussion_r2319014263 ## hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/ContainerSet.java: ## @@ -218,6 +218,29 @@ private void updateContainerId

Re: [PR] [Don't merge] Merge HDDS-5713 disk balancer feature branch to master [ozone]

2025-09-03 Thread via GitHub
Gargi-jais11 commented on code in PR #8932: URL: https://github.com/apache/ozone/pull/8932#discussion_r2319074801 ## hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainer.java: ## @@ -897,6 +901,84 @@ public DataScanResult scanD

Re: [PR] HDDS-13576. Recon reprocess of all tasks should be non-blocking. [ozone]

2025-09-03 Thread via GitHub
devmadhuu commented on code in PR #8962: URL: https://github.com/apache/ozone/pull/8962#discussion_r2318998626 ## hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/ReconTaskControllerImpl.java: ## @@ -423,12 +463,348 @@ public void resetEventBufferOverflowFlag

Re: [PR] [Don't merge] Merge HDDS-5713 disk balancer feature branch to master [ozone]

2025-09-03 Thread via GitHub
sumitagrawl commented on code in PR #8932: URL: https://github.com/apache/ozone/pull/8932#discussion_r2318989284 ## hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainer.java: ## @@ -897,6 +901,84 @@ public DataScanResult scanDa

Re: [PR] [Don't merge] Merge HDDS-5713 disk balancer feature branch to master [ozone]

2025-09-03 Thread via GitHub
sumitagrawl commented on code in PR #8932: URL: https://github.com/apache/ozone/pull/8932#discussion_r2318958402 ## hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/diskbalancer/policy/DefaultVolumeChoosingPolicy.java: ## @@ -0,0 +1,110 @@ +/* + * Li

Re: [PR] [Don't merge] Merge HDDS-5713 disk balancer feature branch to master [ozone]

2025-09-03 Thread via GitHub
sumitagrawl commented on code in PR #8932: URL: https://github.com/apache/ozone/pull/8932#discussion_r2318956476 ## hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/diskbalancer/policy/DefaultVolumeChoosingPolicy.java: ## @@ -0,0 +1,110 @@ +/* + * Li

Re: [PR] [Don't merge] Merge HDDS-5713 disk balancer feature branch to master [ozone]

2025-09-03 Thread via GitHub
sumitagrawl commented on code in PR #8932: URL: https://github.com/apache/ozone/pull/8932#discussion_r2318934855 ## hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/report/DiskBalancerReportPublisher.java: ## @@ -0,0 +1,64 @@ +/* + * Licensed

Re: [PR] HDDS-13576. Recon reprocess of all tasks should be non-blocking. [ozone]

2025-09-03 Thread via GitHub
sumitagrawl commented on code in PR #8962: URL: https://github.com/apache/ozone/pull/8962#discussion_r2318802159 ## hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/recovery/ReconOmMetadataManagerImpl.java: ## @@ -72,6 +74,29 @@ public ReconOmMetadataManagerImpl(Oz

Re: [PR] HDDS-13344. Fix ProxyUserAuthenticationFilter addition in HttpServer2 [ozone]

2025-09-03 Thread via GitHub
dombizita commented on PR #8710: URL: https://github.com/apache/ozone/pull/8710#issuecomment-3248924593 Thank you for the session and merge @fapifta! It's a really good summary, few additional things that came to my mind: - The SCM Web UI has a logLevel endpoint, which will check for

Re: [PR] HDDS-13533. Show the summary of replicas verify checks [ozone]

2025-09-03 Thread via GitHub
Tejaskriya commented on PR #8898: URL: https://github.com/apache/ozone/pull/8898#issuecomment-3248822075 Thanks for the patch @sarvekshayr and the reviews @errose28 @priyeshkaratha @dombizita -- This is an automated message from the Apache Git Service. To respond to the message, please l

Re: [PR] HDDS-13533. Show the summary of replicas verify checks [ozone]

2025-09-03 Thread via GitHub
Tejaskriya merged PR #8898: URL: https://github.com/apache/ozone/pull/8898 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ozone.apac

Re: [PR] HDDS-13518. Move ozone debug ratislogparser to ozone debug ratis parse [ozone]

2025-09-03 Thread via GitHub
sarvekshayr commented on PR #8902: URL: https://github.com/apache/ozone/pull/8902#issuecomment-3248699819 > I couldn't find a solution, it looks like git is "smart" enough to do this, as the `GenericRatisLogParser` was a short class and the header matches with `package-info`. So it's fine b

Re: [PR] HDDS-13518. Move ozone debug ratislogparser to ozone debug ratis parse [ozone]

2025-09-03 Thread via GitHub
dombizita commented on PR #8902: URL: https://github.com/apache/ozone/pull/8902#issuecomment-3248677956 > > It shows that the `GenericRatisLogParser.java` was renamed to `package-info.java`, which can be misleading as the goal is to delete `GenericRatisLogParser` and add a new `package-info

Re: [PR] HDDS-7544. ContainerStateMap uses HashSet to store container replica lists [ozone]

2025-09-03 Thread via GitHub
sodonnel commented on PR #8990: URL: https://github.com/apache/ozone/pull/8990#issuecomment-3248516175 I was thinking something like this would work: ``` --- a/hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/states/ContainerEntry.java +++ b/hadoop-hdds

Re: [PR] [Don't merge] Merge HDDS-5713 disk balancer feature branch to master [ozone]

2025-09-03 Thread via GitHub
Gargi-jais11 commented on code in PR #8932: URL: https://github.com/apache/ozone/pull/8932#discussion_r2318420759 ## hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/report/DiskBalancerReportPublisher.java: ## @@ -0,0 +1,64 @@ +/* + * Licensed

Re: [PR] [Don't merge] Merge HDDS-5713 disk balancer feature branch to master [ozone]

2025-09-03 Thread via GitHub
Gargi-jais11 commented on code in PR #8932: URL: https://github.com/apache/ozone/pull/8932#discussion_r2318409579 ## hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/report/DiskBalancerReportPublisher.java: ## @@ -0,0 +1,64 @@ +/* + * Licensed

[PR] Hdds 13601 [ozone]

2025-09-03 Thread via GitHub
priyeshkaratha opened a new pull request, #8997: URL: https://github.com/apache/ozone/pull/8997 ## What changes were proposed in this pull request? Provide a one-liner summary of the changes in the PR **Title** field above. It should be in the form of `HDDS-1234. Short summary of the ch

Re: [PR] HDDS-13631. Encrypted bucket creation correction on --key option [ozone]

2025-09-03 Thread via GitHub
venkatsambath commented on PR #8996: URL: https://github.com/apache/ozone/pull/8996#issuecomment-3246625601 > Good catch. Thanks @venkatsambath ! Thanks for the review Wei Chiu. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] HDDS-13465. Adding endpoint at Recon to get consolidated storage distribution report [ozone]

2025-09-03 Thread via GitHub
priyeshkaratha commented on PR #8995: URL: https://github.com/apache/ozone/pull/8995#issuecomment-3248286844 @ArafatKhan2198 @devmadhuu @sumitagrawl @ChenSammi Please review the code changes. -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] HDDS-13594. Use a different endpoint for fetching the OM checkpoint tarball. [ozone]

2025-09-03 Thread via GitHub
priyeshkaratha commented on PR #8963: URL: https://github.com/apache/ozone/pull/8963#issuecomment-3248231143 Looks good for me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [PR] HDDS-13576. Recon reprocess of all tasks should be non-blocking. [ozone]

2025-09-03 Thread via GitHub
devmadhuu commented on code in PR #8962: URL: https://github.com/apache/ozone/pull/8962#discussion_r2315984925 ## hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/ReconTaskControllerImpl.java: ## @@ -431,4 +468,359 @@ public boolean hasDeltaTasksFailed() {

Re: [PR] HDDS-13576. Recon reprocess of all tasks should be non-blocking. [ozone]

2025-09-03 Thread via GitHub
sumitagrawl commented on code in PR #8962: URL: https://github.com/apache/ozone/pull/8962#discussion_r2300415543 ## hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/ReconTaskControllerImpl.java: ## @@ -431,4 +457,156 @@ public boolean hasDeltaTasksFailed() {

  1   2   3   4   5   6   7   8   9   10   >