[ https://issues.apache.org/jira/browse/IGNITE-22637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Maksim Zhuravkov updated IGNITE-22637: -------------------------------------- Description: To select a compaction time, each node should provide its MinimalRequiredTime to compaction coordinator. h4. Prerequisites: In {{PartitionListener}} using partitionDataStorage.flush(false) store {{minActiveTxBeginTime}} into another transient field (for example {{snapshottedMinActiveTxBeginTime}}). Need to investigate how to obtain this {{snapshottedMinActiveTxBeginTime}} from {{CatalogCompactionRunner}} in production code (may be introduce necessary methods). Example of this in test code can be found in {{ItCatalogCompactionTest.ensureTimestampStoredInAllReplicas}} h4. Compute minimum required time 1. Get current value of low watermark. 2. Get min of "snapshotted" minActiveTxTime across all partitions (see {{PartitionListener.minActiveTxBeginTime}}. Take min of 1., 2 and return it in {{CatalogCompactionRunner.determineLocalMinimumRequiredTime}} h4. Additional notes Compaction must start only under the following conditions: 1. no active builds of indexes -- check for indexes are done on coordinator node 2. no active rebalance referring an old versions -- rebalance started with the latest version (this point is moved to https://issues.apache.org/jira/browse/IGNITE-23059) Depending on the complexity, the last part may be implemented as a separate ticket, but compaction should not be enabled until all parts are implemented. was: To select a compaction time, each node should provide its MinimalRequiredTime to compaction coordinator. h4. Prerequisites: In {{PartitionListener}} using partitionDataStorage.flush(false) store {{minActiveTxBeginTime}} into another transient field (for example {{snapshottedMinActiveTxBeginTime}}). Need to investigate how to obtain this {{snapshottedMinActiveTxBeginTime}} from {{CatalogCompactionRunner}} in production code (may be introduce necessary methods). Example of this in test code can be found in {{ItCatalogCompactionTest.ensureTimestampStoredInAllReplicas}} h4. Compute minimum required time 1. Get current value of low watermark. 2. Get min of "snapshotted" minActiveTxTime across all partitions (see {{PartitionListener.minActiveTxBeginTime}}. Take min of 1., 2 and return it in {{CatalogCompactionRunner.determineLocalMinimumRequiredTime}} h4. Additional notes Compaction must start only under the following conditions: 1. no active builds of indexes -- check for indexes are done on coordinator node 2. no active rebalance referring an old versions -- rebalance started with the latest version Depending on the complexity, the last part may be implemented as a separate ticket, but compaction should not be enabled until all parts are implemented. > Catalog compaction. Implement remote node part of compaction flow > ----------------------------------------------------------------- > > Key: IGNITE-22637 > URL: https://issues.apache.org/jira/browse/IGNITE-22637 > Project: Ignite > Issue Type: Improvement > Reporter: Maksim Zhuravkov > Assignee: Maksim Zhuravkov > Priority: Major > Labels: ignite-3 > Time Spent: 0.5h > Remaining Estimate: 0h > > To select a compaction time, each node should provide its MinimalRequiredTime > to compaction coordinator. > h4. Prerequisites: > In {{PartitionListener}} using partitionDataStorage.flush(false) store > {{minActiveTxBeginTime}} into another transient field (for example > {{snapshottedMinActiveTxBeginTime}}). > Need to investigate how to obtain this {{snapshottedMinActiveTxBeginTime}} > from {{CatalogCompactionRunner}} in production code (may be introduce > necessary methods). Example of this in test code can be found in > {{ItCatalogCompactionTest.ensureTimestampStoredInAllReplicas}} > h4. Compute minimum required time > 1. Get current value of low watermark. > 2. Get min of "snapshotted" minActiveTxTime across all partitions (see > {{PartitionListener.minActiveTxBeginTime}}. > Take min of 1., 2 and return it in > {{CatalogCompactionRunner.determineLocalMinimumRequiredTime}} > h4. Additional notes > Compaction must start only under the following conditions: > 1. no active builds of indexes -- check for indexes are done on coordinator > node > 2. no active rebalance referring an old versions -- rebalance started with > the latest version (this point is moved to > https://issues.apache.org/jira/browse/IGNITE-23059) > Depending on the complexity, the last part may be implemented as a separate > ticket, but compaction should not be enabled until all parts are implemented. -- This message was sent by Atlassian Jira (v8.20.10#820010)