anton-vinogradov commented on code in PR #13627:
URL: https://github.com/apache/ignite/pull/13627#discussion_r4538200930
##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionsExchangeFuture.java:
##########
@@ -2391,8 +2394,21 @@ private String exchangeTimingsLogMessage(String header,
List<String> timings) {
cleanIdxRebuildFutures = false;
- for (CacheGroupContext grp : cctx.cache().cacheGroups())
- grp.topology().onExchangeDone(this,
grp.affinity().readyAffinity(res), false);
+ for (CacheGroupContext grp : cctx.cache().cacheGroups()) {
+ if (grp.topology().onExchangeDone(this,
grp.affinity().readyAffinity(res), false))
+ locChangedGrps.add(grp);
+ }
+
+ if (!locChangedGrps.isEmpty()
+ &&
cctx.kernalContext().rollingUpgrade().features().isActive(PART_STATE_FROM_OWNER_FEATURE))
{
Review Comment:
Nodes of any version accept this map, so with the check the bug stays until
the upgrade is finalized.
```suggestion
if (!locChangedGrps.isEmpty()) {
```
##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionsExchangeFuture.java:
##########
@@ -142,6 +142,7 @@
import static
org.apache.ignite.internal.processors.cache.ExchangeDiscoveryEvents.serverJoinEvent;
import static
org.apache.ignite.internal.processors.cache.ExchangeDiscoveryEvents.serverLeftEvent;
import static
org.apache.ignite.internal.processors.cache.persistence.snapshot.IgniteSnapshotManager.isSnapshotOperation;
+import static
org.apache.ignite.internal.processors.rollingupgrade.feature.CoreFeatureRegistry.PART_STATE_FROM_OWNER_FEATURE;
Review Comment:
Unused without the feature check in onDone.
```suggestion
```
##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/topology/GridDhtPartitionTopologyImpl.java:
##########
@@ -1616,14 +1619,18 @@ private boolean
shouldOverridePartitionMap(GridDhtPartitionMap currentMap, GridD
this.lostParts = new HashSet<>(lostParts);
if (exchFut != null && !exchFut.activateCluster()) {
+ boolean ownerReports =
+
ctx.kernalContext().rollingUpgrade().features().isActive(PART_STATE_FROM_OWNER_FEATURE);
+
for (Integer part : lostParts) {
GridDhtLocalPartition locPart =
localPartition(part);
// EVICTED partitions should not be marked
directly as LOST, or
// part.clearFuture lifecycle will be broken
after resetting.
// New partition should be created instead.
if (locPart != null && locPart.state() !=
EVICTED) {
- locPart.markLost();
+ if (locPart.markLost() && ownerReports)
+ changed = true;
Review Comment:
No test runs an exchange with partitions already lost, so nothing checks
this resend. Please add such a test or keep master's code here.
```suggestion
for (Integer part : lostParts) {
GridDhtLocalPartition locPart =
localPartition(part);
// EVICTED partitions should not be marked
directly as LOST, or
// part.clearFuture lifecycle will be broken
after resetting.
// New partition should be created instead.
if (locPart != null && locPart.state() !=
EVICTED) {
locPart.markLost();
```
##########
modules/indexing/src/test/java/org/apache/ignite/internal/processors/cache/IndexingCachePartitionLossStaleMapTest.java:
##########
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache;
+
+import org.apache.ignite.Ignite;
+import org.apache.ignite.cache.query.SqlFieldsQuery;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.configuration.IgniteConfiguration;
+import
org.apache.ignite.internal.processors.cache.distributed.CachePartitionLossStaleMapTest;
+import org.junit.Ignore;
+import org.junit.Test;
+
+/**
+ * Runs {@link CachePartitionLossStaleMapTest} with a SQL query. A query fails
if the node that runs it sees a partition
+ * without an owner for the whole SQL retry timeout, 30 s by default.
+ */
+public class IndexingCachePartitionLossStaleMapTest extends
CachePartitionLossStaleMapTest {
Review Comment:
The query picks nodes from the same map that the base test already compares,
so this class adds no check. Please remove it together with runQuery() and the
ignored case.
##########
modules/core/src/main/java/org/apache/ignite/internal/processors/rollingupgrade/feature/CoreFeatureRegistry.java:
##########
@@ -111,4 +111,10 @@ public class CoreFeatureRegistry {
/** Feature that introduces the snapshot list command. */
public static final IgniteFeature SNAPSHOT_LIST_FEATURE = new
IgniteCoreFeature(4);
+
+ /**
+ * Feature that makes only the node holding a partition write its state
after loss detection or after the node owns
+ * a partition that no node has. The other nodes take the state from that
node's partition map.
+ */
+ public static final IgniteFeature PART_STATE_FROM_OWNER_FEATURE = new
IgniteCoreFeature(5);
Review Comment:
If the feature stays, master already uses 5 for
STATISTICS_CACHE_SIZE_TASK_FEATURE.
```suggestion
public static final IgniteFeature PART_STATE_FROM_OWNER_FEATURE = new
IgniteCoreFeature(6);
```
##########
modules/core/src/test/java/org/apache/ignite/internal/processors/cache/distributed/PartitionStateFromOwnerRollingUpgradeTest.java:
##########
@@ -0,0 +1,189 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache.distributed;
+
+import java.util.List;
+import java.util.UUID;
+import org.apache.ignite.Ignite;
+import org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.internal.IgniteEx;
+import org.apache.ignite.internal.TestRecordingCommunicationSpi;
+import
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsSingleMessage;
+import
org.apache.ignite.internal.processors.rollingupgrade.AbstractRollingUpgradeTest;
+import org.apache.ignite.internal.util.typedef.G;
+import org.apache.ignite.plugin.extensions.communication.Message;
+import org.junit.Test;
+
+import static org.apache.ignite.cache.PartitionLossPolicy.IGNORE;
+import static
org.apache.ignite.internal.processors.cache.distributed.CachePartitionLossStaleMapTest.exchangeFinished;
+import static
org.apache.ignite.internal.processors.cache.distributed.CachePartitionLossStaleMapTest.mismatches;
+import static
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPreloader.DFLT_PRELOAD_RESEND_TIMEOUT;
+import static org.apache.ignite.testframework.GridTestUtils.waitForCondition;
+
+/**
+ * Checks that until {@code PART_STATE_FROM_OWNER_FEATURE} is active the nodes
handle lost partitions as the previous
+ * version does: the other nodes write the new owners' state into their
partition maps themselves. Once the feature is
+ * active, they take it from the owners' maps.
+ * <p>
+ * In both parts a server leaves with the only copies of some partitions under
the IGNORE policy, while the maps that
+ * the other servers send outside an exchange are held, so the coordinator
sees only the state it wrote itself. The
+ * coordinator and the client watch the new owners' partitions. The client
hasn't started the cache, so it keeps the
+ * cache's partition map in a client topology.
+ */
+public class PartitionStateFromOwnerRollingUpgradeTest extends
AbstractRollingUpgradeTest {
+ /** */
+ private static final String CACHE = "cache";
+
+ /** */
+ private static final int PARTS = 64;
+
+ /** Version without {@code PART_STATE_FROM_OWNER_FEATURE}. */
+ private static final String OLD_VER = "2.18.0";
+
+ /** Index of the client node, as started by {@link #startCluster(String)}.
*/
+ private static final int CLIENT = 2;
+
+ /** Index of the server that leaves. */
+ private static final int LEAVING = 3;
+
+ /** Time to wait for the partition maps to match, in milliseconds. */
+ private static final long WAIT = 30_000;
+
+ /**
+ * Before the upgrade the coordinator and the client see the new owners
right after the exchange, with the owners'
+ * maps held. Once all nodes are upgraded and the version is finalized,
the feature is active, and they see the new
+ * owners only after the maps arrive. On activation a server resends its
maps, and a client doesn't.
+ *
+ * @throws Exception If failed.
+ */
+ @Test
+ public void testOwnersReportStateAfterUpgrade() throws Exception {
+ startCluster(OLD_VER);
+ startGrid(LEAVING, OLD_VER);
+
+ grid(0).createCache(new CacheConfiguration<Integer, Integer>(CACHE)
+ .setBackups(0)
+ .setPartitionLossPolicy(IGNORE)
+ .setAffinity(new RendezvousAffinityFunction(false, PARTS)));
+
+ awaitPartitionMapExchange();
+
+ IgniteEx oldCrd = stopLeavingWithMapsHeld();
+
+ for (IgniteEx viewer : new IgniteEx[] {oldCrd, grid(CLIENT)}) {
+ List<String> mismatches = mismatches(viewer, CACHE);
+
+ assertTrue(viewer.name() + " didn't write the new owners' state: "
+ mismatches, mismatches.isEmpty());
+ }
+
+ for (Ignite g : G.allGrids()) {
+ if (g != oldCrd && !g.cluster().localNode().isClient()) {
+ assertFalse(g.name() + " sent its partition map before the
feature is active",
+ TestRecordingCommunicationSpi.spi(g).hasBlockedMessages());
+ }
+ }
+
Review Comment:
Once the resend in onDone has no feature check, a server sends its map
before activation too.
```suggestion
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]