anton-vinogradov commented on code in PR #13627:
URL: https://github.com/apache/ignite/pull/13627#discussion_r4538200930


##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionsExchangeFuture.java:
##########
@@ -2391,8 +2394,21 @@ private String exchangeTimingsLogMessage(String header, 
List<String> timings) {
 
                 cleanIdxRebuildFutures = false;
 
-                for (CacheGroupContext grp : cctx.cache().cacheGroups())
-                    grp.topology().onExchangeDone(this, 
grp.affinity().readyAffinity(res), false);
+                for (CacheGroupContext grp : cctx.cache().cacheGroups()) {
+                    if (grp.topology().onExchangeDone(this, 
grp.affinity().readyAffinity(res), false))
+                        locChangedGrps.add(grp);
+                }
+
+                if (!locChangedGrps.isEmpty()
+                    && 
cctx.kernalContext().rollingUpgrade().features().isActive(PART_STATE_FROM_OWNER_FEATURE))
 {

Review Comment:
   Nodes of any version accept this map, so with the check the bug stays until 
the upgrade is finalized.
   ```suggestion
                   if (!locChangedGrps.isEmpty()) {
   ```



##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionsExchangeFuture.java:
##########
@@ -142,6 +142,7 @@
 import static 
org.apache.ignite.internal.processors.cache.ExchangeDiscoveryEvents.serverJoinEvent;
 import static 
org.apache.ignite.internal.processors.cache.ExchangeDiscoveryEvents.serverLeftEvent;
 import static 
org.apache.ignite.internal.processors.cache.persistence.snapshot.IgniteSnapshotManager.isSnapshotOperation;
+import static 
org.apache.ignite.internal.processors.rollingupgrade.feature.CoreFeatureRegistry.PART_STATE_FROM_OWNER_FEATURE;

Review Comment:
   Unused without the feature check in onDone.
   ```suggestion
   ```



##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/topology/GridDhtPartitionTopologyImpl.java:
##########
@@ -1616,14 +1619,18 @@ private boolean 
shouldOverridePartitionMap(GridDhtPartitionMap currentMap, GridD
                         this.lostParts = new HashSet<>(lostParts);
 
                         if (exchFut != null && !exchFut.activateCluster()) {
+                            boolean ownerReports =
+                                
ctx.kernalContext().rollingUpgrade().features().isActive(PART_STATE_FROM_OWNER_FEATURE);
+
                             for (Integer part : lostParts) {
                                 GridDhtLocalPartition locPart = 
localPartition(part);
 
                                 // EVICTED partitions should not be marked 
directly as LOST, or
                                 // part.clearFuture lifecycle will be broken 
after resetting.
                                 // New partition should be created instead.
                                 if (locPart != null && locPart.state() != 
EVICTED) {
-                                    locPart.markLost();
+                                    if (locPart.markLost() && ownerReports)
+                                        changed = true;

Review Comment:
   No test runs an exchange with partitions already lost, so nothing checks 
this resend. Please add such a test or keep master's code here.
   ```suggestion
                               for (Integer part : lostParts) {
                                   GridDhtLocalPartition locPart = 
localPartition(part);
   
                                   // EVICTED partitions should not be marked 
directly as LOST, or
                                   // part.clearFuture lifecycle will be broken 
after resetting.
                                   // New partition should be created instead.
                                   if (locPart != null && locPart.state() != 
EVICTED) {
                                       locPart.markLost();
   ```



##########
modules/indexing/src/test/java/org/apache/ignite/internal/processors/cache/IndexingCachePartitionLossStaleMapTest.java:
##########
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache;
+
+import org.apache.ignite.Ignite;
+import org.apache.ignite.cache.query.SqlFieldsQuery;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.configuration.IgniteConfiguration;
+import 
org.apache.ignite.internal.processors.cache.distributed.CachePartitionLossStaleMapTest;
+import org.junit.Ignore;
+import org.junit.Test;
+
+/**
+ * Runs {@link CachePartitionLossStaleMapTest} with a SQL query. A query fails 
if the node that runs it sees a partition
+ * without an owner for the whole SQL retry timeout, 30 s by default.
+ */
+public class IndexingCachePartitionLossStaleMapTest extends 
CachePartitionLossStaleMapTest {

Review Comment:
   The query picks nodes from the same map that the base test already compares, 
so this class adds no check. Please remove it together with runQuery() and the 
ignored case.



##########
modules/core/src/main/java/org/apache/ignite/internal/processors/rollingupgrade/feature/CoreFeatureRegistry.java:
##########
@@ -111,4 +111,10 @@ public class CoreFeatureRegistry {
 
     /** Feature that introduces the snapshot list command. */
     public static final IgniteFeature SNAPSHOT_LIST_FEATURE = new 
IgniteCoreFeature(4);
+
+    /**
+     * Feature that makes only the node holding a partition write its state 
after loss detection or after the node owns
+     * a partition that no node has. The other nodes take the state from that 
node's partition map.
+     */
+    public static final IgniteFeature PART_STATE_FROM_OWNER_FEATURE = new 
IgniteCoreFeature(5);

Review Comment:
   If the feature stays, master already uses 5 for 
STATISTICS_CACHE_SIZE_TASK_FEATURE.
   ```suggestion
       public static final IgniteFeature PART_STATE_FROM_OWNER_FEATURE = new 
IgniteCoreFeature(6);
   ```



##########
modules/core/src/test/java/org/apache/ignite/internal/processors/cache/distributed/PartitionStateFromOwnerRollingUpgradeTest.java:
##########
@@ -0,0 +1,189 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache.distributed;
+
+import java.util.List;
+import java.util.UUID;
+import org.apache.ignite.Ignite;
+import org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.internal.IgniteEx;
+import org.apache.ignite.internal.TestRecordingCommunicationSpi;
+import 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsSingleMessage;
+import 
org.apache.ignite.internal.processors.rollingupgrade.AbstractRollingUpgradeTest;
+import org.apache.ignite.internal.util.typedef.G;
+import org.apache.ignite.plugin.extensions.communication.Message;
+import org.junit.Test;
+
+import static org.apache.ignite.cache.PartitionLossPolicy.IGNORE;
+import static 
org.apache.ignite.internal.processors.cache.distributed.CachePartitionLossStaleMapTest.exchangeFinished;
+import static 
org.apache.ignite.internal.processors.cache.distributed.CachePartitionLossStaleMapTest.mismatches;
+import static 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPreloader.DFLT_PRELOAD_RESEND_TIMEOUT;
+import static org.apache.ignite.testframework.GridTestUtils.waitForCondition;
+
+/**
+ * Checks that until {@code PART_STATE_FROM_OWNER_FEATURE} is active the nodes 
handle lost partitions as the previous
+ * version does: the other nodes write the new owners' state into their 
partition maps themselves. Once the feature is
+ * active, they take it from the owners' maps.
+ * <p>
+ * In both parts a server leaves with the only copies of some partitions under 
the IGNORE policy, while the maps that
+ * the other servers send outside an exchange are held, so the coordinator 
sees only the state it wrote itself. The
+ * coordinator and the client watch the new owners' partitions. The client 
hasn't started the cache, so it keeps the
+ * cache's partition map in a client topology.
+ */
+public class PartitionStateFromOwnerRollingUpgradeTest extends 
AbstractRollingUpgradeTest {
+    /** */
+    private static final String CACHE = "cache";
+
+    /** */
+    private static final int PARTS = 64;
+
+    /** Version without {@code PART_STATE_FROM_OWNER_FEATURE}. */
+    private static final String OLD_VER = "2.18.0";
+
+    /** Index of the client node, as started by {@link #startCluster(String)}. 
*/
+    private static final int CLIENT = 2;
+
+    /** Index of the server that leaves. */
+    private static final int LEAVING = 3;
+
+    /** Time to wait for the partition maps to match, in milliseconds. */
+    private static final long WAIT = 30_000;
+
+    /**
+     * Before the upgrade the coordinator and the client see the new owners 
right after the exchange, with the owners'
+     * maps held. Once all nodes are upgraded and the version is finalized, 
the feature is active, and they see the new
+     * owners only after the maps arrive. On activation a server resends its 
maps, and a client doesn't.
+     *
+     * @throws Exception If failed.
+     */
+    @Test
+    public void testOwnersReportStateAfterUpgrade() throws Exception {
+        startCluster(OLD_VER);
+        startGrid(LEAVING, OLD_VER);
+
+        grid(0).createCache(new CacheConfiguration<Integer, Integer>(CACHE)
+            .setBackups(0)
+            .setPartitionLossPolicy(IGNORE)
+            .setAffinity(new RendezvousAffinityFunction(false, PARTS)));
+
+        awaitPartitionMapExchange();
+
+        IgniteEx oldCrd = stopLeavingWithMapsHeld();
+
+        for (IgniteEx viewer : new IgniteEx[] {oldCrd, grid(CLIENT)}) {
+            List<String> mismatches = mismatches(viewer, CACHE);
+
+            assertTrue(viewer.name() + " didn't write the new owners' state: " 
+ mismatches, mismatches.isEmpty());
+        }
+
+        for (Ignite g : G.allGrids()) {
+            if (g != oldCrd && !g.cluster().localNode().isClient()) {
+                assertFalse(g.name() + " sent its partition map before the 
feature is active",
+                    TestRecordingCommunicationSpi.spi(g).hasBlockedMessages());
+            }
+        }
+

Review Comment:
   Once the resend in onDone has no feature check, a server sends its map 
before activation too.
   ```suggestion
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to