maksaska commented on PR #13627: URL: https://github.com/apache/ignite/pull/13627#issuecomment-6799226019
> The resend at the end of onDone is enough to fix the race, even if the other nodes keep writing the owner's state themselves, as master does. Please drop PART_STATE_FROM_OWNER_FEATURE, its checks and the rolling upgrade test, or add a test that fails without them. > > The Jira ticket and the PR description still describe the first version, and the description names a test that no longer exists. Please update both. Thanks, you're right: the resend at the end of onDone alone fixes the race. Our tests pass with the remote-map writes left as in master. What's left is a design choice, and I'd like your view on it. The question. After loss detection and ownOrphans, should the other nodes keep writing the owner's partition state into its row themselves, or wait for the owner's map? Why it matters: their writes can name a false owner. A probe test shows it: 1. Node X is still evicting its copy of partition p, which is RENTING: p has moved to node Y. 2. Y, the only owner of p, leaves. Under IGNORE, p gets a new primary Z, not X. 3. Every other node writes OWNING for p into X's row, since detectLostPartitions writes into any row where p isn't EVICTED. 4. X itself doesn't take p, since own() refuses a RENTING partition. X goes on evicting and sends nothing until the eviction ends. The coordinator, Z and a client then see X as an owner of p while X evicts it: local=RENTING seen=OWNING for all 30 s the probe waits. This happens on master and with the resend alone; with this PR they see RENTING. SQL may send p's part of a query to X, since ReducePartitionMapper picks a random owner, and X answers with a retry. Option 1: the other nodes stop writing (this PR). The owner's map is the only source of its state, as at the end of a rebalance. - Gain: no false owner. In the probe, X's row keeps RENTING. - Cost: after a partition loss, the other nodes see no owner for about 1.5 s, until the owner's map comes with the coordinator's resend. A SQL query started there waits and retries in that time; it doesn't fail. - Code: PART_STATE_FROM_OWNER_FEATURE, a resend on its activation and the rolling upgrade test. The writes stop only after the cluster version is finalized. Option 2: the other nodes keep writing, as in master. - Gain: no 1.5 s wait, a smaller change, no feature. - Cost: the false owner from the probe stays, as in master. I lean to option 1: a partition loss is rare, the wait is safe, and the owner stays the single source of its own state. If you agree, I'll add the probe to the PR as a test. If you prefer option 2, I'll drop the feature, its checks and the rolling upgrade test. Which way do you think Ignite should go? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
