shuturmurgh opened a new issue, #20229: URL: https://github.com/apache/pinot/issues/20229
## Problem Segment lineage detects conflicting replacements only through overlapping segment names. When two writers replace the same logical partition (a time window or a date partition) while it is empty, both send `segmentsFrom = []`. Nothing overlaps, so both replacements are accepted, both commit, and the partition is served twice. 1. A calls `startReplaceSegments(from=[], to=[a0])` and uploads `a0`, which stays hidden while A's entry is `IN_PROGRESS`. 2. B lists the partition. `a0` is hidden, so B calls `startReplaceSegments(from=[], to=[b0])`. B is accepted, and A's entry isn't reverted, even with `forceCleanup=true`. 3. Both call `endReplaceSegments`. Both `a0` and `b0` are served. **Affected:** `MaterializedViewTask` APPEND on a new window, and any job that builds or rebuilds a partition through the lineage APIs. For `MaterializedViewTask`, the duplication is silent. The second `appendValid` sees the same fingerprint and treats it as a replay, so both tasks succeed. ## Why existing safeguards miss it - **The scheduler relies on the task state Helix reports.** A deleted task, or a minion that lost its ZK session, can still be running. `TaskStateModel.reset()` cancels without waiting, and the MV executor ignores cancellation. - **`DistributedTaskLockManager` covers too little.** It is table-level, off by default, and held only while tasks are generated. - **`endReplaceSegments` checks only one thing:** that the entry is `IN_PROGRESS`. ## Proposal Add partition-key locks that the controller enforces inside the lineage compare-and-set: - **Acquire, renew, release.** Acquire takes `(table, partitionKey, leaseMs)` and returns a fencing token that increases with every acquisition. - **Fenced lineage writes.** Start, end and revert carry `(partitionKey, token)`, and the controller rejects them unless the token is current. - **Takeover.** Taking over an expired lease reverts that key's `IN_PROGRESS` entries, so a stalled holder fails at its next lineage write. - **Writers list segments after acquiring the lock.** That way `segmentsFrom` is always computed under the lock. Lock state lives in the lineage ZNode `customMap`, or in a separate ZNode written in the same ZK multi-op. The feature is opt-in. A controller that doesn't support it must reject the key rather than ignore it. **Alternative:** an optimistic per-key version, checked at start and end, where the first writer to commit wins. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
