deepthi912 commented on code in PR #17503:
URL: https://github.com/apache/pinot/pull/17503#discussion_r2696535122
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/ConcurrentMapPartitionUpsertMetadataManagerForConsistentDeletes.java:
##########
@@ -193,7 +197,7 @@ currentSegmentName, getAuthoritativeCreationTime(segment),
// New primary key
addDocId(segment, validDocIds, queryableDocIds, newDocId,
recordInfo);
RecordLocation newRecordLocation = new RecordLocation(segment,
newDocId, newComparisonValue, 1);
- _newlyAddedKeys.put(primaryKey, newRecordLocation);
+ _previousKeyToRecordLocationMap.put(primaryKey,
newRecordLocation);
Review Comment:
Hmm, that's how we can track the previous locations when immutable segment
is pushed right? Without this, we cannot identify the previous location in the
immutable segment.
For example:
1. Consuming segment started consuming
2. Immutable segment PK1 pushed via some FIT task, with time stamp T2 came in
3. Consuming segment PK1, with timestamp T3 came in.
4. Now the consuming segment is replaced with a segment on another server
(with lesser consumed rows), maybe with no PK1->T3 entry came in. After the
segment is replaced and keys are reverted, the key has to be pointing to PK1 ->
T2 present in the Immutable Offline segment pushed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
