rohityadav1993 commented on code in PR #15863:
URL: https://github.com/apache/pinot/pull/15863#discussion_r2100027839
##########
pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/upsertcompactmerge/UpsertCompactMergeTaskGenerator.java:
##########
@@ -214,15 +216,24 @@ public List<PinotTaskConfig>
generateTasks(List<TableConfig> tableConfigs) {
continue;
}
// TODO see if multiple groups of same partition can be added
- Map<String, String> configs = new
HashMap<>(getBaseTaskConfigs(tableConfig,
- groups.get(0).stream().map(x ->
x.getSegmentZKMetadata().getSegmentName()).collect(Collectors.toList())));
+
+ List<String> segmentNames =
+ groups.get(0).stream().map(x ->
x.getSegmentZKMetadata().getSegmentName()).collect(Collectors.toList());
+ //get max creation time for the segments across all servers. This will
be used as the creation time of the
+ // merge segment
+ Long maxCreationTimeMillis =
+ getMaxCreationTimeMillis(tableNameWithType, segmentNames,
serverToSegments, serverToEndpoints,
Review Comment:
There is a scenario in addOrReplace where and alrady compacted segment is
getting compacted but we will prefer the exisitng segment for a primary key due
to the [tie breaker
logic](https://github.com/apache/pinot/blob/0f38fd7cd73198024b79d334b35a9829d68bfd3f/pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/BasePartitionUpsertMetadataManager.java#L506):
```
// if both are uploaded segment, prefer standard
UploadedRealtimeSegmentName, if still a tie, then resolve to
// current segment
if (UploadedRealtimeSegmentName.of(currentSegmentName) != null) {
return false;
}
return UploadedRealtimeSegmentName.of(segmentName) != null;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
