rohityadav1993 commented on code in PR #14443:
URL: https://github.com/apache/pinot/pull/14443#discussion_r1841685172
##########
pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-common/src/main/java/org/apache/pinot/plugin/ingestion/batch/common/SegmentGenerationTaskRunner.java:
##########
@@ -167,19 +167,25 @@ private SegmentNameGenerator
getSegmentNameGenerator(SegmentGeneratorConfig segm
return new
InputFileSegmentNameGenerator(segmentNameGeneratorConfigs.get(FILE_PATH_PATTERN),
segmentNameGeneratorConfigs.get(SEGMENT_NAME_TEMPLATE),
inputFileUri, appendUUIDToSegmentName);
case BatchConfigProperties.SegmentNameGeneratorType.UPLOADED_REALTIME:
- Preconditions.checkState(segmentGeneratorConfig.getCreationTime() !=
null,
Review Comment:
uploadTime is a property of the ingestionJob(associated with
`segmentGeneratorConfig.getCreationTime`) to provide deterministic names to
segments based on when the job ran and re-running replaces the segments. Moving
this to segmentNameGeneratorConfig means it is no more a job property and can
be set independently.
The upstream job should have the responsibility to either use its creation
time or default to currentTime/any other time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
