maytasm commented on a change in pull request #11025:
URL: https://github.com/apache/druid/pull/11025#discussion_r605154810
##########
File path:
indexing-service/src/main/java/org/apache/druid/indexing/common/task/AbstractBatchIndexTask.java
##########
@@ -490,6 +491,27 @@ public static boolean isGuaranteedRollup(IndexIOConfig
ioConfig, IndexTuningConf
}
}
+ public static Set<DataSegment> getUsedSegmentsWithinInterval(
+ TaskToolbox toolbox,
+ String dataSource,
+ List<Interval> intervals
+ ) throws IOException
+ {
+ Set<DataSegment> segmentsFoundForDrop = new HashSet<>();
+ List<Interval> condensedIntervals = JodaUtils.condenseIntervals(intervals);
+ if (!intervals.isEmpty()) {
+ Collection<DataSegment> usedSegment =
toolbox.getTaskActionClient().submit(new RetrieveUsedSegmentsAction(dataSource,
null, condensedIntervals, Segments.ONLY_VISIBLE));
+ for (DataSegment segment : usedSegment) {
+ for (Interval interval : condensedIntervals) {
+ if (interval.contains(segment.getInterval())) {
Review comment:
That is not the case. For example, if you have a segment with interval
2000-01-01/2001-01-01 and your interval specified is 2000-04-28/2000-04-29.
Then if the above segment has data for the day 2000-04-28/2000-04-29, it would
be returned by RetrieveUsedSegmentsAction. However, we cannot drop this segment
since the interval specified is 2000-04-28/2000-04-29. We can only drop
segments that starts and ends within the interval specified.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
