MarvinLitt commented on a change in pull request #4189:
URL: https://github.com/apache/carbondata/pull/4189#discussion_r687310576
##########
File path:
hadoop/src/main/java/org/apache/carbondata/hadoop/api/CarbonTableInputFormat.java
##########
@@ -601,4 +599,60 @@ public String getSegmentIdFromFilePath(String filePath) {
}
return CarbonCommonConstants.INVALID_SEGMENT_ID;
}
+
+ /**
+ * return valid segment to access
+ * first check for mapreduce.input.carboninputformat.segmentnumbers"
+ * second check for table property of latest_segment for query
+ */
+ public Segment[] getSegmentsToAccess(JobContext job, ReadCommittedScope
readCommittedScope,
+ List<Segment> validSegments) {
+ String segmentString = job.getConfiguration().get(INPUT_SEGMENT_NUMBERS,
"");
Review comment:
the old getSegmentsToAccess fun just use INPUT_SEGMENT_NUMBERS for input
to get the segment List.
But now we need get segment not just INPUT_SEGMENT_NUMBERS but alse latest
segment. the validSegments is need to use.
if use getSegmentsToAccess(JobContext job, ReadCommittedScope
readCommittedScope) we need to analysis readCommittedScope to validSegments
that the external functions have been implemented.
so i choose func overload to do this function.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
