kunal642 commented on a change in pull request #4189:
URL: https://github.com/apache/carbondata/pull/4189#discussion_r686928708
##########
File path:
hadoop/src/main/java/org/apache/carbondata/hadoop/api/CarbonTableInputFormat.java
##########
@@ -601,4 +599,60 @@ public String getSegmentIdFromFilePath(String filePath) {
}
return CarbonCommonConstants.INVALID_SEGMENT_ID;
}
+
+ /**
+ * return valid segment to access
+ * first check for mapreduce.input.carboninputformat.segmentnumbers"
+ * second check for table property of latest_segment for query
+ */
+ public Segment[] getSegmentsToAccess(JobContext job, ReadCommittedScope
readCommittedScope,
+ List<Segment> validSegments) {
+ String segmentString = job.getConfiguration().get(INPUT_SEGMENT_NUMBERS,
"");
+ boolean queryLatestSegment = false;
+ if (null != carbonTable) {
+ queryLatestSegment = Boolean.parseBoolean(carbonTable.getTableInfo()
+ .getFactTable().getTableProperties()
+ .getOrDefault(CarbonCommonConstants.TABLE_QUERY_LATEST_SEGMENT,
"false"));
+ }
+ if (segmentString.trim().isEmpty()) {
+ if (!queryLatestSegment) {
+ return new Segment[0];
+ } else {
+ List<Segment> segments = getLatestSegment(validSegments);
+ return segments.toArray(new Segment[0]);
+ }
+ } else {
+ List<Segment> segments = Segment.toSegmentList(segmentString.split(","),
readCommittedScope);
+ if (!queryLatestSegment) {
+ return segments.toArray(new Segment[0]);
+ } else {
+ List<Segment> latestSegment;
+ if (segments.size() > 0 &&
segments.get(0).getSegmentNo().equalsIgnoreCase("*")) {
+ latestSegment = getLatestSegment(validSegments);
+ } else {
+ latestSegment = getLatestSegment(segments);
+ }
+ return latestSegment.toArray(new Segment[0]);
+ }
+ }
+ }
+
+ /**
+ * get the latest segment
+ * @param validSegments the in put segment for search
+ * @return the latest segment for query
+ */
+ public List<Segment> getLatestSegment(List<Segment> validSegments) {
Review comment:
if we need a single segment, then why return type is List<Segment>?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
