ajantha-bhat commented on a change in pull request #3606: [CARBONDATA-3681]
Change default compressor to zstd
URL: https://github.com/apache/carbondata/pull/3606#discussion_r376948840
##########
File path:
core/src/main/java/org/apache/carbondata/core/util/path/CarbonTablePath.java
##########
@@ -285,17 +286,38 @@ public static String getSegmentPath(String tablePath,
String segmentId) {
}
/**
- * Gets data file name only with out path
- *
- * @param filePartNo data file part number
- * @param taskNo task identifier
- * @param factUpdateTimeStamp unique identifier to identify an update
- * @return gets data file name only with out path
+ * Gets data file name only, without parent path
*/
public static String getCarbonDataFileName(Integer filePartNo, String
taskNo, int bucketNumber,
- int batchNo, String factUpdateTimeStamp, String segmentNo) {
- return DATA_PART_PREFIX + filePartNo + "-" + taskNo + BATCH_PREFIX +
batchNo + "-"
- + bucketNumber + "-" + segmentNo + "-" + factUpdateTimeStamp +
CARBON_DATA_EXT;
+ int batchNo, String factUpdateTimeStamp, String segmentNo, String
compressor) {
+ Objects.requireNonNull(filePartNo);
+ Objects.requireNonNull(taskNo);
+ Objects.requireNonNull(factUpdateTimeStamp);
+ Objects.requireNonNull(compressor);
+
+ // Start from CarbonData 2.0, the data file name patten is:
+ // partNo-taskNo-batchNo-bucketNo-segmentNo-timestamp.compressor.carbondata
+ // For example:
+ // part-0-0_batchno0-0-0-1580982686749.zstd.carbondata
+ //
+ // If the compressor name is missing, the file is compressed by snappy,
which is
+ // the default compressor in CarbonData 1.x
+
+ return new StringBuilder()
+ .append(DATA_PART_PREFIX)
+ .append(filePartNo)
+ .append("-")
+ .append(taskNo)
+ .append(BATCH_PREFIX)
+ .append(batchNo)
+ .append("-")
+ .append(bucketNumber)
+ .append("-")
+ .append(segmentNo)
+ .append("-")
+ .append(factUpdateTimeStamp)
Review comment:
supposed to append compressor name here ? I see that argument `compressor`
is not used.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
