[GitHub] [carbondata] niuge01 commented on a change in pull request #3606: [CARBONDATA-3681] Change default compressor to zstd
niuge01 commented on a change in pull request #3606: [CARBONDATA-3681] Change default compressor to zstd URL: https://github.com/apache/carbondata/pull/3606#discussion_r376770522 ## File path: core/src/main/java/org/apache/carbondata/core/util/path/CarbonTablePath.java ## @@ -285,17 +286,39 @@ public static String getSegmentPath(String tablePath, String segmentId) { } /** - * Gets data file name only with out path - * - * @param filePartNo data file part number - * @param taskNo task identifier - * @param factUpdateTimeStamp unique identifier to identify an update - * @return gets data file name only with out path + * Gets data file name only, without parent path */ public static String getCarbonDataFileName(Integer filePartNo, String taskNo, int bucketNumber, - int batchNo, String factUpdateTimeStamp, String segmentNo) { -return DATA_PART_PREFIX + filePartNo + "-" + taskNo + BATCH_PREFIX + batchNo + "-" -+ bucketNumber + "-" + segmentNo + "-" + factUpdateTimeStamp + CARBON_DATA_EXT; + int batchNo, String factUpdateTimeStamp, String segmentNo, String compressor) { +Objects.requireNonNull(filePartNo); +Objects.requireNonNull(taskNo); +Objects.requireNonNull(factUpdateTimeStamp); +Objects.requireNonNull(compressor); + +// Start from CarbonData 2.0, the data file name patten is: +// partNo-taskNo-batchNo-bucketNo-segmentNo-timestamp.compressor.carbondata +// For example: +// part-0-0_batchno0-0-0-1580982686749.zstd.carbondata +// +// If the compressor name is missing, the file is compressed by snappy, which is +// the default compressor in CarbonData 1.x + +return new StringBuffer().append(DATA_PART_PREFIX) Review comment: There is need not use StringBuffer to build string, just use string concat will be ok. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] niuge01 commented on a change in pull request #3606: [CARBONDATA-3681] Change default compressor to zstd
niuge01 commented on a change in pull request #3606: [CARBONDATA-3681] Change default compressor to zstd URL: https://github.com/apache/carbondata/pull/3606#discussion_r376770113 ## File path: core/src/main/java/org/apache/carbondata/core/readcommitter/LatestFilesReadCommittedScope.java ## @@ -163,7 +163,7 @@ public SegmentRefreshInfo getCommittedSegmentRefreshInfo(Segment segment, Update return segmentRefreshInfo; } - private String getSegmentID(String carbonIndexFileName, String indexFilePath) { + private String getTimestamp(String carbonIndexFileName, String indexFilePath) { Review comment: Why change method name to getTimestamp? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] niuge01 commented on a change in pull request #3606: [CARBONDATA-3681] Change default compressor to zstd
niuge01 commented on a change in pull request #3606: [CARBONDATA-3681] Change default compressor to zstd URL: https://github.com/apache/carbondata/pull/3606#discussion_r376769368 ## File path: core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java ## @@ -1083,7 +1083,7 @@ private CarbonCommonConstants() { * The optional values are 'SNAPPY','GZIP','BZIP2','LZ4','ZSTD' and empty. * Specially, empty means that Carbondata will not compress the sort temp files. */ - public static final String CARBON_SORT_TEMP_COMPRESSOR_DEFAULT = "SNAPPY"; + public static final String CARBON_SORT_TEMP_COMPRESSOR_DEFAULT = "zstd"; Review comment: ```suggestion public static final String CARBON_SORT_TEMP_COMPRESSOR_DEFAULT = "ZSTD"; ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services