johnsolomonj commented on code in PR #18185:
URL: https://github.com/apache/pinot/pull/18185#discussion_r3344604114
##########
pinot-controller/src/main/java/org/apache/pinot/controller/util/ServerSegmentMetadataReader.java:
##########
@@ -144,6 +159,50 @@ public TableMetadataInfo
getAggregatedTableMetadataFromServer(String tableNameWi
}
return l;
}));
+ // Aggregate per-column compression stats from server responses
+ List<ColumnCompressionStatsInfo> serverColStats =
tableMetadataInfo.getColumnCompressionStats();
+ if (serverColStats != null) {
+ for (ColumnCompressionStatsInfo info : serverColStats) {
+ // Skip columns with no meaningful compression data (old raw
segments without persisted codec)
+ if (info.getCodec() == null && !info.hasDictionary()) {
+ continue;
+ }
+ String col = info.getColumn();
+ long[] accum = columnCompressionAccum.computeIfAbsent(col, k ->
new long[2]);
+ // Only accumulate uncompressed size when it is a real value (not
the -1 sentinel from dict columns)
+ if (info.getUncompressedSizeInBytes() >= 0) {
+ accum[0] += info.getUncompressedSizeInBytes();
+ }
+ accum[1] += info.getCompressedSizeInBytes();
+ if (info.getCodec() != null) {
+ columnCodecMap.merge(col, info.getCodec(),
+ (existing, incoming) -> existing.equals(incoming) ? existing
: "MIXED");
+ }
+ columnHasDictMap.put(col, info.hasDictionary());
Review Comment:
Fixed. `hasDictionary` has been removed from the DTO entirely. Encoding type
is now fully expressed through codec ("DICT_ENCODED" / codec string / "MIXED"),
so the mixed-segment ambiguity is gone.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
