deepthi912 commented on code in PR #19349:
URL: https://github.com/apache/pinot/pull/19349#discussion_r3902063426
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/startree/StarTreeBuilderUtils.java:
##########
@@ -283,6 +286,37 @@ public static boolean
shouldModifyExistingStarTrees(List<StarTreeV2BuilderConfig
return false;
}
+ /// Returns the first dimension of the given star-tree that the segment can
no longer back with a dictionary
+ /// encoded forward index, or `null` if the star-tree is loadable.
+ ///
+ /// A star-tree stores its dimension values as dictionary ids in a fixed-bit
forward index whose bit width is read
+ /// from the *main* column metadata at load time. Re-encoding a dimension
column to raw (e.g. after adding it to
+ /// `noDictionaryColumns`) therefore leaves the star-tree unreadable, and
loading the segment fails.
+ @Nullable
+ public static String findUnloadableDimension(StarTreeV2Metadata
starTreeMetadata, SegmentMetadata segmentMetadata) {
+ for (String dimension : starTreeMetadata.getDimensionsSplitOrder()) {
+ ColumnMetadata columnMetadata =
segmentMetadata.getColumnMetadataFor(dimension);
+ if (columnMetadata == null || !columnMetadata.hasDictionary()) {
Review Comment:
Yup this `columnMetadata.hasDictionary()`, this check should handle for the
dictionary even for `RAW` dimensions...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
