siddharthteotia commented on code in PR #9454:
URL: https://github.com/apache/pinot/pull/9454#discussion_r982044603
##########
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/segment/index/loader/SegmentPreProcessorTest.java:
##########
@@ -304,6 +306,28 @@ public void testEnableFSTIndexOnExistingColumnDictEncoded()
checkFSTIndexCreation(EXISTING_STRING_COL_DICT, 9, 4,
_newColumnsSchemaWithFST, false, false, 26);
}
+ @Test
+ public void testForwardIndexHandler()
+ throws Exception {
+ Map<String, ChunkCompressionType> compressionConfigs = new HashMap<>();
+ ChunkCompressionType newCompressionType = ChunkCompressionType.ZSTANDARD;
+ compressionConfigs.put(EXISTING_STRING_COL_RAW, newCompressionType);
+ _indexLoadingConfig.setCompressionConfigs(compressionConfigs);
+ _indexLoadingConfig.setNoDictionaryColumns(new HashSet<String>() {{
+ add(EXISTING_STRING_COL_RAW);
+ }});
+
+ // Test1: Rewriting forward index will be a no-op for v1 segments. Default
LZ4 compressionType will be retained.
+ constructV1Segment();
+ checkForwardIndexCreation(EXISTING_STRING_COL_RAW, 5, 3, _schema, false,
false, false, 0, ChunkCompressionType.LZ4);
+
+ // Convert the segment to V3.
+ new SegmentV1V2ToV3FormatConverter().convert(_indexDir);
+
+ // Test2: Now forward index will be rewritten with ZSTANDARD
compressionType.
Review Comment:
Can we also cover the following scenarios if not already covered in
`ForwardIndexHandler` ?
- The column on which a `CHANGE_RAW_COMPRESSION_TYPE` action is needed,
another orthogonal action on the same column is needed (e.g range index or a
text index which can be enabled on noDict columns) by the default column
handler. Run some queries that touch the column in SELECT and WHERE to leverage
the changed fwd index (with new comp codec) and newly added inv index.
Basically just to ensure that newly introduced handler works with the existing
handlers and nothing is broken, their sequencing is correct etc.
- Similarly, the column on which a `CHANGE_RAW_COMPRESSION_TYPE` action is
needed, another orthogonal action on a different column is needed. Run some
queries as above.....This is again to ensure we haven't somehow messed with
existing handlers across same or different columns.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
