xiangfu0 commented on PR #18229: URL: https://github.com/apache/pinot/pull/18229#issuecomment-5322876871
Thanks, Jackie. I have started splitting this into GitHub-native stacked PRs: - #19284 introduces the DSL and `ForwardIndexConfig.codecSpec` wiring first. The outer `CODEC(...)` wrapper is removed; the syntax is now an ordered invocation list such as `DELTA,ZSTD(3)`. - #19285 adds only the bounded internal runtime and the four compression handlers. It remains inert because all layer-1 gates stay closed. - #19281 and #19282 extract two independent correctness prerequisites so they can be reviewed and merged separately. The V7 reader/writer, activation, and transforms will remain separate follow-up stack layers rather than returning to one large PR. Variable-length values are deferred from the first new-format slice, not a permanent non-goal. A single legacy-compatible compression spec can eventually reuse the existing variable-byte readers and writers. General chained or non-default codec pipelines need an explicit variable-width envelope because the current proposed V7 layout assumes fixed entry width, derives decoded bytes from document count times width, and uses fixed-width random access. Numeric transforms such as DELTA and packing transforms also remain INT/LONG-specific. I will keep variable-width SV/MV pipeline support as a linked follow-up after the fixed-width format contract is reviewed rather than silently freezing the current fixed-width assumptions into its future design. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
