sezruby opened a new pull request, #12760: URL: https://github.com/apache/gluten/pull/12760
## What changes are proposed in this pull request? Add a `parquet.enable.page.index` write option that maps to Velox's `ParquetWriterOptions::enableWritePageIndex` (facebookincubator/velox#18325), mirroring the existing `parquet.enable.dictionary` passthrough. When enabled, the native Parquet writer emits the column index and offset index (page index); when disabled, they are omitted. - `GlutenConfig` (Scala and C++): define the `parquet.enable.page.index` key. - `VeloxParquetWriterInjects`: forward the option into the native write config. - `VeloxWriterUtils::makeParquetWriteOption`: set `enableWritePageIndex`. - Document the option in the Parquet write configuration table. This also bumps the pinned Velox from `dft-2026_08_05` to `dft-2026_08_12`, the first snapshot that includes facebookincubator/velox#18325 (the `enableWritePageIndex` field). ## How was this patch tested? Added a `VeloxParquetWriteSuite` test that writes with `parquet.enable.page.index` set to both `true` and `false`, asserts (via parquet-mr footer inspection) that the Parquet `ColumnIndex`/`OffsetIndex` references are present exactly when the option is enabled, and verifies the written data round-trips. Ran the test locally against a Velox build that includes the `enableWritePageIndex` field; it passes — the page index is written when the option is enabled and omitted when disabled. ## Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 4.8) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
