vinooganesh opened a new issue, #3863: URL: https://github.com/apache/parquet-java/issues/3863
### Describe the enhancement requested Follow-up from @xiaoxuandev's review comment on #3397: https://github.com/apache/parquet-java/pull/3397#discussion_r4289000466 The ALP writer's parameter search (`AlpCodec#findBestDoubleParams`) encodes the full vector for every candidate (e, f) pair. Measured on 1M doubles with 20480-value pages, `DoubleAlpValuesWriter` takes about 70 ns/value, against about 7 ns/value for `PlainValuesWriter`. The spec's Parameter Selection section describes sampling as an option (256 values per vector), and the reference implementation samples 32. A strided sample brought the writer to about 24 ns/value, with byte-identical output on decimal data that still decodes bit-exactly. One case needs handling before this is safe. On random doubles the sampled version produced about 27% larger output, because a sample containing a single encodable value scores bit width 0 while the full vector then needs about 60 bits. Sampling needs a guard for the mostly-exceptions case, or a fallback to the full search when the sample looks unreliable. ### Component(s) Core -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
