cshuo opened a new pull request, #20671:
URL: https://github.com/apache/hudi/pull/20671

   ### Describe the issue this Pull Request addresses
   
   Closes https://github.com/apache/hudi/issues/20669.
   
   Native Parquet and ORC log writers build Hudi record-key bloom filters that 
their read paths do not use. MDT bloom-filter indexing also excludes log files, 
leaving unnecessary per-record CPU work and footer metadata.
   
   ### Summary and Changelog
   
   - Disable Hudi record-key bloom generation for native Parquet logs across 
the Avro, Spark, and Flink writer factories, including when BLOOM/GLOBAL_BLOOM 
is configured.
   - Disable the equivalent bloom generation for native ORC logs. Preserve 
existing base-file and HFile behavior.
   - Add focused real-file tests for native data/delete logs and base files, 
asserting bloom and associated Hudi min/max record-key metadata presence.
   
   ### Impact
   
   Native Parquet/ORC logs omit the unused bloom filter and its associated Hudi 
min/max record-key metadata. Physical column statistics and required native-log 
metadata remain unchanged. No new configuration is introduced.
   
   The local Parquet writer benchmark in the linked issue measured 8.5–36.1% 
lower write time and 1.1–38.1% smaller files across the tested workloads. These 
are local writer measurements, not end-to-end throughput; ORC was not 
benchmarked.
   
   ### Risk Level
   
   Low. The change is limited to native Parquet/ORC log paths; base-file bloom 
policies and HFile key-lookup bloom filters are preserved.
   
   Validation: 23 relevant tests passed with Java 11 (Spark 3.5 / Flink 1.20): 
Avro factory (4), ORC reader/writer (12), Spark factory (2), Flink factory (1), 
and Flink Parquet config injector (4). Checkstyle passed for the four affected 
writer/test modules. The full repository test suite was not run.
   
   ### Documentation Update
   
   Updated the existing Parquet bloom-filter configuration description to state 
that native Parquet logs do not write record-key bloom filters.
   
   ### Contributor's checklist
   
   - [ ] Read through [contributor's 
guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [x] Enough context is provided in the sections above
   - [x] Adequate tests were added if applicable
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to