DarkWanderer opened a new pull request, #12321:
URL: https://github.com/apache/arrow-rs/pull/12321

   # Which issue does this PR close?
   
   - None filed. Found while benchmarking dictionary-based row group pruning in 
apache/datafusion#23851.
   
   # Rationale for this change
   
   `ParquetRecordBatchStreamBuilder::get_column_chunk_dictionary`, 
`ParquetMetaDataReader::read_column_dictionary` and 
`ParquetMetaDataReader::read_column_dictionary_async` fail with `Dictionary 
page encoding must be PLAIN, got PLAIN_DICTIONARY` when a dictionary page 
header declares the deprecated `PLAIN_DICTIONARY` encoding.
   
   Older writers label dictionary pages that way, for example parquet-cpp 1.x, 
which wrote the ClickBench `hits` files. The array readers already decode such 
pages as `PLAIN`, so the column data of these files reads fine, but their 
dictionaries cannot be read through these APIs.
   
   # What changes are included in this PR?
   
   `decode_dictionary_page` accepts `PLAIN_DICTIONARY` as well as `PLAIN` for 
the dictionary page encoding, as the array readers do. Other encodings are 
still rejected.
   
   # Are these changes tested?
   
   Yes. A unit test rewrites a dictionary page header to `PLAIN_DICTIONARY` and 
checks that the dictionary decodes to the expected values. The existing test 
that rejects `RLE_DICTIONARY` is unchanged.
   
   # Are there any significant user-facing changes?
   
   No.
   
   # AI usage disclosure
   
   The fix, the test and this description were written with Claude Code (Claude 
Opus 5.5).
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)
   
   https://claude.ai/code/session_012oZ19LDPpsfabaCZBZHoDu
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to