d33bs opened a new pull request, #609: URL: https://github.com/apache/parquet-format/pull/609
### Rationale for this change The dictionary encoding docs currently say that writers fall back to PLAIN encoding when a dictionary gets too large. That is too narrow. Writers can stop using dictionary encoding and write later data pages with another valid encoding. The actual encoding is already stored in each data page header, so readers should use that field instead of assuming that fallback pages are PLAIN. ### What changes are included in this PR? This PR updates `Encodings.md` to clarify that dictionary fallback encoding is chosen by the writer and recorded in the data page header. It also clarifies that the documented dictionary data page layout applies only to dictionary-encoded data pages. ### Do these changes have PoC implementations? I feel no PoC implementation is needed, but please let me know if you think otherwise. This is a documentation clarification for behavior that the format already supports. Closes #404 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
