JFinis opened a new pull request, #633: URL: https://github.com/apache/parquet-format/pull/633
FINITE_VECTOR, as proposed in https://github.com/apache/parquet-format/pull/624 has two shortcomings that this PR is trying to avoid: * This PR does not tie finiteness to the list/vector type but instead to the contained float. This way, we can have finite floats that aren't vectors. They are useful by themselves, so having the finiteness annotation at the type to which it applies makes this more composable. * This PR does not introduce FINITE_VECTOR as a new type that basically mimics LIST with a fixed length. Instead, the LIST type itself receives an optional fixed length field. This way, legacy readers can still read vectors and simply interpret them as lists. Also, by decoupling finiteness from fixed length and not calling it vector, we gain general fixed-size lists for *all* types, instead of having just some blessed types be able to live in fixed-size lists / vectors. <!-- Thanks for opening a pull request! If you're new to Parquet-Format, information on how to contribute can be found here: https://parquet.apache.org/docs/contribution-guidelines/contributing Please open a GitHub issue for this pull request: https://github.com/apache/parquet-format/issues/new/choose and format pull request title as below: GH-${GITHUB_ISSUE_ID}: ${SUMMARY} or simply use the title below if it is a minor issue: MINOR: ${SUMMARY} --> ### Rationale for this change ### What changes are included in this PR? ### Do these changes have PoC implementations? <!-- Please uncomment the line below and replace ${GITHUB_ISSUE_ID} with the actual Github issue id. --> <!-- Closes #${GITHUB_ISSUE_ID} --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
