sdf-jkl commented on issue #50622: URL: https://github.com/apache/arrow/issues/50622#issuecomment-5094693400
@HuaHuaY yes, that makes sense. I updated the proposed table in the issue description, the precision bounds now constrain both the Parquet and the Arrow columns. There is a related gap upstream: the shredding spec never states the decimal precision rules at all. The bands only appear in the [Decimal table in VariantEncoding.md](https://github.com/apache/parquet-format/blob/master/VariantEncoding.md#encoding-types), while the decimal rows in VariantShredding.md are unbounded DECIMAL(P, S). So the Arrow table has to carry the bounds explicitly, and VariantShredding.md could use the same clarification or a cross-reference. > So, is the final conclusion that we should update the Arrow documentation? How should we go about driving this forward? Update the Arrow doc and make an upstream Parquet PR to add decimal precision clarification/cross-reference to the shredding schema. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
