Hello everyone! I would like to propose a new canonical extension type for BigDecimal (arbitrary-precision decimals). While Arrow supports fixed-precision Decimal128 and Decimal256, these types require a uniform scale for an entire column. Many modern data engines (PostgreSQL, Trino, Oracle) utilize numeric types that allow for variable scale per row. This flexibility is the primary driver for this proposal, as it enables the representation of data that cannot be shoehorned into a fixed-scale schema without loss of integrity or precision.
This proposal [1] aims to provide a high-fidelity transport format for these values to ensure data integrity when moving data between these systems and Arrow-native engines. I look forward to your feedback! Thank you. --Mandy [1]: https://docs.google.com/document/d/10YNDPW9068RMsvAV_pjmC-uuerwWEwdhGqJARyVvnWI/edit?usp=sharing
