Hi Serge, Costas, and everyone, Thank you for bringing this proposal forward. We strongly agree that decimal floating point is an important missing type in Parquet. It matters for financial, regulatory, scientific, and cross-database workloads, where a shared representation can simplify implementation, validation, auditing, and certification.
Our earlier sketch <https://docs.google.com/document/d/1j_Q6vnn6Nhy60K4o0tdC91kE5vKGNJaoDOAm71KLzNw/edit?usp=sharing> was under-specified. Rather than defend that exact layout, we would like to make the requirements explicit and develop a precise, open format around them. After evaluating the IEEE decimal formats, we do not believe BID is the right foundation for a database-oriented Parquet type because it does not meet three essential requirements: 1. Lossless compatibility with the 38-digit SQL numeric ecosystem. 2. One cohort-canonical encoding for each finite numerical value. 3. Efficient execution of common and frequent database operations. 1. The type must cover 38 digits Precision 38 is the common boundary across SQL Server, Snowflake, Spark DecimalType, Arrow Decimal128, Iceberg decimal, Trino DECIMAL, DuckDB, and deployed Parquet DECIMAL(38,s) data. A decimal-floating interchange type for this ecosystem should represent that domain without rounding or loss. Decimal128 provides only 34 digits. Under the proposed generalized layout, the first sufficient width is decimal160: decimal128: 16 bytes, 34 digits decimal160: 20 bytes, 43 digits proposed: 18 bytes, 38 digits Decimal160 may follow the generalized layout mathematically, but it has no established implementation ecosystem. The Intel and Boost libraries cited in the proposal stop at decimal128, and we are not aware of an interoperable decimal160 codec in production use. The main advantage of choosing IEEE—reusing existing formats and libraries—therefore ends below the precision Parquet needs. At 160 bits, implementations must build a new codec anyway. A signed 128-bit significand represents every 38-digit value and matches the representation already used by much of the analytical stack: an integer significand plus scale or exponent. Adding a per-value exponent is a direct extension of that model. With an approximately two-byte exponent, the representation also covers the finite decimal64 and decimal128 exponent ranges. We would deliberately not inherit decimal160's much larger exponent range; SQL interoperability needs 38 digits far more than it needs that additional range. IEEE decimal64 and decimal128 remain useful where systems already use them. Their adoption across databases, programming languages, and general-purpose decimal libraries is limited, however, and decimal160 has essentially none. For most engines, BID would be a boundary codec rather than a native execution representation. 2. Values must be cohort-canonical IEEE decimal permits multiple encodings of the same numerical value: 1 × 10^0 10 × 10^-1 100 × 10^-2 Preserving the writer's cohort member conflicts with database numerical equality and with Parquet features defined over encoded bytes. Parquet Bloom filters hash the PLAIN-encoded value. A filter containing 100 × 10^-2 can therefore report that 1 × 10^0 is absent even though the values compare numerically equal. If that result is used for row-group pruning, the query can return an incorrect result. Avoiding this requires either a unique stored representation, a new logical hash for Bloom filters, or disabling Bloom-filter pruning for decimal-floating equality. A cohort-canonical representation is the simplest and most generally useful solution: remove trailing decimal zeros from every nonzero significand while the exponent can be increased, and define one encoding for zero. This also gives stable equality, hashing, and dictionary behavior. It intentionally does not preserve per-value quantum or representation identity, which are not part of ordinary SQL numeric equality. BID's non-canonical significands, which readers must interpret as zero, add further representation redundancy and decoder obligations without improving database interchange. Parquet should require writers to emit, and readers to operate on, one canonical finite representation. 3. BID adds cost to the hot path BID operations must decode the combination field, handle alternate significand layouts, validate canonicality, and repack results. Preferred-exponent and cohort semantics add further bookkeeping. A separate integer significand and exponent avoids this work and aligns directly with how many engines already execute decimal arithmetic. Our internal measurements on AWS Graviton3, comparing Intel's BID128 implementation with our optimized significand-and-exponent representation, showed: - 6x faster addition and up to 20x faster sums and reductions. - 28x faster multiplication. - 4x to 7x faster equality comparisons, including measured non-default cases. We expect the optimized default path to cover more than 95% of operations. Complex cases fall back to a general decimal library and approach normal library performance, because arithmetic rather than representation conversion then dominates. These are internal results from one platform and implementation, but they reinforce the structural concern: BID places recurring encode/decode work directly on common database operations. Recommended direction Parquet should support the useful database semantics—finite values, positive and negative infinity, and a canonical NaN—without requiring every implementation to preserve execution-level IEEE details such as signaling behavior, NaN payloads, preferred exponents, arbitrary cohort identity, or non-canonical encodings. We recommend a database-oriented representation with: - A 16-byte signed significand supporting 38 decimal digits. - A compact per-value decimal exponent of approximately two bytes. - A required cohort-canonical representation for every finite value. - Special values represented by selected patterns in the unused 128-bit significand range, allowing inexpensive classification. This suggests an approximately 18-byte fixed-width value. The exact encoding remains to be specified openly and unambiguously. Decimal64 and decimal128 cannot represent the existing 38-digit SQL domain. Decimal160 has no implementation ecosystem and retains BID's redundant encodings and execution overhead. For these reasons, we recommend that Parquet standardize a cohort-canonical, 38-digit significand-and-exponent representation rather than BID decimal64, decimal128, or decimal160. Best, Thomas From: "serge rielau.com" <[email protected]> > To: "[email protected]" <[email protected]> > Cc: > Bcc: > Date: Wed, 29 Jul 2026 17:16:10 +0000 > Subject: Proposal: Extensible Decimal Floating-Point Type > Hi folks, > > We’re interested in supporting a one-and-done extensible decimal floating > point data type. > The proposal follows in the footsteps of the recent TIMESTAMP(unit) > proposal to parameterize the number of significant digits. > This approach accommodates storage compactness, as well as handle current > and future needs by existing and new engines. > We base our proposal on IEEE 754 to exploit an existing, well thought out, > layout. > > If you are interested, please have a look and leave comments in the > document. > > Cheers > Serge Rielau > > [image: > AHkbwyJwbrQe-2FQdDaZUBu85mluiT3sRcwA9xtlo8S3ablbMxbDKTyQbajeAP5uDI5Fcdtqby0BSvSl4Kf3Lqp8uq-D8u916vWMOaxz82bWPttxXngW8sg=w1200-h630-p.png] > > Extensible Decimal Floating-Point Type for Parquet > <https://docs.google.com/document/d/104397AVUqg_JSlzGBdpa3D98X6Dd-RONIgABn3omBcw/edit?usp=sharing> > docs.google.com > <https://docs.google.com/document/d/104397AVUqg_JSlzGBdpa3D98X6Dd-RONIgABn3omBcw/edit?usp=sharing> > > <https://docs.google.com/document/d/104397AVUqg_JSlzGBdpa3D98X6Dd-RONIgABn3omBcw/edit?usp=sharing> > > -- THOMAS KISSINGER Staff Software Engineer MOBILE +49 174-2195270 EMAIL [email protected] Snowflake Inc. 135 Constitution Drive Menlo Park, CA 94025, USA
