Hi all, I would like to start a discussion on the SPIP to add a new Spark SQL data type: DECFLOAT (IEEE 754 decimal64 / decimal128), for base-10 floating-point decimals with per-value exponents.
JIRA ID: https://issues.apache.org/jira/browse/SPARK-58820 Brief summary: DECFLOAT is a new IEEE 754 decimal floating-point type (decimal64 / decimal128, i.e. DECFLOAT(16) / DECFLOAT(34)) that closes the gap between DECIMAL, which caps at precision 38 with a fixed per-column scale, and DOUBLE, whose binary rounding makes fractions like 0.1 inexact: each value keeps its own exponent, arithmetic is decimal, and it supports signed zero, Inf, and NaN. It is additive, round-trips through a proposed Parquet logical type for cross-engine interop, and would ship config-gated during incubation like TIME, leaving existing DECIMAL / DOUBLE behavior unchanged. Additional information is available in the SPIP document: https://docs.google.com/document/d/1qnLXm0ldHSwPSJs_Q5SzGyyTKMMEqeX5rm8Fh4rvV3E Please provide your feedback on the approach, scope, and the proposal described in the document. Thank you! Best, Uroš --------------------------------------------------------------------- To unsubscribe e-mail: [email protected]
