shrirangmhalgi opened a new pull request, #58786: URL: https://github.com/apache/spark/pull/58786
### What changes were proposed in this pull request? Replace the hardcoded scale of 10 with `DecimalType.DEFAULT_SCALE` (18) in `OracleDialect.getCatalystType` for bare `NUMBER` columns (precision=0, scale=-127). ### Why are the changes needed? Oracle's bare `NUMBER` type (declared without explicit precision or scale) reports precision=0 and scale=-127 via JDBC `ResultSetMetaData`. Spark's `OracleDialect` maps these to `DecimalType(38, 10)` with a hardcoded scale of 10, causing **silent precision loss** for values with more than 10 decimal places. For example, `0.0000123456789123456` (17 decimal places) is silently truncated to `0.0000123457` - with no warning or error. The hardcoded 10 was an arbitrary pick from 2015 (PR #8780), explicitly labeled "sub-optimal" and "works in our domain for now" by the original author. `DecimalType.DEFAULT_SCALE` (18) is Spark's own standard fallback for unknown scale, already used by `TeradataDialect` via `SYSTEM_DEFAULT` for the same situation. No other JDBC dialect hardcodes 10. ### Does this PR introduce _any_ user-facing change? Yes. Bare `NUMBER` columns now map to `DecimalType(38, 18)` instead of `DecimalType(38, 10)`. This preserves up to 18 decimal places instead of 10, reducing silent precision loss. Columns with explicit `NUMBER(p,s)` are not affected. ### How was this patch tested? Updated existing `OracleDialect jdbc type mapping` test to expect `DEFAULT_SCALE` and added a new regression test that verifies bare `NUMBER` (precision=0), `FLOAT` (scale=-127), and explicit `NUMBER(10)` (unaffected). ### Was this patch authored or co-authored using generative AI tooling? Yes. Co-Authored using Claude-Opus 4.8 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
