Yifan Cai created CASSANALYTICS-210:
---------------------------------------
Summary: Report decimal/varint values that exceed the configured
BigNumberConfig instead of silently returning NULL
Key: CASSANALYTICS-210
URL: https://issues.apache.org/jira/browse/CASSANALYTICS-210
Project: Apache Cassandra Analytics
Issue Type: Improvement
Components: Reader
Reporter: Yifan Cai
SparkDecimal and SparkVarInt declare Spark types from BigNumberConfig (default
Decimal(38,19) and Decimal(38,0)) and convert values with Decimal.apply without
a range check. A value that does not fit (for example 1e20 for a decimal) is
written as NULL by Spark, or rounded when the scale exceeds 19, with no error
or warning. BigNumberConfig can be set per column, so this is a documented
limit, but it fails silently. This change detects values that do not fit and
fails with an error naming the column, with an opt-out to keep the current
behavior.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
