[jira] [Created] (CALCITE-6733) Type inferred by coercion for comparisons with decimal is too narrow

[Mihai Budiu (Jira)]

Mihai Budiu created CALCITE-6733:
------------------------------------

             Summary: Type inferred by coercion for comparisons with decimal is 
too narrow
                 Key: CALCITE-6733
                 URL: https://issues.apache.org/jira/browse/CALCITE-6733
             Project: Calcite
          Issue Type: Bug
          Components: core
    Affects Versions: 1.38.0
            Reporter: Mihai Budiu


The following statement fails at runtime:
{code}
SELECT ASCII('8') >= ABS(1.1806236821);
{code}

with the exception:

{code}
java.lang.ArithmeticException: Value 56 cannot be represented as a DECIMAL(11, 
10)
        at 
org.apache.calcite.linq4j.tree.Primitive.checkOverflow(Primitive.java:410)
        at 
org.apache.calcite.linq4j.tree.Primitive.integerDecimalCast(Primitive.java:515)
{code}

I have introduced this bug when reworking the type coercion.
The cause is an incorrect derivation of precision/scale in the common type used 
for comparisons;

{code}
        RelDataType result =
            factory.createSqlType(type2.getSqlTypeName(),
                Math.max(type1.getPrecision(), type2.getPrecision()), 
type2.getScale());
{code}

This is wrong, because the precision is the total number of digits, and not the 
number of digits before the decimal point. The precision is derived is not 
sufficient to store the integer number.




--
This message was sent by Atlassian Jira
(v8.20.10#820010)