I’ve got a good one for y’all today. In our engine, we don’t care about precision, but we *do* care about scale. This is because we have mechanisms for handling arbitrarily wide decimals, but we at least require that all decimals in a column have the same scale. It is possible for us to express that in the Calcite type system?
This usually isn’t a problem but we’ve hit a pretty trivial case today, specifically: union over decimal producing inputs. It’s possible that the same columnar position is typed as decimal, with at least one of those inputs having a different scale than the others. We intended to handle this by inserting casts to the least restrictive type (which typically derives a decimal with the max scale, effectively padding out lower scale numbers), and this works for the simple cases. The problem is when one of those columns has the type system’s max precision and a lower scale. In this case, the least restrictive type is actually the *lowest* scale, highest precision type. This is easy to encounter with sum, since the default implementation of deriveSumType uses the type systems max precision. For example: select x from table union all select y from otherTable select x from table union all select sum(y) from otherTable (where x is scale 3 and y is scale 2) In the first case, the least restrictive type is decimal with scale 3, and we would cast y to that type, losing no information in the process. But in the second case, the sum(y) produces a type with the type systems max precision, and so there is no room for the scale to be padded out, and so the least restrictive type is a decimal with scale 2, and now x would be down-cast to the lower scale, which is undesirable for us. I’ve played around with setting the precision of our decimal columns or the max precision to PRECISION_NOT_SPECIFIED, but that seems to have the effect that scale is no longer respected or propagated through the type system. Our immediate plan is to override deriveSumType to simply propagate the operand type, but that’s an ugly hack (and one with limited effect to boot). Any better ideas? Thanks! -Ian J. Bertolacci
