Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/21599
@cloud-fan @gatorsmile the main issue which is causing the UT failures,
now, is that since before we were allowing overflows, in aggregations we could
have an overflow eventually fixed by another overflow on the opposite direction
(see the two UT failures in the Range suite).
The problem here is that we can know whether an overflow occurs only
checking at the incoming operands, so we cannot defer after the whole
aggregation is performed to check if the overflow occurred or not. The way
MySQL deals with this case is returning a `DECIMAL` as result of the
aggregation (it the starting datatype is BIGINT, then a `DECIMAL(41, 0)`).
Probably we can do something similar, but that would be a quite huge behavior
change and I am not sure it is acceptable in a minor release.
Therefore, I'd like to know your thoughts about this, since on one side it
is a pretty serious bug which should be fixed, on the other I cannot think of
any solution which doesn't involve behavior changes.
Thank you.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
