cloud-fan commented on issue #27627:
URL: https://github.com/apache/spark/pull/27627#issuecomment-618337440
Then why can't we use a simple `isEmpty` flag? Null is skipped so the
`isEmpty` flag is true if the data are all nulls.
cloud-fan commented on issue #27627: [WIP][SPARK-28067][SQL] Fix incorrect
results for decimal aggregate sum by returning null on decimal overflow
URL: https://github.com/apache/spark/pull/27627#issuecomment-615065652
@skambha check the result in PostgreSQL:
```
cloud0fan=# create
cloud-fan commented on issue #27627: [WIP][SPARK-28067][SQL] Fix incorrect
results for decimal aggregate sum by returning null on decimal overflow
URL: https://github.com/apache/spark/pull/27627#issuecomment-613912451
> So if we have 2 null rows, the expected result should be null and not
cloud-fan commented on issue #27627: [WIP][SPARK-28067][SQL] Fix incorrect
results for decimal aggregate sum by returning null on decimal overflow
URL: https://github.com/apache/spark/pull/27627#issuecomment-611308104
> A function implementor would need to know to use DecimalSum instead of
cloud-fan commented on issue #27627: [WIP][SPARK-28067][SQL] Fix incorrect
results for decimal aggregate sum by returning null on decimal overflow
URL: https://github.com/apache/spark/pull/27627#issuecomment-598002388
> Is the expectation that the user would now call this from their
cloud-fan commented on issue #27627: [WIP][SPARK-28067][SQL] Fix incorrect
results for decimal aggregate sum by returning null on decimal overflow
URL: https://github.com/apache/spark/pull/27627#issuecomment-597440834
@skambha great analysis!
I agree with you that we need another
cloud-fan commented on issue #27627: [WIP][SPARK-28067][SQL] Fix incorrect
results for decimal aggregate sum by returning null on decimal overflow
URL: https://github.com/apache/spark/pull/27627#issuecomment-596918804
@skambha thanks for your investigation! Very helpful.
I'm trying
cloud-fan commented on issue #27627: [WIP][SPARK-28067][SQL] Fix incorrect
results for decimal aggregate sum by returning null on decimal overflow
URL: https://github.com/apache/spark/pull/27627#issuecomment-595600181
> Addition of two decimal values ( expression coming from sum) that
cloud-fan commented on issue #27627: [WIP][SPARK-28067][SQL] Fix incorrect
results for decimal aggregate sum by returning null on decimal overflow
URL: https://github.com/apache/spark/pull/27627#issuecomment-595330926
Note: the semantic of most aggregate functions are skipping nulls. So
cloud-fan commented on issue #27627: [WIP][SPARK-28067][SQL] Fix incorrect
results for decimal aggregate sum by returning null on decimal overflow
URL: https://github.com/apache/spark/pull/27627#issuecomment-595319742
> looks like the overflow happens during partial aggregate (
cloud-fan commented on issue #27627: [WIP][SPARK-28067][SQL] Fix incorrect
results for decimal aggregate sum by returning null on decimal overflow
URL: https://github.com/apache/spark/pull/27627#issuecomment-595112843
Ideally, under ANSI mode, decimal sum should also fail if overflows.
cloud-fan commented on issue #27627: [WIP][SPARK-28067][SQL] Fix incorrect
results for decimal aggregate sum by returning null on decimal overflow
URL: https://github.com/apache/spark/pull/27627#issuecomment-595067684
cc @viirya @maropu as well
cloud-fan commented on issue #27627: [WIP][SPARK-28067][SQL] Fix incorrect
results for decimal aggregate sum by returning null on decimal overflow
URL: https://github.com/apache/spark/pull/27627#issuecomment-595067541
find a way to reproduce without join
```
scala> val decimalStr =
cloud-fan commented on issue #27627: [WIP][SPARK-28067][SQL] Fix incorrect
results for decimal aggregate sum by returning null on decimal overflow
URL: https://github.com/apache/spark/pull/27627#issuecomment-594644908
I tried some queries locally but can't reproduce. Is it only a problem
cloud-fan commented on issue #27627: [WIP][SPARK-28067][SQL] Fix incorrect
results for decimal aggregate sum by returning null on decimal overflow
URL: https://github.com/apache/spark/pull/27627#issuecomment-594010285
> In case of the whole stage codegen, you can see the decimal ‘+’
cloud-fan commented on issue #27627: [WIP][SPARK-28067][SQL] Fix incorrect
results for decimal aggregate sum by returning null on decimal overflow
URL: https://github.com/apache/spark/pull/27627#issuecomment-593977066
OK to test
cloud-fan commented on issue #27627: [WIP][SPARK-28067][SQL] Fix incorrect
results for decimal aggregate sum by returning null on decimal overflow
URL: https://github.com/apache/spark/pull/27627#issuecomment-589530921
> Sum does not take care of possibility of overflow for the intermediate
17 matches
Mail list logo
