Github user mallman commented on the issue:
https://github.com/apache/spark/pull/21320
>> Hello, we've been using your patch at Stripe and we've found something
that looks like a new bug:
>
> Thank you for sharing this, @xinxin-stripe. This is very helpful. I will
investigate and report back.
I have not been able to reproduce this issue with this branch at commit
0e5594b6ac1dcb94f3f0166e66a7d4e7eae3d00c. However, I'm seeing the same failure
scenario as yours on VideoAmp's internal 2.1, 2.2 and 2.3 backports of this
branch. I think the reason for this difference is that our internal branches
(and probably yours) incorporate rules to support pruning for aggregations.
That functionality was removed from this PR.
I will fix this and share the fix with you. It would help if you could send
me a scenario where you can reproduce this failure with a Spark SQL query.
Query plans for datasets built from SQL queries tend to be much more readable.
Consider e-mailing me directly on this issue because it does not appear to
be strictly related to this PR. My e-mail address is m...@allman.ms. Thanks
again!
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
