Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21537
AnalysisBarrier does not introduce a behavior change. However, this
requires our analyzer rules must be idempotent. The most recent correctness bug
also shows another big potential hole
https://issues.apache.org/jira/browse/SPARK-25051. It could break Analyzer
rules without notice, since AnalysisBarrier is a leaf node. Basically, this is
a disaster for Spark 2.3 release. You can count how many regressions we found
after the release. This is bad. Unfortunately, I made the merge without enough
consideration. If possible, I would revert that PR, but it is too late.
Now, we are facing the similar issue again. We should stop continuing this
direction without a proper design and review. Designing/enhancing the codegen
framework requires more inputs from the experts in this area. I do not think
the current design is well discussed, even if I saw some discussions in the
initial PR. I am not the expert in this area. That is why I pinged @kiszk to
drive this.
BTW, the internal APIs we can change easily, but the design need a review
especially for the core components of Spark SQL.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
