Re: [SPARK-24865] Remove AnalysisBarrier

2018-07-19 Thread Reynold Xin
A work-in-progress PR: https://github.com/apache/spark/pull/21822 The PR also adds the infrastructure to throw exceptions in test mode when the various transform methods are used as part of analysis. Unfortunately there are couple edge cases that do need that, and as a result there is this ugly by

[SPARK-24865] Remove AnalysisBarrier

2018-07-19 Thread Reynold Xin
We have had multiple bugs introduced by AnalysisBarrier. In hindsight I think the original design before analysis barrier was much simpler and requires less developer knowledge of the infrastructure. As long as analysis barrier is there, developers writing various code in analyzer will have to be