Github user dvogelbacher commented on the issue: https://github.com/apache/spark/pull/21993 @gatorsmile yes, I found that workaround. Very useful :) I think it would still be good to handle this better by default. I can see that introducing such an arbitrary configuration param doesn't seem optimal and am open to better suggestions. Not sure if blacklisting case-when statements outright is the right way to go. That could have negative perf impacts as well? And it wouldn't handle the case in the unit test where we have exponential growth when adding/subtracting columns (though that example might be somewhat contrived). Maybe we should just not collapse if the number of leaf expressions in the collapsed project is higher than the sum of the number of leaf expressions in the original statements?
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org