Please correct me if I'm missing something. At a glance, your statements look correct if I understand correctly. I guess it might be simply missed, but it sounds as pretty trivial one as only a line can be removed safely which won't affect anything. (filterNot should be retained even we remove the line to prevent extended rules to break this)
On Sun, Apr 12, 2020 at 9:54 PM Jacek Laskowski <ja...@japila.pl> wrote: > Hi, > > I'm curious why there is a need to include InferFiltersFromConstraints > logical optimization in operatorOptimizationRuleSet value [1] that seems to > be only to exclude it later [2]? > > In other words, I think that simply removing InferFiltersFromConstraints > from operatorOptimizationRuleSet value [1] would make no change (except > removing a "dead code"). > > Does this make sense? Could I be missing something? > > [1] > https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala#L80 > [2] > https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala#L115 > > Pozdrawiam, > Jacek Laskowski > ---- > https://about.me/JacekLaskowski > "The Internals Of" Online Books <https://books.japila.pl/> > Follow me on https://twitter.com/jaceklaskowski > > <https://twitter.com/jaceklaskowski> >
