It does push the predicate. But as a relations are generic and might or might not handle some of the predicates , it needs to apply filter of un-handled predicates.
Regards, Rishitesh Mishra, SnappyData . (http://www.snappydata.io/) https://in.linkedin.com/in/rishiteshmishra On Wed, May 11, 2016 at 6:27 AM, Telmo Rodrigues < telmo.galante.rodrig...@gmail.com> wrote: > Hello, > > I have a question about the Catalyst optimizer in Spark 1.6. > > initial logical plan: > > !'Project [unresolvedalias(*)] > !+- 'Filter ('t.id = 1) > ! +- 'Join Inner, Some(('t.id = 'u.id)) > ! :- 'UnresolvedRelation `t`, None > ! +- 'UnresolvedRelation `u`, None > > > logical plan after optimizer execution: > > Project [id#0L,id#1L] > !+- Filter (id#0L = cast(1 as bigint)) > ! +- Join Inner, Some((id#0L = id#1L)) > ! :- Subquery t > ! : +- Relation[id#0L] JSONRelation > ! +- Subquery u > ! +- Relation[id#1L] JSONRelation > > > Shouldn't the optimizer push down predicates to subquery t in order to the > filter be executed before join? > > Thanks > > >
