Spark 1.6 Catalyst optimizer

[Telmo Rodrigues]

Hello,

I have a question about the Catalyst optimizer in Spark 1.6.

initial logical plan:

!'Project [unresolvedalias(*)]
!+- 'Filter ('t.id = 1)
!   +- 'Join Inner, Some(('t.id = 'u.id))
!      :- 'UnresolvedRelation `t`, None
!      +- 'UnresolvedRelation `u`, None


logical plan after optimizer execution:

Project [id#0L,id#1L]
!+- Filter (id#0L = cast(1 as bigint))
!   +- Join Inner, Some((id#0L = id#1L))
!      :- Subquery t
!          :  +- Relation[id#0L] JSONRelation
!      +- Subquery u
!          +- Relation[id#1L] JSONRelation


Shouldn't the optimizer push down predicates to subquery t in order to the
filter be executed before join?

Thanks