Hi, I am trying to use predicate pushdown in ORC and I was expecting that it would be used when one tries to query an ORC table in Hive. But based on the query plan generated, I don't see that happening. I am missing some configurations or this is not supported at all.
PS: I have tried the following over 1.5.1 and even 1.6 release branch. val hiveContext = new org.apache.spark.sql.hive.HiveContext(sc) hiveContext.setConf("spark.sql.orc.filterPushdown", "true") // orc_table is a hive table val query = hiveContext.sql("SELECT COUNT(*) FROM orc_table WHERE key > 10") scala> query.explain == Physical Plan == TungstenAggregate(key=[], functions=[(count(1),mode=Final,isDistinct=false)], output=[_c0#41L]) TungstenExchange SinglePartition TungstenAggregate(key=[], functions=[(count(1),mode=Partial,isDistinct=false)], output=[currentCount#44L]) Project Filter (key#39 > 10) HiveTableScan [key#39], (MetastoreRelation default, orc_table, None) Thanks, Tejas