[ https://issues.apache.org/jira/browse/SPARK-10623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yin Huai resolved SPARK-10623. ------------------------------ Resolution: Fixed Fix Version/s: 1.5.1 1.6.0 Issue resolved by pull request 8799 [https://github.com/apache/spark/pull/8799] > NoSuchElementException thrown when ORC predicate push-down is turned on > ----------------------------------------------------------------------- > > Key: SPARK-10623 > URL: https://issues.apache.org/jira/browse/SPARK-10623 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 1.4.0, 1.4.1, 1.5.0 > Reporter: Ram Sriharsha > Assignee: Zhan Zhang > Priority: Blocker > Fix For: 1.6.0, 1.5.1 > > > Turning on predicate pushdown for ORC datasources results in a > {{NoSuchElementException}}: > {noformat} > scala> val df = sqlContext.sql("SELECT name FROM people WHERE age < 15") > df: org.apache.spark.sql.DataFrame = [name: string] > scala> sqlContext.setConf("spark.sql.orc.filterPushdown", "true") > scala> df.explain > == Physical Plan == > java.util.NoSuchElementException > {noformat} > Disabling the pushdown makes things work again: > {noformat} > scala> sqlContext.setConf("spark.sql.orc.filterPushdown", "false") > scala> df.explain > == Physical Plan == > Project [name#6] > Filter (age#7 < 15) > Scan > OrcRelation[file:/home/mydir/spark-1.5.0-SNAPSHOT/test/people][name#6,age#7] > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org