[ https://issues.apache.org/jira/browse/SPARK-22220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16197899#comment-16197899 ]
Dian Fay commented on SPARK-22220: ---------------------------------- I did find a slightly more detailed mention in the Zeppelin logs, but it's no stack trace: WARN [2017-10-09 23:28:39,445] ({pool-2-thread-32} NotebookServer.java[afterStatusChange]:2058) - Job 20171006-205907_443943867 is finished, status: ERROR, exception: null, result: %text org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 549.0 failed 4 times, most recent failure: Lost task 0.3 in stage 549.0 (TID 28218, wsdemo-dash03.dequecloud.com, executor 1): java.lang.NullPointerException > Spark SQL: LATERAL VIEW OUTER null pointer exception with GROUP BY > ------------------------------------------------------------------ > > Key: SPARK-22220 > URL: https://issues.apache.org/jira/browse/SPARK-22220 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.1.1 > Environment: We have Zeppelin using Spark and Livy (error is > reproducible without Livy) on an Ambari cluster. > Reporter: Dian Fay > > Given a DataFrame having the fields name (a string) and tags (an array of > strings), the following Spark SQL query fails with a NullPointerException: > {code} > SELECT name, tag, COUNT(*) > FROM records > LATERAL VIEW OUTER explode(tags) AS tag > GROUP BY name, tag > {code} > {code} > org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in > stage 137.0 failed 4 times, most recent failure: Lost task 0.3 in stage 137.0 > (TID 9109, $hostname, executor 1): java.lang.NullPointerException > {code} > The query is successful without the "outer", but obviously this excludes rows > with empty tags arrays. A version with outer but without aggregation also > succeeds, making it possible to work around this issue with a subquery: > {code} > SELECT name, tag > FROM records > LATERAL VIEW OUTER explode(tags) AS tag > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org