dciborow created SPARK-15005:
--------------------------------
Summary: Usage of Temp Table twice in Hive query fails with bad
error
Key: SPARK-15005
URL: https://issues.apache.org/jira/browse/SPARK-15005
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 1.6.0
Reporter: dciborow
Priority: Minor
When converting a Hive ETL process from Hive to Spark, adjustments might be
made to the query. One adjustment is that the Hive query might query from the
same table more then once in an join the results together. When Spark tries to
process this query it provides an very poor error message, that does not help
the user determine what has gone wrong. It should be simple to detect this, and
properly report it to the user.
Sample Query that contains the error(edited for this post so might not run)
SELECT
| enc.id
| enc.name,
| enc.sum
| FROM
| (
| SELECT
| *
| FROM
| table1
| JOIN
| (
| SELECT
| id,
| SUM(impressions) AS
| sum_impressions,
| FROM
| table1 enc
| GROUP BY
| enc.id) enc1
| ON
| (
| enc.id = enc1.id)
Error Message(had to edit to remove a bunch of field names, but tried to leave
everything I could)
16/04/28 15:47:09 INFO ParseDriver: Parse Completed
org.apache.spark.sql.AnalysisException: resolved attribute(s) [_id#3372,],
[HiveWindowFunction#org.apache.hadoop.hive.ql.udf.generic.GenericUDAFSum(unique_audience#3380)
windowspecdefinition(id#3372,ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED
FOLLOWING) AS
_we0#530,HiveWindowFunction#org.apache.hadoop.hive.ql.udf.generic.GenericUDAFSum(total_impressions#3382)
windowspecdefinition(id#3372,,ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED
FOLLOWING) AS _we1#531], [id#3372,];
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
