Pierre Borckmans created SPARK-5454:
---------------------------------------
Summary: [SQL] Self join with ArrayType columns problems
Key: SPARK-5454
URL: https://issues.apache.org/jira/browse/SPARK-5454
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 1.2.0
Reporter: Pierre Borckmans
Weird behaviour when performing self join on a table with some ArrayType field.
(potential bug ?)
I have set up a minimal non working example here:
https://gist.github.com/pierre-borckmans/4853cd6d0b2f2388bf4f
In a nutshell, if the ArrayType column used for the pivot is created manually
in the StructType definition, everything works as expected.
However, if the ArrayType pivot column is obtained by a sql query (be it by
using a "array" wrapper, or using a collect_list operator for instance), then
results are completely off.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
