Hey guys,
I'm trying to save Dataframe in CSV format after performing unionAll
operations on it.
But I get this exception -
Exception in thread "main"
org.apache.spark.sql.catalyst.errors.package$TreeNodeException: execute,
tree:
TungstenExchange hashpartitioning(mId#430,200)
I'm saving it by
df.write.format("com.databricks.spark.csv").options(Map("mode" ->
"DROPMALFORMED", "delimiter" -> "\t", "header" -> "true")).save(bakDir +
latest)
It works perfectly if I don't do the unionAll operation.
I see that the format isn't different by printing the part of the results.
Any help regarding this would be appreciated.
