.
Am I correct in my expectations?
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/spark-sql-left-join-gives-KryoException-Buffer-overflow-tp10157p11432.html
Sent from the Apache Spark User List mailing list archive at Nabble.com
-spark-user-list.1001560.n3.nabble.com/spark-sql-left-join-gives-KryoException-Buffer-overflow-tp10157p11432.html
To unsubscribe from spark sql left join gives KryoException: Buffer overflow,
click here.
NAML
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com
-spark-user-list.1001560.n3.nabble.com/spark-sql-left-join-gives-KryoException-Buffer-overflow-tp10157p11432.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: user-unsubscr
Hi Michael,
Thanks for the suggestion. In my query, both table are too large to use
broadcast join.
When SPARK-2211 is done, will spark sql automatically choose join
algorithms?
Is there some way to manually hint the optimizer?
2014-07-19 5:23 GMT+08:00 Michael Armbrust mich...@databricks.com:
When SPARK-2211 is done, will spark sql automatically choose join
algorithms?
Is there some way to manually hint the optimizer?
Ideally we will select the best algorithm for you. We are also considering
ways to allow the user to hint.
Hi,
We have a query with left joining and got this error:
Caused by: org.apache.spark.SparkException: Job aborted due to stage
failure: Task 1.0:0 failed 4 times, most recent failure: Exception failure
in TID 5 on host ip-10-33-132-101.us-west-2.compute.internal:
Unfortunately, this is a query where we just don't have an efficiently
implementation yet. You might try switching the table order.
Here is the JIRA for doing something more efficient:
https://issues.apache.org/jira/browse/SPARK-2212
On Fri, Jul 18, 2014 at 7:05 AM, Pei-Lun Lee
