Solve it by remove lazy identity.
2.HiveContext.sql("cache table feature as "select * from src where ...)
which result size is only 100K
---------- Forwarded message ----------
From: Jone Zhang <joyoungzh...@gmail.com>
Date: 2017-05-10 19:10 GMT+08:00
Subject: Why spark.sql.autoBroadcastJoinThreshold not available
To: "user @spark/'user @spark'/spark users/user@spark" <
u...@spark.apache.org>
Now i use spark1.6.0 in java
I wish the following sql to be executed in BroadcastJoin way
*select * from sample join feature*
This is my step
1.set spark.sql.autoBroadcastJoinThreshold=100M
2.HiveContext.sql("cache lazy table feature as "select * from src where
...) which result size is only 100K
3.HiveContext.sql("select * from sample join feature")
Why the join is SortMergeJoin?
Grateful for any idea!
Thanks.
