Re: Why spark.sql.autoBroadcastJoinThreshold not available

2017-05-15 Thread Jone Zhang
in the Spark 2.x. Can you try it on Spark 2.0? > > Yong > > -- > *From:* Jone Zhang <joyoungzh...@gmail.com> > *Sent:* Wednesday, May 10, 2017 7:10 AM > *To:* user @spark/'user @spark'/spark users/user@spark > *Subject:* Why spark.sql.autoBroadcastJ

Re: Why spark.sql.autoBroadcastJoinThreshold not available

2017-05-15 Thread Yong Zhang
broadcast join. This is fixed in the Spark 2.x. Can you try it on Spark 2.0? Yong From: Jone Zhang <joyoungzh...@gmail.com> Sent: Wednesday, May 10, 2017 7:10 AM To: user @spark/'user @spark'/spark users/user@spark Subject: Why spark.sql.autoBroadcastJoinT

Why spark.sql.autoBroadcastJoinThreshold not available

2017-05-10 Thread Jone Zhang
Now i use spark1.6.0 in java I wish the following sql to be executed in BroadcastJoin way *select * from sample join feature* This is my step 1.set spark.sql.autoBroadcastJoinThreshold=100M 2.HiveContext.sql("cache lazy table feature as "select * from src where ...) which result size is only 100K