Hi Hao, I'm using inner join as Broadcast join didn't work for left joins (thanks for the links for the latest improvements).
And I'm using HiveConext and it worked in a previous build (10/12) when joining 15 dimension tables. Jianshi On Thu, Nov 27, 2014 at 8:35 AM, Cheng, Hao <hao.ch...@intel.com> wrote: > Are all of your join keys the same? and I guess the join type are all > “Left” join, https://github.com/apache/spark/pull/3362 probably is what > you need. > > > > And, SparkSQL doesn’t support the multiway-join (and multiway-broadcast > join) currently, https://github.com/apache/spark/pull/3270 should be > another optimization for this. > > > > > > *From:* Jianshi Huang [mailto:jianshi.hu...@gmail.com] > *Sent:* Wednesday, November 26, 2014 4:36 PM > *To:* user > *Subject:* Auto BroadcastJoin optimization failed in latest Spark > > > > Hi, > > > > I've confirmed that the latest Spark with either Hive 0.12 or 0.13.1 fails > optimizing auto broadcast join in my query. I have a query that joins a > huge fact table with 15 tiny dimension tables. > > > > I'm currently using an older version of Spark which was built on Oct. 12. > > > > Anyone else has met similar situation? > > > > -- > > Jianshi Huang > > LinkedIn: jianshi > Twitter: @jshuang > Github & Blog: http://huangjs.github.com/ > -- Jianshi Huang LinkedIn: jianshi Twitter: @jshuang Github & Blog: http://huangjs.github.com/
