(HIVE-1642) Convert join queries to map-join based on size of table/row

2010-12-23 Thread ashu99
Hi, I set hive.auto.convert.join=true and run the following query: select t1.foo, count(t2.bar) from invites t1 join invites t2 on (t1.foo=t2.foo) group by t1.foo; I did not see it ran as map side join. Did I miss something? Is there any precondition for this feature to work? Thanks.

Re: (HIVE-1642) Convert join queries to map-join based on size of table/row

2010-12-26 Thread ashu99
Thanks for the reply. I want to get clarification on this feature. If one of the two joining tables table t1 is smaller than 25M and is sharded, how does this feature work? Suppose there are joins on multiple tables such as t1, t2 and t3. If t1 and t2 are smaller than 25M and co-located with joi

Re: (HIVE-1642) Convert join queries to map-join based on size of table/row

2010-12-27 Thread ashu99
A question about the design doc: "If one of the tables is large and others are small enough to run Map Join, then the Conditional Task will pick the corresponding Map Join Local Task to run." Here you pick one table as big, hash all other tables into memory by join key individually. If it works, i

Re: (HIVE-1642) Convert join queries to map-join based on size of table/row

2010-12-28 Thread ashu99
Thanks! On Mon, Dec 27, 2010 at 11:56 PM, Liyin Tang wrote: > Yes. Only execute one of the them. > > On 27 December 2010 23:43, wrote: > > > A question about the design doc: > > > > "If one of the tables is large and others are small enough to run Map > Join, > > then the Conditional Task will