I usehive 1.1.0 + tez0.53 set hive.mapjoin.smalltable.filesize = 60000000; set hive.auto.convert.join=true; set hive.auto.convert.join.noconditionaltask=true set hive.auto.convert.join.noconditionaltask.size=10000000
[email protected] From: Vikram Dixit Date: 2015-05-13 08:47 To: user Subject: Re: hive on tez not convert map join to broadcast join Hi, Can you share your configuration values for the following when you run in tez: hive.auto.convert.join hive.auto.convert.join.noconditionaltask hive.auto.convert.join.noconditionaltask.size What are the sizes of the tables? Stored in which format? Size of the tables and size of the tables on disc? Thanks Vikram. From: "[email protected]" <[email protected]> Reply-To: user <[email protected]> Date: Tuesday, May 12, 2015 at 5:42 PM To: user <[email protected]> Subject: hive on tez not convert map join to broadcast join In MR query plan is Map Join Operator condition map: Left Outer Join0 to 1 keys: 0 ordr_code (type: string), cart_prod_id (type: bigint) 1 parnt_ordr_code (type: string), comb_prod_id (type: bigint) outputColumnNames: _col1, _col2, _col3, _col5, _col10, _col11, _col15, _col16, But in tez Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE) No broadcast edge Reducer 3 <- Map 5 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE) Merge Join Operator condition map: Left Outer Join0 to 1 keys: 0 ordr_code (type: string), cart_prod_id (type: bigint) 1 parnt_ordr_code (type: string), comb_prod_id (type: bigint) [email protected]
mrplan.TXT
Description: Binary data
tezplan.TXT
Description: Binary data
