Hi All, We have ORC table which is of 2 GB size. When we perform operation on top of this ORC table, Tez always deduce 1009 reducer every time. I searched 1009 is considered as Maximum value of number of Tez task. Is there a way to reduce the number of reducer. I see file generated underlying ORC some of them 500 MB or 1 GB etc. Is there way to distribute file size to same value/same size.
My Second scenario, we have join on 5 tables all of them are left join. Query goes fast till reached 99%. From 99% to 100% it takes too much time. We are not involving our partition column as part of LEFT JOIN Statement, Is there better way to resolving issues on 99% hanging condition. My table is of 20 GB we are left joining with another table ( 9,00,00,000) records. Mahens
