Hi,
We have a complex query that involves several left outer joins resulting in
8 M/R jobs in Hive.During execution of one of the stages ( after three M/R
has run) the M/R job fails due to few Reduce tasks failing due to
inactivity.
Most of the reduce tasks go through fine ( within 3 mins) but
Just curious if you've tried using Hive's explain method to see what IT
thinks of your query.
On Fri, Aug 24, 2012 at 9:36 AM, Himanish Kushary himan...@gmail.comwrote:
Hi,
We have a complex query that involves several left outer joins resulting
in 8 M/R jobs in Hive.During execution of one
Why don't you try splitting the big query into smaller ones?
On Fri, Aug 24, 2012 at 10:20 AM, Tim Havens timhav...@gmail.com wrote:
Just curious if you've tried using Hive's explain method to see what IT
thinks of your query.
On Fri, Aug 24, 2012 at 9:36 AM, Himanish Kushary
It is not clear from your post but your job is always failing during the
same step? Or only sometimes? Or only once?
Since it's a hive query I would modify it to find the root cause.
First create temporary files which are the results from the three first
M/R.
Then run the fourth M/R on it and try
My two cents,
Try checking if there is a skew in the input to that reducer compared to
other reducers. This happens sometimes in joins where some reducers have
large amount of input data and keep running forever.
On Fri, Aug 24, 2012 at 11:41 PM, Bertrand Dechoux decho...@gmail.comwrote:
It
