hadoop/hive data loading

[hadoopman]

When we load data into hive sometimes we've run into situations where 
the load fails and the logs show a heap out of memory error.  If I load 
just a few days (or months) of data then no problem.  But then if I try 
to load two years (for example) of data then I've seen it fail.  Not 
with every feed but certain ones.

Sometimes I've been able to split the data and get it to load.  An 
example of one type of feed I'm working on is the apache web server 
access logs.  Generally it works.  But there are times when I need to 
load more than a few months of data and get the memory heap errors in 
the task logs.

Generally how do people load their data into Hive?  We have a process 
where we first copy it to hdfs then from there we run a staging process 
to get it into hive.  Once that completes we perform a union all then 
overwrite table partition.  Usually it's during the union all stage that 
we see these errors appear.

Also is there a log which tells you which log it fails on?  I can see 
which task/job failed but not finding which file it's complaining 
about.  I figure that might help a bit..

Thanks!