hi , I write a hive like :
hive -e "
set hive.exec.compress.output=true;
set mapred.output.compression.codec=org.apache.hadoop.io.compress.GzipCodec;
set hive.merge.mapredfiles=true;
set mapred.job.name=my test job;
INSERT OVERWRITE DIRECTORY '/user/test/$year/$month/$day/'
select *
from
    test.table
where
    hostname = 'localhost'
    and substr(request_uri, 0, 4) = '/test/'
);"

I run this hive every day and when it run  it is splited into to two map
reduce job.
But this job failed sometimes, with below errors:

java.lang.RuntimeException: Hive Runtime Error while closing operators
        at org.apache.hadoop.hive.ql.exec.ExecMapper.close(ExecMapper.java:226)
        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
        at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
        at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to
rename output from:
hdfs://dbbak10-002:8020/tmp/hive-hadoop/hive_2013-02-17_00-10-09_598_42232731534306869/_task_tmp.-ext-10001/_tmp.000002_0
to: 
hdfs://dbbak10-002:8020/tmp/hive-hadoop/hive_2013-02-17_00-10-09_598_42232731534306869/_tmp.-ext-10001/000002_0.gz
        at 
org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.commit(FileSinkOperator.java:199)
        at 
org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.access$300(FileSinkOperator.java:101)
        at 
org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:718)
        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:557)
        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566)
        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566)
        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566)
        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566)
        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566)
        at org.apache.hadoop.hive.ql.exec.ExecMapper.close(ExecMapper.java:193)
        ... 8 more

And this error was happeded accidentally.

I don't know why, If who know, please tell me , thank you very much.

Reply via email to