Hi Ashish, I recently upgraded to Hadoop 0.20.1 and also am now feeding log data in compressed gzip form to a hive sequence table, and I haven't seen the errors any more.
Thanks! Ryan On Mon, Oct 12, 2009 at 4:44 PM, Ashish Thusoo <[email protected]> wrote: > This is not expected. Can you send the query and the explain plan for the > query. > > Ashish > > ------------------------------ > *From:* Ryan LeCompte [mailto:[email protected]] > *Sent:* Sunday, October 11, 2009 4:02 PM > *To:* [email protected] > *Subject:* Failures during map/reduce when running a hive query > > Hello all, > > My hive queries are returning back successfully, however if I did in the > map/reduce job that's running as a result of running the query, I see the > following errors in a lot of map tasks that get killed: > > java.lang.RuntimeException: Map operator initialization failed > at > org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:110) > at > org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58) > > at > org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:83) > at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34) > at > org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58) > > at > org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:83) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:328) > at org.apache.hadoop.mapred.Child.main(Child.java:155) > Caused by: java.lang.NullPointerException > > at > org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:207) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:316) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:353) > > at > org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:338) > at > org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:59) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:316) > > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:353) > at > org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:338) > at > org.apache.hadoop.hive.ql.exec.Operator.initializeOp(Operator.java:324) > > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:316) > at > org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:301) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:316) > > at > org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:82) > > ... 7 more > > > The reducers also fail at times with similar exceptions/errors. > > Is this expected, or do I have something misconfigured? I don't have the > hive distribution on all of my hadoop nodes, just the single node where I > run the queries. I am using 0.4-rc2. > > Thanks, > Ryan > >
