[
https://issues.apache.org/jira/browse/HIVE-4216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Dere updated HIVE-4216:
-----------------------------
Status: Open (was: Patch Available)
Cancelling patch .. looks like TestHBaseCliDriver fails with the following
error:
2013-12-20 15:02:44,090 ERROR CliDriver (SessionState.java:printError(440)) -
Failed with exception java.io.IOException:java.lang.NullPointerException
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:570)
at
org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:508)
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:137)
at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1540)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:280)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:422)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:357)
at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:828)
at
org.apache.hadoop.hive.cli.TestHBaseCliDriver.runTest(TestHBaseCliDriver.java:108)
at
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_queries(TestHBaseCliDriver.java:90)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:243)
at junit.framework.TestSuite.run(TestSuite.java:238)
at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.extensions.TestSetup.run(TestSetup.java:27)
at
org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:264)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124)
at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.mapreduce.task.TaskAttemptContextImpl.<init>(TaskAttemptContextImpl.java:49)
at
org.apache.hadoop.mapreduce.task.TaskAttemptContextImpl.<init>(TaskAttemptContextImpl.java:44)
at
org.apache.hadoop.hive.shims.Hadoop23Shims$1.<init>(Hadoop23Shims.java:116)
at
org.apache.hadoop.hive.shims.Hadoop23Shims.newTaskAttemptContext(Hadoop23Shims.java:116)
at
org.apache.hadoop.hive.hbase.HiveHBaseTableInputFormat.getRecordReader(HiveHBaseTableInputFormat.java:175)
at
org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:445)
at
org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:534)
... 33 more
> TestHBaseMinimrCliDriver throws weird error with HBase 0.94.5 and Hadoop 23
> and test is stuck infinitely
> --------------------------------------------------------------------------------------------------------
>
> Key: HIVE-4216
> URL: https://issues.apache.org/jira/browse/HIVE-4216
> Project: Hive
> Issue Type: Bug
> Components: StorageHandler
> Affects Versions: 0.12.0, 0.11.0, 0.9.0
> Environment: Hadoop 23.X
> Reporter: Viraj Bhat
> Fix For: 0.13.0
>
> Attachments: HIVE-4216.1.patch
>
>
> After upgrading to Hadoop 23 and HBase 0.94.5 compiled for Hadoop 23. The
> TestHBaseMinimrCliDriver, fails after performing the following steps
> Update "hbase_bulk.m" with the following properties
> set mapreduce.totalorderpartitioner.naturalorder=false;
> set mapreduce.totalorderpartitioner.path=/tmp/hbpartition.lst;
> Otherwise I keep seeing: "_partition.lst" not found exception in the mappers,
> even though set total.order.partitioner.path=/tmp/hbpartition.lst is set.
> When the test runs, the 3 reducer phase of the second query fails with the
> following error, but the MiniMRCluster keeps spinning up new reducer and the
> test is stuck infinitely.
> {code}
> insert overwrite table hbsort
> select distinct value,
> case when key=103 then cast(null as string) else key end,
> case when key=103 then ''
> else cast(key+1 as string) end
> from src
> cluster by value;
> {code}
> The stack trace I see in the syslog for the Node Manager is the following:
> ==============================================================
> 13-03-20 16:26:48,942 FATAL [IPC Server handler 17 on 55996]
> org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task:
> attempt_1363821864968_0003_r_000002_0 - exited : java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row (tag=0)
> {"key":{"reducesinkkey0":"val_200"},"value":{"_col0":"val_200","_col1":"200","_col2":"201.0"},"alias":0}
> at
> org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:268)
> at
> org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:448)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:399)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:157)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1212)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:152)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row (tag=0)
> {"key":{"reducesinkkey0":"val_200"},"value":{"_col0":"val_200","_col1":"200","_col2":"201.0"},"alias":0}
> at
> org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:256)
> ... 7 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveRecordWriter(HiveFileFormatUtils.java:237)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.createBucketFiles(FileSinkOperator.java:477)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:525)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
> at
> org.apache.hadoop.hive.ql.exec.ExtractOperator.processOp(ExtractOperator.java:45)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at
> org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:247)
> ... 7 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.mapreduce.TaskID$CharTaskTypeMaps.getRepresentingCharacter(TaskID.java:265)
> at org.apache.hadoop.mapreduce.TaskID.appendTo(TaskID.java:153)
> at
> org.apache.hadoop.mapreduce.TaskAttemptID.appendTo(TaskAttemptID.java:119)
> at
> org.apache.hadoop.mapreduce.TaskAttemptID.toString(TaskAttemptID.java:151)
> at java.lang.String.valueOf(String.java:2826)
> at
> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:209)
> at
> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.<init>(FileOutputCommitter.java:69)
> at
> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat.getRecordWriter(HFileOutputFormat.java:90)
> at
> org.apache.hadoop.hive.hbase.HiveHFileOutputFormat.getFileWriter(HiveHFileOutputFormat.java:67)
> at
> org.apache.hadoop.hive.hbase.HiveHFileOutputFormat.getHiveRecordWriter(HiveHFileOutputFormat.java:104)
> at
> org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getRecordWriter(HiveFileFormatUtils.java:246)
> at
> org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveRecordWriter(HiveFileFormatUtils.java:234)
> ... 14 more
> ==============================================================
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
