Hi,
I am wondering there is someone who encountered the same (or similar) problem
while usingHive and looking for a solution it.
I am running a daily batch to import data from mysql to hive by using beeline
and HS2.Yesterday, the job failed with errors that I havn't seen before.The
following is an excerpt from the HS2 log:...2015-05-19 01:13:52,483 INFO
exec.Task (SessionState.java:printInfo(417)) - Loading data to table
kago_comicoshop.addresses partition (dt=2015-05-18) from
hdfs://mycluster/tmp/hive-hive/hive_2015-05-19_01-13-36_529_2684279023796649469-3/-ext-10000
2015-05-19 01:13:52,746 ERROR metastore.RetryingHMSHandler
(RetryingHMSHandler.java:invoke(157)) - Retrying HMSHandler after 1000 ms
(attempt 1 of 1) with error: javax.jdo.JDOFatalInternalException: Invalid index
1 for DataStoreMapping. at
org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:591)
at
org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:732)
at
org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:752)
at
org.apache.hadoop.hive.metastore.ObjectStore.addPartition(ObjectStore.java:1196)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606) at
org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:108)
at com.sun.proxy.$Proxy12.addPartition(Unknown Source) at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.append_partition_common(HiveMetaStore.java:1547)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.append_partition_with_environment_context(HiveMetaStore.java:1602)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606) at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:105)
at
com.sun.proxy.$Proxy13.append_partition_with_environment_context(Unknown
Source) at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.appendPartition(HiveMetaStoreClient.java:424)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.appendPartition(HiveMetaStoreClient.java:418)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606) at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89)
at com.sun.proxy.$Proxy16.appendPartition(Unknown Source) at
org.apache.hadoop.hive.ql.metadata.Hive.getPartition(Hive.java:1612) at
org.apache.hadoop.hive.ql.metadata.Hive.loadPartition(Hive.java:1233) at
org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:409) at
org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:151) at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:65)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1485) at
org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1263) at
org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1091) at
org.apache.hadoop.hive.ql.Driver.run(Driver.java:931) at
org.apache.hadoop.hive.ql.Driver.run(Driver.java:926)
...NestedThrowablesStackTrace:Invalid index 1 for
DataStoreMapping.org.datanucleus.exceptions.NucleusException: Invalid index 1
for DataStoreMapping. at
org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:591)
at
org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:732)
at
org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:752)
...
When I restarted HS2 and rerun the batch, it went well without any problem.
Best wishes,Han-Cheol
趙漢哲 CHO, HAN-CHEOL (Ph.D)
データ研究室 / 社員
--> 〒105-6322 東京都港区虎ノ門１－23-1 虎ノ門ヒルズ森タワー22階
Email hancheol....@nhn-playart.com Messenger
NHN PlayArt 株式会社
