No. We did not see any issue with create table (without select). Create table
as select also passes sometimes.
We are using MySQL for our metastore.
Will get back to you on load at the time of create. Is there a direct way to
know the load?
So, what is the solution/workaround if there is a metastore issue?
Thanks
Amareshwari
On 10/1/10 11:38 AM, "Ning Zhang" <nzh...@facebook.com> wrote:
Amareshwari, this should be a metastore issue. Did you see this kind of issue
in create table (without select) alone?
Which RDBMS are you using for metastore? MySQL or Oracle? Was your
database/thrift server heavy-loaded at that time?
On Sep 30, 2010, at 10:28 PM, Amareshwari Sri Ramadasu wrote:
Carl, we are using trunk version.
On 10/1/10 10:16 AM, "Carl Steinbach" <c...@cloudera.com
<x-msg://6/c...@cloudera.com> > wrote:
Hi Amareshwari,
Which version of Hive are you using to run the Hive metastore server?
Thanks.
Carl
On Thu, Sep 30, 2010 at 9:25 PM, Amareshwari Sri Ramadasu
<amar...@yahoo-inc.com <x-msg://6/amar...@yahoo-inc.com> > wrote:
Hi,
Create table as select queries fail with
org.apache.thrift.TApplicationException in our clusters for some queries.
Following is the stack trace for the exception :
Error in metadata: org.apache.thrift.TApplicationException: Internal error
processing create_table
org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.thrift.TApplicationException: Internal error processing
create_table
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:405)
at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:2465)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:180)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:108)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:895)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:764)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:640)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:140)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:199)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:353)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: org.apache.thrift.TApplicationException: Internal error processing
create_table
at
org.apache.thrift.TApplicationException.read(TApplicationException.java:107)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_create_table(ThriftHiveMetastore.java:566)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.create_table(ThriftHiveMetastore.java:549)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:281)
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:399)
When we tried with hive.metastore.connect.retries=10, it fails though it
succeeds occasionally.
The metastore logs have the following exception
Internal error processing create_table
java.lang.RuntimeException: Commit is called, but transaction is not active.
Either there are mismatching open and
close calls or rollback was called in the same trasaction
at
org.apache.hadoop.hive.metastore.ObjectStore.commitTransaction(ObjectStore.java:250)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:795)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.access$600(HiveMetaStore.java:79)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler$13.run(HiveMetaStore.java:816)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler$13.run(HiveMetaStore.java:813)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.executeWithRetry(HiveMetaStore.java:234)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table(HiveMetaStore.java:813)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table.process(ThriftHiveMetastore.java:1992)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor.process(ThriftHiveMetastore.java:1644)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:252)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Can somebody help us find the root cause of the problem?
Thanks
Amareshwari
