create table exception
what could be the reason for create table test_table(a int); FAILED: Error in metadata: MetaException(message:javax.jdo.JDOException: Couldnt obtain a new sequence (unique id) : Binary logging not possible. Message: Transaction level 'READ-COMMITTED' in InnoDB is not safe for binlog mode 'STATEMENT' NestedThrowables: java.sql.SQLException: Binary logging not possible. Message: Transaction level 'READ-COMMITTED' in InnoDB is not safe for binlog mode 'STATEMENT') FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Re: HIVE::START WITH and CONNECT BY implementation in Hive
Look up "Transitive Closure" It's a fast technique to analyze hierarchical data w/o proprietary SQL extensions. - Douglas On Oct 20, 2014, at 3:12 AM, yogesh dhari mailto:yogeshh...@gmail.com>> wrote: Hello All How can we achive start with .. connect by clause can be used to select data that has a hierarchical relationship (usually some sort of parent->child (boss->employee or thing->parts). into Hive, or what would be the other work around to implement this. Please suggest Thanks in advance Yogesh
Re: select * from table and select column from table in hive
whats your table create ddl? is the data in csv like format? On 21 Oct 2014 00:26, "Raj Hadoop" wrote: > I am able to see the data in the table for all the columns when I issue > the following - > > SELECT * FROM t1 WHERE dt1='2013-11-20' > > > But I am unable to see the column data when i issue the following - > > SELECT cust_num FROM t1 WHERE dt1='2013-11-20' > > The above shows null values. > > How should I debug this ? > >
select * from table and select column from table in hive
I am able to see the data in the table for all the columns when I issue the following - SELECT * FROM t1 WHERE dt1='2013-11-20' But I am unable to see the column data when i issue the following - SELECT cust_num FROM t1 WHERE dt1='2013-11-20' The above shows null values. How should I debug this ?
UDF related: org.apache.hive.com.esotericsoftware.kryo.KryoException
Hello, I am experiencing inconsistent behavior when trying to use UDF on 0.13.1 on Amazon's EMR (AMI 3.2.1). I generated a uber jar and deployed a UDF like so: create temporary function someFunction as "hive.udf.localization.MyUDF" using jar "s3://waze.mapreduce.shared/scripts/Hive/MyHive.jar"; I am having these 2 (related?) problems: 1) When I simply try to use my UDF I get Error: java.lang.RuntimeException: org.apache.hive.com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException Serialization trace: childRectangles (SomeClass1) statesTree (SomeClass2) states (SomeClass3) genericUDF (org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc) colExprMap (org.apache.hadoop.hive.ql.exec.SelectOperator) childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator) aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork) at org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:360) at org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:271) at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:254) at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:438) at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:431) at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:587) at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.(MapTask.java:169) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:410) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162) Caused by: org.apache.hive.com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException... After digging I found this: http://mail-archives.apache.org/mod_mbox/hive-dev/201408.mbox/%3CJIRA.12733732.1407927053435.81293.1408008733570@arcas%3E *and after setting * *hive.plan.serialization.format=javaXML* *the UDF is running OK on my test data set of 50 lines.* 2) When running the UDF in a more complex 2-joins query I am getting a somewhat related error: org.apache.hive.com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException Serialization trace: org.apache.hadoop.hive.ql.parse.SemanticException: Generate Map Join Task Error: java.lang.NullPointerException Serialization trace: childRectangles (SomeClass1) statesTree (SomeClass2) states (SomeClass3) genericUDF (org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc) colExprMap (org.apache.hadoop.hive.ql.exec.SelectOperator) childOperators (org.apache.hadoop.hive.ql.exec.JoinOperator) opParseCtxMap (org.apache.hadoop.hive.ql.plan.MapWork) mapWork (org.apache.hadoop.hive.ql.plan.MapredWork)... *This does not go away even after setting hive.plan.serialization.format=javaXML * Can someone please advise? Many thanks, Harel.
Re: Hivemetastore Error: Duplicate entry
We also encounter this problem. It seems that it happens in various situations. Currently we patch https://issues.apache.org/jira/browse/HIVE-7167 which has some retries logics in init HiveMetaStoreClient and hope to reduce the occurrence of this problem. Thanks. 2014-07-02 22:40 GMT+08:00 hadoop hive : > Seems that you are inserting duplicate record into that global table. > > Can you show the query ??? > On Jun 26, 2014 8:19 AM, "张伟" wrote: > >> I run hive-0.13.1 on hadoop-2.2.0. When I insert into an orc table, then >> i get the following ERROR, indicating that Hive is trying to insert >> duplicate entry to a table which is called "GLOBAL_PRIVS". Any idea how to >> fix it? >> >> Exception in thread "main" java.lang.RuntimeException: >> java.lang.RuntimeException: Unable to instantiate >> org.apache.hadoop.hive.metastore.HiveMetaStoreClient >> at >> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:346) >> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681) >> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at org.apache.hadoop.util.RunJar.main(RunJar.java:212) >> Caused by: java.lang.RuntimeException: Unable to instantiate >> org.apache.hadoop.hive.metastore.HiveMetaStoreClient >> at >> org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1412) >> at >> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:62) >> at >> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:72) >> at >> org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2453) >> at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2465) >> at >> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:340) >> ... 7 more >> Caused by: java.lang.reflect.InvocationTargetException >> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native >> Method) >> at >> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) >> at >> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) >> at java.lang.reflect.Constructor.newInstance(Constructor.java:526) >> at >> org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1410) >> ... 12 more >> Caused by: javax.jdo.JDODataStoreException: Exception thrown flushing >> changes to datastore >> NestedThrowables: >> java.sql.BatchUpdateException: Duplicate entry 'admin-ROLE-All-admin-ROLE' >> for key 'GLOBALPRIVILEGEINDEX' >> at >> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451) >> at >> org.datanucleus.api.jdo.JDOTransaction.commit(JDOTransaction.java:165) >> at >> org.apache.hadoop.hive.metastore.ObjectStore.commitTransaction(ObjectStore.java:406) >> at >> org.apache.hadoop.hive.metastore.ObjectStore.grantPrivileges(ObjectStore.java:3877) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at >> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:108) >> at com.sun.proxy.$Proxy7.grantPrivileges(Unknown Source) >> at >> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultRoles(HiveMetaStore.java:567) >> at >> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:398) >> at >> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.(HiveMetaStore.java:356) >> at >> org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:54) >> at >> org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:59) >> at >> org.apache.hadoop.hive.metastore.HiveMetaStore.newHMSHandler(HiveMetaStore.java:4944) >> at >> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:171) >> ... 17 more >> >
Differenct between Parquet file Vs Hive RC file format
Hi , Can anyone please tell the difference between Parquet file format and Hive RC file format. Both the file formats are *columnar storage*, then what are the advantages / disadvantage over others. -- -- Regards, RAVI PRASAD. T
HIVE::START WITH and CONNECT BY implementation in Hive
Hello All How can we achive *start with .. connect by* clause can be used to select data that has a hierarchical relationship (usually some sort of parent->child (boss->employee or thing->parts). into Hive, or what would be the other work around to implement this. Please suggest Thanks in advance Yogesh
