[jira] [Updated] (HIVE-10464) resolved
[ https://issues.apache.org/jira/browse/HIVE-10464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ankush updated HIVE-10464: -- Description: resolved (was: Could you please let me know how i find the kryo version that i using ? Please help me on this, We are just running HQL (Hive) queries) Summary: resolved (was: How i find the kryo version ) resolved Key: HIVE-10464 URL: https://issues.apache.org/jira/browse/HIVE-10464 Project: Hive Issue Type: Improvement Reporter: ankush resolved -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-7711) Error Serializing GenericUDF
[ https://issues.apache.org/jira/browse/HIVE-7711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14509422#comment-14509422 ] ankush commented on HIVE-7711: -- Could you please let me know how i find the kryo version ? how i find the kryo version that i using ? Please help Error Serializing GenericUDF Key: HIVE-7711 URL: https://issues.apache.org/jira/browse/HIVE-7711 Project: Hive Issue Type: Bug Affects Versions: 0.13.0 Reporter: Dr. Christian Betz Attachments: HIVE-7711.1.patch.txt I get an exception running a job with a GenericUDF in HIVE 0.13.0 (which was ok in HIVE 0.12.0). The org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc is serialized using Kryo, trying to serialize stuff in my GenericUDF which is not serializable (doesn't implement Serializable). Switching to Kryo made the comment in ExprNodeGenericFuncDesc obsolte: /** * In case genericUDF is Serializable, we will serialize the object. * * In case genericUDF does not implement Serializable, Java will remember the * class of genericUDF and creates a new instance when deserialized. This is * exactly what we want. */ Find the stacktrace below, however, the description above should be clear. Exception in thread main org.apache.hive.com.esotericsoftware.kryo.KryoException: java.lang.UnsupportedOperationException Serialization trace: value (java.util.concurrent.atomic.AtomicReference) state (clojure.lang.Atom) state (udfs.ArraySum) genericUDF (org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc) colExprMap (org.apache.hadoop.hive.ql.exec.SelectOperator) childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator) aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork) mapWork (org.apache.hadoop.hive.ql.plan.MapredWork) at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125) at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507) at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694) at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106) at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507) at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694) at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106) at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507) at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694) at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106) at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507) at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776) at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:139) at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:17) at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694) at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106) at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507) at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776) at org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:112) at org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:18) at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694) at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106) at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507) at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776) at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:139) at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:17) at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694) at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106) at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507) at
[jira] [Commented] (HIVE-10464) How i find the kryo version
[ https://issues.apache.org/jira/browse/HIVE-10464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14509489#comment-14509489 ] ankush commented on HIVE-10464: --- thank you Lefty How i find the kryo version Key: HIVE-10464 URL: https://issues.apache.org/jira/browse/HIVE-10464 Project: Hive Issue Type: Improvement Reporter: ankush Could you please let me know how i find the kryo version that i using ? Please help me on this, We are just running HQL (Hive) queries -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10464) How i find the kryo version
[ https://issues.apache.org/jira/browse/HIVE-10464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14509493#comment-14509493 ] ankush commented on HIVE-10464: --- i ask the que on the u...@hive.apache.org mailing list. Thank you How i find the kryo version Key: HIVE-10464 URL: https://issues.apache.org/jira/browse/HIVE-10464 Project: Hive Issue Type: Improvement Reporter: ankush Could you please let me know how i find the kryo version that i using ? Please help me on this, We are just running HQL (Hive) queries -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9967) h
[ https://issues.apache.org/jira/browse/HIVE-9967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ankush updated HIVE-9967: - Description: thank (was: hive show roles; FAILED: SemanticException The current builtin authorization in Hive is incomplete and disabled. Error from Hive: error code: '0' error message: 'ExecuteStatement finished with operation state: CLOSED_STATE') Environment: h (was: when running a query to retrieve row data , receiving the other error ExecuteStatement finished with operation state: CLOSED_STATE) Summary: h (was: 'ExecuteStatement finished with operation state: CLOSED_STATE') h - Key: HIVE-9967 URL: https://issues.apache.org/jira/browse/HIVE-9967 Project: Hive Issue Type: Bug Components: HBase Handler, Hive, HiveServer2, ODBC Affects Versions: 0.13.1 Environment: h Reporter: ankush Labels: hadoop, performance thank -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9720) Metastore does not properly migrate column stats when renaming a table across databases.
[
https://issues.apache.org/jira/browse/HIVE-9720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361842#comment-14361842
]
ankush commented on HIVE-9720:
--
Hi Chaoyu,
i m facing the issue in hive.can u suggest on this plz
hive show roles;
FAILED: SemanticException The current builtin authorization in Hive is
incomplete and disabled.
Error from Hive: error code: '0' error message: 'ExecuteStatement finished with
operation state: CLOSED_STATE'
Metastore does not properly migrate column stats when renaming a table across
databases.
Key: HIVE-9720
URL: https://issues.apache.org/jira/browse/HIVE-9720
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.13.1
Reporter: Alexander Behm
Assignee: Chaoyu Tang
Fix For: 1.2.0
Attachments: HIVE-9720.1.patch, HIVE-9720.1.patch, HIVE-9720.patch
It appears that the Hive Metastore does not properly migrate column
statistics when renaming a table across databases. While renaming across
databases is not supported in HiveQL, it can be done via the Metastore Thrift
API.
The problem is that such a newly renamed table cannot be dropped (unless
renamed back to its original database/name).
Here are steps for reproducing the issue.
1. From the Hive shell/beeline:
{code}
create database db1;
create database db2;
create table db1.mv (i int);
use db1;
analyze table mv compute statistics for columns i;
{code}
2. From a Java program:
{code}
public static void main(String[] args) throws Exception {
HiveConf conf = new HiveConf(MetaStoreClientPool.class);
HiveMetaStoreClient hiveClient = new HiveMetaStoreClient(conf);
Table t = hiveClient.getTable(db1, mv);
t.setDbName(db2);
t.setTableName(mv2);
hiveClient.alter_table(db1, mv, t);
}
{code}
3. From the Hive shell/beeline:
{code}
drop table db2.mv2;
{code}
Stack shown when running 3:
{code}
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask.
MetaException(message:javax.jdo.JDODataStoreException: Exception thrown
flushing changes to datastore
at
org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
at
org.datanucleus.api.jdo.JDOTransaction.commit(JDOTransaction.java:165)
at
org.apache.hadoop.hive.metastore.ObjectStore.commitTransaction(ObjectStore.java:411)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:108)
at com.sun.proxy.$Proxy0.commitTransaction(Unknown Source)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_table_core(HiveMetaStore.java:1389)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_table_with_environment_context(HiveMetaStore.java:1525)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:106)
at com.sun.proxy.$Proxy1.drop_table_with_environment_context(Unknown
Source)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$drop_table_with_environment_context.getResult(ThriftHiveMetastore.java:8072)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$drop_table_with_environment_context.getResult(ThriftHiveMetastore.java:8056)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at
org.apache.hadoop.hive.metastore.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:48)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:244)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
NestedThrowablesStackTrace:
java.sql.BatchUpdateException: Batch entry 0 DELETE FROM TBLS WHERE
TBL_ID='1621' was aborted. Call getNextException to see the cause.
at
[jira] [Commented] (HIVE-9948) SparkUtilities.getFileName passes File.separator to String.split() method
[ https://issues.apache.org/jira/browse/HIVE-9948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361844#comment-14361844 ] ankush commented on HIVE-9948: -- Hi Xuefu, can u suggest on this, hive show roles; FAILED: SemanticException The current builtin authorization in Hive is incomplete and disabled. Error from Hive: error code: '0' error message: 'ExecuteStatement finished with operation state: CLOSED_STATE' SparkUtilities.getFileName passes File.separator to String.split() method - Key: HIVE-9948 URL: https://issues.apache.org/jira/browse/HIVE-9948 Project: Hive Issue Type: Bug Components: Spark Reporter: Alexander Pivovarov Assignee: Alexander Pivovarov Priority: Minor Fix For: 1.2.0 Attachments: HIVE-9948.1.patch String.split() method expects regex. This is why File.separator can not be passed to split. In this particular case we can use FilenameUtils.getName to get file name -- This message was sent by Atlassian JIRA (v6.3.4#6332)
