Review Request 45525: HIVE-12484 : Show meta operations on HS2 web UI
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45525/
---
Review request for hive.
Bugs: HIVE-12484
https://issues.apache.org/jira/browse/HIVE-12484
Repository: hive-git
Description
---
HIVE-12484 : Show meta operations on HS2 web UI
Expands display availablity to meta operations
as well as "set" ("command") operations.
Moves OperationDisplay to Operation level.
Added getDisplayQuery() method for all meta
operations to specify the query string to
display in the UI.
Diffs
-
itests/hive-unit/src/test/java/org/apache/hive/service/cli/session/TestQueryDisplay.java
418f71eb87cdd519677b2f5a59c67099f704ec80
ql/src/java/org/apache/hadoop/hive/ql/QueryDisplay.java
d582bc063fc150002a01d63451ae6632fca29ac1
service/src/jamon/org/apache/hive/tmpl/QueryProfileTmpl.jamon
8d51a73f7ac6c5eb5fe44c5525ed840b641c5c6b
service/src/java/org/apache/hive/service/cli/operation/GetCatalogsOperation.java
8868ec18e0f50f6a434eb0498e4a923b0a151b97
service/src/java/org/apache/hive/service/cli/operation/GetColumnsOperation.java
35b6c5209025a0e9b5c3732493afd9d30d4f4a1f
service/src/java/org/apache/hive/service/cli/operation/GetFunctionsOperation.java
8db2e62604345a2d3b9193c497080f3f2977f2ac
service/src/java/org/apache/hive/service/cli/operation/GetSchemasOperation.java
d6f6280f1c398fa0239cb52070ff8c40168215bf
service/src/java/org/apache/hive/service/cli/operation/GetTableTypesOperation.java
a09b39a4e0855152925085e46b2e0a703b32de7f
service/src/java/org/apache/hive/service/cli/operation/GetTablesOperation.java
740b85158a6008ecece592d72cd743d8f5a433f4
service/src/java/org/apache/hive/service/cli/operation/GetTypeInfoOperation.java
2a0fec27715daba4307f661a6f65c3cd4d43c975
service/src/java/org/apache/hive/service/cli/operation/HiveCommandOperation.java
f5a9771449a7cc1fb5ac5c0b6f2fd6ab032a2162
service/src/java/org/apache/hive/service/cli/operation/MetadataOperation.java
285b4f94ec828a357da4c2e7dc96c64bd459ff80
service/src/java/org/apache/hive/service/cli/operation/Operation.java
d9a273b1b95adf7a6a68fa013042da2408904d64
service/src/java/org/apache/hive/service/cli/operation/OperationDisplay.java
PRE-CREATION
service/src/java/org/apache/hive/service/cli/operation/OperationDisplayCache.java
PRE-CREATION
service/src/java/org/apache/hive/service/cli/operation/OperationManager.java
56a9c18bc118b8d67f87e5500ec7d4442a90a869
service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java
04d816a72afcadbff26f5eb5d68003a7344e31c9
service/src/java/org/apache/hive/service/cli/operation/SQLOperationDisplay.java
d2ca1e7b79e9722ad0ce990e96bf3fc9297ca7fd
service/src/java/org/apache/hive/service/cli/operation/SQLOperationDisplayCache.java
4a33d37a88a7216d832a5b9ab4b7534091129be7
service/src/java/org/apache/hive/service/servlet/QueryProfileServlet.java
8fa447a386c5a7a000b04453618f502e1681364e
service/src/resources/hive-webapps/hiveserver2/hiveserver2.jsp
8b46550b603a51ba6143c36746029475d7e11a40
Diff: https://reviews.apache.org/r/45525/diff/
Testing
---
Tested with different sql, metadata and set operations.
Added tests to TestQueryDisplay
Thanks,
Mohit Sabharwal
[jira] [Created] (HIVE-13397) ORC CompressedStream.read implementation not compatible with SerializationUtils in exceptional case
Yuxing Yao created HIVE-13397:
-
Summary: ORC CompressedStream.read implementation not compatible
with SerializationUtils in exceptional case
Key: HIVE-13397
URL: https://issues.apache.org/jira/browse/HIVE-13397
Project: Hive
Issue Type: Bug
Reporter: Yuxing Yao
I used a orc file with some bytes corrupt to test whether the reader is
vulnerable to attacks. And I found there's a point in CompressedStream not
acting correctly when handling the exceptional case, and will cause a infinite
loop.
In
org.apache.hadoop.hive.ql.io.orc.InStream$CompressedStream.read(byte[] data,
int offset, int length)
when the data is incorrect, it will return -1 to mark that is exceptional case,
(Not throwing exception which is the right behavior for this buffered read).
and in
SerializationUtils.readLongBE(InStream in, long[] buffer, int start, int
numHops, int numBytes)
// bulk read to buffer
int bytesRead = in.read(readBuffer, 0, toRead);
while (bytesRead != toRead) {
bytesRead += in.read(readBuffer, bytesRead, toRead - bytesRead);
}
it do not check the -1 return value. and add the the bytesRead directly, which
caused the infinite loop.
Either change the read to throw exception or add -1 check here can fix this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-13396) LLAP: Include hadoop-metrics2.properties file LlapServiceDriver
Prasanth Jayachandran created HIVE-13396: Summary: LLAP: Include hadoop-metrics2.properties file LlapServiceDriver Key: HIVE-13396 URL: https://issues.apache.org/jira/browse/HIVE-13396 Project: Hive Issue Type: Bug Components: llap Affects Versions: 2.1.0 Reporter: Prasanth Jayachandran Assignee: Prasanth Jayachandran Attachments: HIVE-13396.1.patch LlapServiceDriver should package hadoop-metrics2.properties file to support dumping llap daemon metrics to hadoop metric sinks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-13395) Lost Update problem in ACID
Eugene Koifman created HIVE-13395: - Summary: Lost Update problem in ACID Key: HIVE-13395 URL: https://issues.apache.org/jira/browse/HIVE-13395 Project: Hive Issue Type: Bug Components: Transactions Affects Versions: 2.0.0, 1.2.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Priority: Critical ACID users can run into Lost Update problem. In Hive 1.2, Driver.recordValidTxns() (which records the snapshot to use for the query) is called in Driver.compile(). Now suppose to concurrent "update T set x = x + 1" are executed. (for simplicity assume there is exactly 1 row in T) What can happen is that both compile at the same time (more precisely before acquireLocksAndOpenTxn() in runInternal() is called) and thus will lock in the same snapshot, say the value of x = 7 in this snapshot. Now 1 will get the lock on the row, the second will block. Now 1, makes x = 8 and commits. Now 2 proceeds and makes x = 8 again since in it's snapshot x is still 7. This specific issue is solved in Hive 1.3/2.0 (HIVE-11077 which is a large patch that deals with multi-statement txns) by moving recordValidTxns() after locks are acquired which reduces the likelihood of this but doesn't eliminate the problem. Even in 1.3 version of the code, you could have the same issue. Assume the same 2 queries: Both start a txn, say txnid 9 and 10. Say 10 gets the lock first, 9 blocks. 10 updates the row (so x = 8) and thus ReaderKey.currentTransactionId=10. 10 commits. Now 9 can proceed and it will get a snapshot that includes 10, i.e. it will see x = 8 and it will write x = 9, but it will set ReaderKey.currentTransactionId = 9. Thus when merge logic runs, it will see x = 8 is the later version of this row, i.e. lost update. The problem is that locks alone are insufficient for MVCC architecture. At lower level Row ID has (originalTransactionId, rowid, bucket id, currentTransactionId) and since on update/delete we do a table scan, we could check that we are about to write a row with currentTransactionId < (currentTransactionId of row we've read) and fail the query. Currently, currentTransactionId is not surfaced at higher level where this check can be made. This would not work (efficiently) longer term where we want to support fast update on user defined PK vis streaming ingest. Also, this would not work with multi statement txns since in that case we'd lock in the snapshot at the start of the txn, but then 2nd, 3rd etc queries would use the same snapshot and the locks for these queries would be acquired after the snapshot is locked in so this would be the same situation as pre HIVE-11077. A more robust solution (commonly used with MVCC) is to keep track of start and commit time (logical counter) or each transaction to detect if two txns overlap. The 2nd part is to keep track of write-set, i.e. which data (rows, partitions, whatever appropriate level of granularity is) were modified by any txn and if 2 txns overlap in time and wrote the same element, abort later one. This is called first-committer-wins rule. This requires a MS DB schema change It would be most convenient to use the same sequence for txnId, start and commit time (in which case txnid=start time). In this case we'd need to add 1 filed to TXNS table. The complication here is that we'll be using elements of the sequence faster and they are used as part of file name of delta and base dir and currently limited to 7 digits which can be exceeded. So this would require some thought to handling upgrade/migration. Also, write-set tracking requires either additional metastore table or keeping info in HIVE_LOCKS around longer with new state. In the short term, on SQL side of things we could (in auto commit mode only) acquire the locks first and then open the txn AND update these locks with txn id. This implies another Thrift change to pass in lockId to openTxn. The same would not work for Streaming API since it opens several txns at once and then acquires locks for each. (Not sure if that's is an issue or not since Streaming only does Insert). Either way this feels hacky. Here is one simple example why we need Write-Set tracking for multi-statement txns Consider transactions T ~1~ and T ~2~: T ~1~: r ~1~\[x] -> w ~1~\[y] -> c ~1~ T ~2~: w ~2~\[x] -> w ~2~\[y] -> c ~2~ Suppose the order of operations is r ~1~\[x] w ~2~\[x] then a conventional R/W lock manager w/o MVCSS will block the write from T ~2~ With MVCC we don't want readers to interfere with writers and so the following schedule is possible (because Hive's semi-shared (write) don't conflict with shared (read) locks) in Hive's current implementation. r ~1~\[x] w ~2~\[x] w ~2~\[y] c ~2~ w ~1~\[y] c ~1~ By the time w ~1~\[y] happens, T ~2~ has committed and released it's locks.
[jira] [Created] (HIVE-13394) Analyze table fails in tez on empty partitions
Vikram Dixit K created HIVE-13394:
-
Summary: Analyze table fails in tez on empty partitions
Key: HIVE-13394
URL: https://issues.apache.org/jira/browse/HIVE-13394
Project: Hive
Issue Type: Bug
Components: Tez
Affects Versions: 2.0.0, 1.2.1
Reporter: Vikram Dixit K
Assignee: Vikram Dixit K
{code}
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:352)
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:237)
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:252)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:150)
... 14 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.ArrayIndexOutOfBoundsException: 0
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.process(GroupByOperator.java:766)
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:343)
... 17 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: 0
at
org.apache.hadoop.hive.ql.udf.generic.NumDistinctValueEstimator.deserialize(NumDistinctValueEstimator.java:219)
at
org.apache.hadoop.hive.ql.udf.generic.NumDistinctValueEstimator.(NumDistinctValueEstimator.java:112)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDAFComputeStats$GenericUDAFNumericStatsEvaluator.merge(GenericUDAFComputeStats.java:556)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:188)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.updateAggregations(GroupByOperator.java:612)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processAggr(GroupByOperator.java:851)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processKey(GroupByOperator.java:695)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.process(GroupByOperator.java:761)
... 18 more
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1
killedTasks:0, Vertex vertex_145591034_27748_1_01 [Reducer 2] killed/failed
due to:OWN_TASK_FAILURE]DAG did not succeed due to VERTEX_FAILURE.
failedVertices:1 killedVertices:0
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-13393) Beeline: Print help message for the --incremental option
Vaibhav Gumashta created HIVE-13393: --- Summary: Beeline: Print help message for the --incremental option Key: HIVE-13393 URL: https://issues.apache.org/jira/browse/HIVE-13393 Project: Hive Issue Type: Bug Components: Beeline Affects Versions: 2.0.0, 1.2.1 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta beeline --help doesn't print the usage tips for the --incremental option. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-13392) disable speculative execution for ACID Compactor
Eugene Koifman created HIVE-13392:
-
Summary: disable speculative execution for ACID Compactor
Key: HIVE-13392
URL: https://issues.apache.org/jira/browse/HIVE-13392
Project: Hive
Issue Type: Bug
Components: Transactions
Affects Versions: 1.0.0
Reporter: Eugene Koifman
Assignee: Eugene Koifman
https://developer.yahoo.com/hadoop/tutorial/module4.html
Speculative execution is enabled by default. You can disable speculative
execution for the mappers and reducers by setting the
mapred.map.tasks.speculative.execution and
mapred.reduce.tasks.speculative.execution JobConf options to false,
respectively.
CompactorMR is currently not set up to handle speculative execution and may
lead to something like
{code}
2016-02-08 22:56:38,256 WARN [main] org.apache.hadoop.mapred.YarnChild:
Exception running child :
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException):
Failed to CREATE_FILE
/apps/hive/warehouse/service_logs_v2/ds=2016-01-20/_tmp_6cf08b9f-c2e2-4182-bc81-e032801b147f/base_13858600/bucket_4
for DFSClient_attempt_1454628390210_27756_m_01_1_131224698_1 on
172.18.129.12 because this file lease is currently owned by
DFSClient_attempt_1454628390210_27756_m_01_0_-2027182532_1 on 172.18.129.18
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.recoverLeaseInternal(FSNamesystem.java:2937)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:2562)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInt(FSNamesystem.java:2451)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:2335)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:688)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:397)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2151)
{code}
Short term: disable speculative execution for this job
Longer term perhaps make each task write to dir with UUID...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-13391) add an option to LLAP to use keytab to read data
Sergey Shelukhin created HIVE-13391: --- Summary: add an option to LLAP to use keytab to read data Key: HIVE-13391 URL: https://issues.apache.org/jira/browse/HIVE-13391 Project: Hive Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin This can be used for non-doAs case to allow access to clients who don't propagate HDFS tokens. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-13390) HiveServer2: Add more test to ZK service discovery using MiniHS2
Vaibhav Gumashta created HIVE-13390: --- Summary: HiveServer2: Add more test to ZK service discovery using MiniHS2 Key: HIVE-13390 URL: https://issues.apache.org/jira/browse/HIVE-13390 Project: Hive Issue Type: Bug Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-13389) LLAP external submission client ends up attempting to find an LLAP instance based on the submitting user instead of the hive user
Jason Dere created HIVE-13389: - Summary: LLAP external submission client ends up attempting to find an LLAP instance based on the submitting user instead of the hive user Key: HIVE-13389 URL: https://issues.apache.org/jira/browse/HIVE-13389 Project: Hive Issue Type: Sub-task Components: llap Reporter: Jason Dere Assignee: Jason Dere In the LLAP external client, the LLAP Zookeeper registry creates the Zk path based on the current user, but the user specified here should really be the user running LLAP. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-13388) Fix inconsistent content due to Thrift changes
Wei Zheng created HIVE-13388:
Summary: Fix inconsistent content due to Thrift changes
Key: HIVE-13388
URL: https://issues.apache.org/jira/browse/HIVE-13388
Project: Hive
Issue Type: Bug
Components: Hive
Affects Versions: 2.1.0
Reporter: Wei Zheng
Assignee: Wei Zheng
HIVE-12442 and HIVE-12862 are related here.
If one wants to make some thrift change by following instruction here:
https://cwiki.apache.org/confluence/display/Hive/HowToContribute#HowToContribute-GeneratingThriftCode
When they first execute (i.e. in a clean environment)
{code}
mvn clean install -Pthriftif -DskipTests -Dthrift.home=/usr/local -Phadoop-2
{code}
The following content will show up
{code}
$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Untracked files:
(use "git add ..." to include in what will be committed)
service-rpc/src/gen/thrift/gen-py/__init__.py
service/src/gen/
nothing added to commit but untracked files present (use "git add" to track)
{code}
They should have been included in the codebase.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-13387) Beeline fails silently from missing dependency
Reuben Kuhnert created HIVE-13387:
-
Summary: Beeline fails silently from missing dependency
Key: HIVE-13387
URL: https://issues.apache.org/jira/browse/HIVE-13387
Project: Hive
Issue Type: Task
Reporter: Reuben Kuhnert
Assignee: Reuben Kuhnert
Priority: Minor
Beeline fails to connect because {{HiveSqlException}} dependency is not on
classpath:
{code}
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hive.beeline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:52)
at
org.apache.hive.beeline.BeeLine.execCommandWithPrefix(BeeLine.java:1077)
at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1116)
at org.apache.hive.beeline.BeeLine.initArgs(BeeLine.java:762)
at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:841)
at
org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:493)
at org.apache.hive.beeline.BeeLine.main(BeeLine.java:476)
Caused by: java.lang.NoClassDefFoundError:
org/apache/hive/service/cli/HiveSQLException
at org.apache.hive.jdbc.HiveConnection.(HiveConnection.java:131)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at java.sql.DriverManager.getConnection(DriverManager.java:187)
at
org.apache.hive.beeline.DatabaseConnection.connect(DatabaseConnection.java:141)
at
org.apache.hive.beeline.DatabaseConnection.getConnection(DatabaseConnection.java:205)
at org.apache.hive.beeline.Commands.connect(Commands.java:1393)
at org.apache.hive.beeline.Commands.connect(Commands.java:1314)
... 11 more
Caused by: java.lang.ClassNotFoundException:
org.apache.hive.service.cli.HiveSQLException
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 19 more
{code}
This happens when trying to run beeline as a standalone java application:
{code}
sircodesalot@excalibur:~/Dev/Cloudera/hive/beeline$ mvn exec:java
-Dexec.args='-u jdbc:hive2://localhost:1 sircodesalot'
-Dexec.mainClass="org.apache.hive.beeline.BeeLine"
[INFO] Scanning for projects...
[INFO]
[INFO]
[INFO] Building Hive Beeline 2.1.0-SNAPSHOT
[INFO]
[INFO]
[INFO] --- exec-maven-plugin:1.4.0:java (default-cli) @ hive-beeline ---
Connecting to jdbc:hive2://localhost:1
ERROR StatusLogger No log4j2 configuration file found. Using default
configuration: logging only errors to the console.
org/apache/hive/service/cli/HiveSQLException
Beeline version ??? by Apache Hive
// HERE: This will never connect because of ClassNotFoundException.
0: jdbc:hive2://localhost:1 (closed)>
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-13386) NPE when trying to read parquet file with null values created by Spark
Marek Pasieka created HIVE-13386: Summary: NPE when trying to read parquet file with null values created by Spark Key: HIVE-13386 URL: https://issues.apache.org/jira/browse/HIVE-13386 Project: Hive Issue Type: Bug Reporter: Marek Pasieka Trying to "select * from table limit 1" from hive table throws: Failed with exception java.io.IOException:java.lang.NullPointerException Steps to reproduce: 1. Run spark sql/streaming job, write dataframe with fields that CONTAIN null values (null instead of String/null instead of number) as parquet file. 2. Create external partitioned hive table on top of the parquet files. MSCK repair table. 3. select query returns NPE as described. Not sure if partitioning is relevant, but the tabled is partitioned by string value. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 44524: HIVE-11484 : Fix ObjectInspector for Char and VarChar
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44524/ --- (Updated March 30, 2016, 1:43 p.m.) Review request for hive and Amareshwari Sriramadasu. Changes --- Added testcases Bugs: HIVE-11484 https://issues.apache.org/jira/browse/HIVE-11484 Repository: hive-git Description --- Fixed ObjectInspectorConverter and Java ObjectInspector for Char and Varchar Diffs (updated) - common/src/java/org/apache/hadoop/hive/common/type/HiveBaseChar.java 3514726 common/src/java/org/apache/hadoop/hive/common/type/HiveVarchar.java 969d474 common/src/test/org/apache/hadoop/hive/common/type/TestHiveBaseChar.java 2848465 serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaHiveCharObjectInspector.java f429709 serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaHiveVarcharObjectInspector.java a8e34ff serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 5eb41d5 serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestObjectInspectorConverters.java dd18517 serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestStandardObjectInspectors.java 7d87666 Diff: https://reviews.apache.org/r/44524/diff/ Testing --- Yes. Thanks, Deepak Barr
[jira] [Created] (HIVE-13385) [Cleanup] Streamline Beeline instantiation
Reuben Kuhnert created HIVE-13385:
-
Summary: [Cleanup] Streamline Beeline instantiation
Key: HIVE-13385
URL: https://issues.apache.org/jira/browse/HIVE-13385
Project: Hive
Issue Type: Task
Reporter: Reuben Kuhnert
Assignee: Reuben Kuhnert
Janitorial. Remove circular dependencies in {{BeelineCommandLineCompleter}}.
Stream line code readability.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-13384) Failed to create HiveMetaStoreClient object with proxy user when Kerberos enabled
Bing Li created HIVE-13384:
--
Summary: Failed to create HiveMetaStoreClient object with proxy
user when Kerberos enabled
Key: HIVE-13384
URL: https://issues.apache.org/jira/browse/HIVE-13384
Project: Hive
Issue Type: Improvement
Components: Metastore
Affects Versions: 1.2.1, 1.2.0
Reporter: Bing Li
I wrote a Java client to talk with HiveMetaStore. (Hive 1.2.0)
But found that it can't new a HiveMetaStoreClient object successfully via a
proxy using in Kerberos env.
===
15/10/13 00:14:38 ERROR transport.TSaslTransport: SASL negotiation failure
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException:
No valid credentials provided (Mechanism level: Failed to find any Kerberos
tgt)]
at
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
at
org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94)
at
org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271)
==
When I debugging on Hive, I found that the error came from open() method in
HiveMetaStoreClient class.
Around line 406,
transport = UserGroupInformation.getCurrentUser().doAs(new
PrivilegedExceptionAction() { //FAILED, because the current user
doesn't have the cridential
But it will work if I change above line to
transport = UserGroupInformation.getCurrentUser().getRealUser().doAs(new
PrivilegedExceptionAction() { //PASS
I found DRILL-3413 fixes this error in Drill side as a workaround. But if I
submit a mapreduce job via Pig/HCatalog, it runs into the same issue again when
initialize the object via HCatalog.
It would be better to fix this issue in Hive side.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-13383) RetryingMetaStoreClient retries non retriable embedded metastore client
Thejas M Nair created HIVE-13383: Summary: RetryingMetaStoreClient retries non retriable embedded metastore client Key: HIVE-13383 URL: https://issues.apache.org/jira/browse/HIVE-13383 Project: Hive Issue Type: Bug Reporter: Thejas M Nair Assignee: Thejas M Nair Embedded metastore clients can't be retried, they throw an exception - "For direct MetaStore DB connections, we don't support retries at the client level." This tends to mask the real error that caused the attempts to retry. RetryingMetaStoreClient shouldn't even attempt to reconnect when direct/embedded metastore client is used. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
