[jira] [Created] (HIVE-18376) Update committer-list
Chris Drome created HIVE-18376:
--
Summary: Update committer-list
Key: HIVE-18376
URL: https://issues.apache.org/jira/browse/HIVE-18376
Project: Hive
Issue Type: Bug
Reporter: Chris Drome
Assignee: Chris Drome
Priority: Trivial
Adding new entry to committer-list:
{noformat}
+
+cdrome
+Chris Drome
+https://www.oath.com/;>Oath
+
{noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HIVE-17275) Auto-merge fails on writes of UNION ALL output to ORC file with dynamic partitioning
Chris Drome created HIVE-17275:
--
Summary: Auto-merge fails on writes of UNION ALL output to ORC
file with dynamic partitioning
Key: HIVE-17275
URL: https://issues.apache.org/jira/browse/HIVE-17275
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 2.2.0
Reporter: Chris Drome
Assignee: Chris Drome
If dynamic partitioning is used to write the output of UNION or UNION ALL
queries into ORC files with hive.merge.tezfiles=true, the merge step fails as
follows:
{noformat}
2017-08-08T11:27:19,958 ERROR [e7b1f06d-d632-408a-9dff-f7ae042cd25a main]
SessionState: Vertex failed, vertexName=File Merge,
vertexId=vertex_1502216690354_0001_33_00, diagnostics=[Task failed,
taskId=task_1502216690354_0001_33_00_00, diagnostics=[TaskAttempt 0 failed,
info=[Error: Error while running task ( failure ) :
attempt_1502216690354_0001_33_00_00_0:java.lang.RuntimeException:
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
java.io.IOException: Multiple partitions for one merge mapper:
hdfs://localhost:39943/build/ql/test/data/warehouse/partunion1/.hive-staging_hive_2017-08-08_11-27-09_105_286405133968521828-1/-ext-10002/part1=2014/1
NOT EQUAL TO
hdfs://localhost:39943/build/ql/test/data/warehouse/partunion1/.hive-staging_hive_2017-08-08_11-27-09_105_286405133968521828-1/-ext-10002/part1=2014/2
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:211)
at
org.apache.hadoop.hive.ql.exec.tez.MergeFileTezProcessor.run(MergeFileTezProcessor.java:42)
at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1807)
at
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
at
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: Multiple
partitions for one merge mapper:
hdfs://localhost:39943/build/ql/test/data/warehouse/partunion1/.hive-staging_hive_2017-08-08_11-27-09_105_286405133968521828-1/-ext-10002/part1=2014/1
NOT EQUAL TO
hdfs://localhost:39943/build/ql/test/data/warehouse/partunion1/.hive-staging_hive_2017-08-08_11-27-09_105_286405133968521828-1/-ext-10002/part1=2014/2
at
org.apache.hadoop.hive.ql.exec.tez.MergeFileRecordProcessor.processRow(MergeFileRecordProcessor.java:225)
at
org.apache.hadoop.hive.ql.exec.tez.MergeFileRecordProcessor.run(MergeFileRecordProcessor.java:154)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:185)
... 14 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.io.IOException: Multiple partitions for one merge mapper:
hdfs://localhost:39943/build/ql/test/data/warehouse/partunion1/.hive-staging_hive_2017-08-08_11-27-09_105_286405133968521828-1/-ext-10002/part1=2014/1
NOT EQUAL TO
hdfs://localhost:39943/build/ql/test/data/warehouse/partunion1/.hive-staging_hive_2017-08-08_11-27-09_105_286405133968521828-1/-ext-10002/part1=2014/2
at
org.apache.hadoop.hive.ql.exec.OrcFileMergeOperator.processKeyValuePairs(OrcFileMergeOperator.java:169)
at
org.apache.hadoop.hive.ql.exec.OrcFileMergeOperator.process(OrcFileMergeOperator.java:72)
at
org.apache.hadoop.hive.ql.exec.tez.MergeFileRecordProcessor.processRow(MergeFileRecordProcessor.java:216)
... 16 more
Caused by: java.io.IOException: Multiple partitions for one merge mapper:
hdfs://localhost:39943/build/ql/test/data/warehouse/partunion1/.hive-staging_hive_2017-08-08_11-27-09_105_286405133968521828-1/-ext-10002/part1=2014/1
NOT EQUAL TO
hdfs://localhost:39943/build/ql/test/data/warehouse/partunion1/.hive-staging_hive_2017-08-08_11-27-09_105_286405133968521828-1/-ext-10002/part1=2014/2
at
org.apache.hadoop.hive.ql.exec.AbstractFileMergeOperator.checkPartitionsMatch(AbstractFileMergeOperator.java:180)
at
org.apache.hadoop.hive.ql.exec.AbstractFileMergeOperator.fixTmpPath(AbstractFileMergeOperator.java:197)
at
[jira] [Created] (HIVE-14870) OracleStore: RawStore implementation optimized for Oracle
Chris Drome created HIVE-14870: -- Summary: OracleStore: RawStore implementation optimized for Oracle Key: HIVE-14870 URL: https://issues.apache.org/jira/browse/HIVE-14870 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Chris Drome Assignee: Chris Drome Attachments: OracleStoreDesignProposal.pdf -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-14344) Intermittent failures caused by leaking delegation tokens
Chris Drome created HIVE-14344:
--
Summary: Intermittent failures caused by leaking delegation tokens
Key: HIVE-14344
URL: https://issues.apache.org/jira/browse/HIVE-14344
Project: Hive
Issue Type: Bug
Components: Tez
Affects Versions: 2.1.0, 1.2.1
Reporter: Chris Drome
Assignee: Chris Drome
We have experienced random job failures caused by leaking delegation tokens.
The Tez child task will fail because it is attempting to read from the
delegation tokens directory of a different (related) task.
Failure results in the following type of stack trace:
{noformat}
2016-07-21 16:57:18,061 [FATAL] [TezChild] |tez.ReduceRecordSource|:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row (tag=0)
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:370)
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:292)
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:249)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:148)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:137)
at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:362)
at
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)
at
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1738)
at
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
at
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.RuntimeException: java.io.IOException: Exception reading
file:/grid/4/tmp/yarn-local/usercache/.../appcache/application_1468602386465_489814/container_e02_1468602386465_489814_01_01/container_tokens
at
org.apache.hadoop.hive.ql.exec.persistence.RowContainer.first(RowContainer.java:237)
at
org.apache.hadoop.hive.ql.exec.persistence.RowContainer.first(RowContainer.java:74)
at
org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genUniqueJoinObject(CommonJoinOperator.java:650)
at
org.apache.hadoop.hive.ql.exec.CommonJoinOperator.checkAndGenObject(CommonJoinOperator.java:756)
at
org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinObject(CommonMergeJoinOperator.java:316)
at
org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinOneGroup(CommonMergeJoinOperator.java:279)
at
org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinOneGroup(CommonMergeJoinOperator.java:272)
at
org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.process(CommonMergeJoinOperator.java:258)
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:361)
... 17 more
Caused by: java.lang.RuntimeException: java.io.IOException: Exception reading
file:/grid/4/tmp/yarn-local/usercache/.../appcache/application_1468602386465_489814/container_e02_1468602386465_489814_01_01/container_tokens
at
org.apache.hadoop.mapreduce.security.TokenCache.mergeBinaryTokens(TokenCache.java:141)
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:119)
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:100)
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:80)
at
org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:206)
at
org.apache.hadoop.mapred.SequenceFileInputFormat.listStatus(SequenceFileInputFormat.java:45)
at
org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:315)
at
org.apache.hadoop.hive.ql.exec.persistence.RowContainer.first(RowContainer.java:222)
... 25 more
Caused by: java.io.IOException: Exception reading
[jira] [Created] (HIVE-13990) Client should not check dfs.namenode.acls.enabled to determine if extended ACLs are supported
Chris Drome created HIVE-13990: -- Summary: Client should not check dfs.namenode.acls.enabled to determine if extended ACLs are supported Key: HIVE-13990 URL: https://issues.apache.org/jira/browse/HIVE-13990 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 1.2.1 Reporter: Chris Drome dfs.namenode.acls.enabled is a server side configuration and the client should not presume to know how the server is configured. Barring a method for querying the NN whether ACLs are supported the client should try and catch the appropriate exception. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-13989) Extended ACLs are not handled according to specification
Chris Drome created HIVE-13989: -- Summary: Extended ACLs are not handled according to specification Key: HIVE-13989 URL: https://issues.apache.org/jira/browse/HIVE-13989 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 1.2.1 Reporter: Chris Drome Assignee: Chris Drome -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-13756) Map failure attempts to delete reducer _temporary directory on multi-query pig query
Chris Drome created HIVE-13756: -- Summary: Map failure attempts to delete reducer _temporary directory on multi-query pig query Key: HIVE-13756 URL: https://issues.apache.org/jira/browse/HIVE-13756 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 2.0.0, 1.2.1 Reporter: Chris Drome Assignee: Chris Drome A pig script, executed with multi-query enabled, that reads the source data and writes it as-is into TABLE_A as well as performing a group-by operation on the data which is written into TABLE_B can produce erroneous results if any map fails. This results in a single MR job that writes the map output to a scratch directory relative to TABLE_A and the reducer output to a scratch directory relative to TABLE_B. If one or more maps fail it will delete the attempt data relative to TABLE_A, but it also deletes the _temporary directory relative to TABLE_B. This has the unintended side-effect of preventing subsequent maps from committing their data. This means that any maps which successfully completed before the first map failure will have its data committed as expected, other maps not, resulting in an incomplete result set. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-13754) Fix resource leak in HiveClientCache
Chris Drome created HIVE-13754:
--
Summary: Fix resource leak in HiveClientCache
Key: HIVE-13754
URL: https://issues.apache.org/jira/browse/HIVE-13754
Project: Hive
Issue Type: Bug
Components: Clients
Affects Versions: 2.0.0, 1.2.1
Reporter: Chris Drome
Assignee: Chris Drome
Found that the {{users}} reference count can go into negative values, which
prevents {{tearDownIfUnused}} from closing the client connection when called.
This leads to a build up of clients which have been evicted from the cache, are
no longer in use, but have not been shutdown.
GC will eventually call {{finalize}}, which forcibly closes the connection and
cleans up the client, but I have seen as many as several hundred open client
connections as a result.
The main resource for this is caused by RetryingMetaStoreClient, which will
call {{reconnect}} on acquire, which calls {{close}}. This will decrement
{{users}} to -1 on the reconnect, then acquire will increase this to 0 while
using it, and back to -1 when it releases it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-8414) HiveConnection.openSession should throw exception if session handle is null
Chris Drome created HIVE-8414: - Summary: HiveConnection.openSession should throw exception if session handle is null Key: HIVE-8414 URL: https://issues.apache.org/jira/browse/HIVE-8414 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 0.14.0, 0.15.0 Reporter: Chris Drome Assignee: Chris Drome Priority: Minor HiveConnection should verify the session handle in openSession and throw a SQLException if null. Ran into this problem by trying to send configs that change restricted params. Without a valid session handle, all subsequent operation with the connection will fail. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8414) HiveConnection.openSession should throw exception if session handle is null
[ https://issues.apache.org/jira/browse/HIVE-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-8414: -- Attachment: HIVE-8414-1.patch Attached preliminary fix. Will add unittest to exercise. HiveConnection.openSession should throw exception if session handle is null --- Key: HIVE-8414 URL: https://issues.apache.org/jira/browse/HIVE-8414 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 0.14.0, 0.15.0 Reporter: Chris Drome Assignee: Chris Drome Priority: Minor Attachments: HIVE-8414-1.patch HiveConnection should verify the session handle in openSession and throw a SQLException if null. Ran into this problem by trying to send configs that change restricted params. Without a valid session handle, all subsequent operation with the connection will fail. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-7195) Improve Metastore performance
[ https://issues.apache.org/jira/browse/HIVE-7195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030047#comment-14030047 ] Chris Drome commented on HIVE-7195: --- We ([~mithun], [~thiruvel], [~selinazh]) have done some work in this area for hive-0.12. Some of the improvements include: 1) Disabling the datanucleus cache to reduce the memory usage in the metastore. 2) Actively close datanucleus query-related resources to allow the memory the be reclaimed. 3) Optimizations to answer metadata-only queries directly from the metastore without launching MR jobs. 4) Optimizations to direct SQL statements. 5) Schema changes to speed up DROP TABLE statements. 6) Added client and server side parameters to restrict the maximum number of partitions that can be retrieved. We are currently looking into: 1) Reducing the client time required to retrieve HDFS file information. 2) Using light-weight partition objects where possible to reduce the time and memory on client/server. If I've forgotten anything Mithun, Thiruvel, or Selina can add more information. Improve Metastore performance - Key: HIVE-7195 URL: https://issues.apache.org/jira/browse/HIVE-7195 Project: Hive Issue Type: Improvement Reporter: Brock Noland Priority: Critical Even with direct SQL, which significantly improves MS performance, some operations take a considerable amount of time, when there are many partitions on table. Specifically I believe the issue: * When a client gets all partitions we do not send them an iterator, we create a collection of all data and then pass the object over the network in total * Operations which require looking up data on the NN can still be slow since there is no cache of information and it's done in a serial fashion * Perhaps a tangent, but our client timeout is quite dumb. The client will timeout and the server has no idea the client is gone. We should use deadlines, i.e. pass the timeout to the server so it can calculate that the client has expired. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-7189) Hive does not store column names in ORC
Chris Drome created HIVE-7189: - Summary: Hive does not store column names in ORC Key: HIVE-7189 URL: https://issues.apache.org/jira/browse/HIVE-7189 Project: Hive Issue Type: Bug Components: File Formats Affects Versions: 0.13.0, 0.12.0 Reporter: Chris Drome We uncovered the following discrepancy between writing ORC files through Pig and Hive: ORCFile header contains the name of the columns. Storing through Pig (ORCStorage or HCatStorer), the column names are stored fine. But when stored through hive they are stored as _col0, _col1,,_col99 and hive uses the partition schema to map the column names. Reading the same file through Pig then has problems as user will have to manually map columns. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6765) ASTNodeOrigin unserializable leads to fail when join with view
[ https://issues.apache.org/jira/browse/HIVE-6765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13989148#comment-13989148 ] Chris Drome commented on HIVE-6765: --- [~adrian-wang] this does not appear to be an issue with hive-0.13 as they are using Kryo for XML serialization. For clarification, I found that several setting can impact whether this problem arises or not. hive.auto.convert.join.noconditionaltask.size is used to determine the big table candidate set. Then hive.mapjoin.smalltable.filesize is used as a cutoff to determine whether a map-size join should be performed. I found that when hive.auto.convert.join is true, it will try to perform a map-side join first. Based on hive.auto.convert.join.noconditionaltask.size it will return a set of big table candidates to the physical optimizer. The physical optimizer will use hive.mapjoin.smalltable.filesize to determine whether the map-side join should proceed. If not the clonePlan method is called, which manifests the problem. I don't think it is solely influenced by the size of the tables involved in the join. In my tests, shrinking the size of the table allows the map-side join to proceed, while increasing the size of the table causes this failure. ASTNodeOrigin unserializable leads to fail when join with view -- Key: HIVE-6765 URL: https://issues.apache.org/jira/browse/HIVE-6765 Project: Hive Issue Type: Bug Affects Versions: 0.12.0 Reporter: Adrian Wang Fix For: 0.13.0 Attachments: HIVE-6765.patch.1 when a view contains a UDF, and the view comes into a JOIN operation, Hive will encounter a bug with stack trace like Caused by: java.lang.InstantiationException: org.apache.hadoop.hive.ql.parse.ASTNodeOrigin at java.lang.Class.newInstance0(Class.java:359) at java.lang.Class.newInstance(Class.java:327) at sun.reflect.GeneratedMethodAccessor84.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6987) Metastore qop settings won't work with Hadoop-2.4
[
https://issues.apache.org/jira/browse/HIVE-6987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13984728#comment-13984728
]
Chris Drome commented on HIVE-6987:
---
I have a preliminary patch for this issue. Will upload shortly.
Metastore qop settings won't work with Hadoop-2.4
-
Key: HIVE-6987
URL: https://issues.apache.org/jira/browse/HIVE-6987
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.14.0
Reporter: Vaibhav Gumashta
Fix For: 0.14.0
[HADOOP-10211|https://issues.apache.org/jira/browse/HADOOP-10211] made a
backward incompatible change due to which the following hive call returns a
null map:
{code}
MapString, String hadoopSaslProps = ShimLoader.getHadoopThriftAuthBridge().
getHadoopSaslProperties(conf);
{code}
Metastore uses the underlying hadoop.rpc.protection values to set the qop
between metastore client/server.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Assigned] (HIVE-6987) Metastore qop settings won't work with Hadoop-2.4
[
https://issues.apache.org/jira/browse/HIVE-6987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Drome reassigned HIVE-6987:
-
Assignee: Chris Drome
Metastore qop settings won't work with Hadoop-2.4
-
Key: HIVE-6987
URL: https://issues.apache.org/jira/browse/HIVE-6987
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.14.0
Reporter: Vaibhav Gumashta
Assignee: Chris Drome
Fix For: 0.14.0
[HADOOP-10211|https://issues.apache.org/jira/browse/HADOOP-10211] made a
backward incompatible change due to which the following hive call returns a
null map:
{code}
MapString, String hadoopSaslProps = ShimLoader.getHadoopThriftAuthBridge().
getHadoopSaslProperties(conf);
{code}
Metastore uses the underlying hadoop.rpc.protection values to set the qop
between metastore client/server.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6987) Metastore qop settings won't work with Hadoop-2.4
[
https://issues.apache.org/jira/browse/HIVE-6987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Drome updated HIVE-6987:
--
Attachment: HIVE-6987-1.patch.txt
Metastore qop settings won't work with Hadoop-2.4
-
Key: HIVE-6987
URL: https://issues.apache.org/jira/browse/HIVE-6987
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.14.0
Reporter: Vaibhav Gumashta
Assignee: Chris Drome
Fix For: 0.14.0
Attachments: HIVE-6987-1.patch.txt
[HADOOP-10211|https://issues.apache.org/jira/browse/HADOOP-10211] made a
backward incompatible change due to which the following hive call returns a
null map:
{code}
MapString, String hadoopSaslProps = ShimLoader.getHadoopThriftAuthBridge().
getHadoopSaslProperties(conf);
{code}
Metastore uses the underlying hadoop.rpc.protection values to set the qop
between metastore client/server.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6987) Metastore qop settings won't work with Hadoop-2.4
[
https://issues.apache.org/jira/browse/HIVE-6987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13984902#comment-13984902
]
Chris Drome commented on HIVE-6987:
---
You are correct. Sorry for the confusion.
Metastore qop settings won't work with Hadoop-2.4
-
Key: HIVE-6987
URL: https://issues.apache.org/jira/browse/HIVE-6987
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.14.0
Reporter: Vaibhav Gumashta
Assignee: Chris Drome
Fix For: 0.14.0
[HADOOP-10211|https://issues.apache.org/jira/browse/HADOOP-10211] made a
backward incompatible change due to which the following hive call returns a
null map:
{code}
MapString, String hadoopSaslProps = ShimLoader.getHadoopThriftAuthBridge().
getHadoopSaslProperties(conf);
{code}
Metastore uses the underlying hadoop.rpc.protection values to set the qop
between metastore client/server.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6987) Metastore qop settings won't work with Hadoop-2.4
[
https://issues.apache.org/jira/browse/HIVE-6987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Drome updated HIVE-6987:
--
Attachment: (was: HIVE-6987-1.patch.txt)
Metastore qop settings won't work with Hadoop-2.4
-
Key: HIVE-6987
URL: https://issues.apache.org/jira/browse/HIVE-6987
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.14.0
Reporter: Vaibhav Gumashta
Assignee: Chris Drome
Fix For: 0.14.0
[HADOOP-10211|https://issues.apache.org/jira/browse/HADOOP-10211] made a
backward incompatible change due to which the following hive call returns a
null map:
{code}
MapString, String hadoopSaslProps = ShimLoader.getHadoopThriftAuthBridge().
getHadoopSaslProperties(conf);
{code}
Metastore uses the underlying hadoop.rpc.protection values to set the qop
between metastore client/server.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6987) Metastore qop settings won't work with Hadoop-2.4
[
https://issues.apache.org/jira/browse/HIVE-6987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Drome updated HIVE-6987:
--
Assignee: (was: Chris Drome)
Metastore qop settings won't work with Hadoop-2.4
-
Key: HIVE-6987
URL: https://issues.apache.org/jira/browse/HIVE-6987
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.14.0
Reporter: Vaibhav Gumashta
Fix For: 0.14.0
[HADOOP-10211|https://issues.apache.org/jira/browse/HADOOP-10211] made a
backward incompatible change due to which the following hive call returns a
null map:
{code}
MapString, String hadoopSaslProps = ShimLoader.getHadoopThriftAuthBridge().
getHadoopSaslProperties(conf);
{code}
Metastore uses the underlying hadoop.rpc.protection values to set the qop
between metastore client/server.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6900) HostUtil.getTaskLogUrl signature change causes compilation to fail
[ https://issues.apache.org/jira/browse/HIVE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13969265#comment-13969265 ] Chris Drome commented on HIVE-6900: --- [~navis] I spoke with some Hadoop core engineers and they commented that Hive should not use HostUtil as it is marked as private (and unstable). Also, TaskLogServlet is not supported in MR2, so we just need to handle the case where mapreduce.framework.name != yarn (and yarn-tez I assume). We might want to add yarn-tez to the condition. HostUtil.getTaskLogUrl signature change causes compilation to fail -- Key: HIVE-6900 URL: https://issues.apache.org/jira/browse/HIVE-6900 Project: Hive Issue Type: Bug Components: Shims Affects Versions: 0.13.0, 0.14.0 Reporter: Chris Drome Attachments: HIVE-6900.1.patch.txt The signature for HostUtil.getTaskLogUrl has changed between Hadoop-2.3 and Hadoop-2.4. Code in shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java works with Hadoop-2.3 method and causes compilation failure with Hadoop-2.4. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-6900) HostUtil.getTaskLogUrl signature change causes compilation to fail
Chris Drome created HIVE-6900: - Summary: HostUtil.getTaskLogUrl signature change causes compilation to fail Key: HIVE-6900 URL: https://issues.apache.org/jira/browse/HIVE-6900 Project: Hive Issue Type: Bug Components: Shims Affects Versions: 0.13.0, 0.14.0 Reporter: Chris Drome The signature for HostUtil.getTaskLogUrl has changed between Hadoop-2.3 and Hadoop-2.4. Code in shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java works with Hadoop-2.3 method and causes compilation failure with Hadoop-2.4. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6447) Bucket map joins in hive-tez
[ https://issues.apache.org/jira/browse/HIVE-6447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13967287#comment-13967287 ] Chris Drome commented on HIVE-6447: --- Thanks for the clarification. We are using a newer version of Tez, which caused the problem. Bucket map joins in hive-tez Key: HIVE-6447 URL: https://issues.apache.org/jira/browse/HIVE-6447 Project: Hive Issue Type: Bug Components: Tez Affects Versions: tez-branch Reporter: Vikram Dixit K Assignee: Vikram Dixit K Fix For: 0.13.0, 0.14.0 Attachments: HIVE-6447.1.patch, HIVE-6447.10.patch, HIVE-6447.11.patch, HIVE-6447.12.patch, HIVE-6447.13.patch, HIVE-6447.2.patch, HIVE-6447.3.patch, HIVE-6447.4.patch, HIVE-6447.5.patch, HIVE-6447.6.patch, HIVE-6447.7.patch, HIVE-6447.8.patch, HIVE-6447.9.patch, HIVE-6447.WIP.patch Support bucket map joins in tez. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6029) Add default authorization on database/table creation
[ https://issues.apache.org/jira/browse/HIVE-6029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13850977#comment-13850977 ] Chris Drome commented on HIVE-6029: --- [~brocknoland] the initial patch was only intended for informational purposes as requested by [~thejas]. There is much more clean-up to be done, so please do not consider this yet. I will try to look at your rebased patch in the next couple of days. Thanks for reviewing. Add default authorization on database/table creation Key: HIVE-6029 URL: https://issues.apache.org/jira/browse/HIVE-6029 Project: Hive Issue Type: Improvement Components: Authorization, Metastore Affects Versions: 0.10.0 Reporter: Chris Drome Assignee: Chris Drome Priority: Minor Attachments: HIVE-6029-1.patch.txt, HIVE-6029.2.patch Default authorization privileges are not set when a database/table is created. This allows a user to create a database/table and not be able to access it through Sentry. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Created] (HIVE-6029) Add default authorization on database/table creation
Chris Drome created HIVE-6029: - Summary: Add default authorization on database/table creation Key: HIVE-6029 URL: https://issues.apache.org/jira/browse/HIVE-6029 Project: Hive Issue Type: Improvement Components: Authorization, Metastore Affects Versions: 0.10.0 Reporter: Chris Drome Assignee: Chris Drome Priority: Minor Default authorization privileges are not set when a database/table is created. This allows a user to create a database/table and not be able to access it through Sentry. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Updated] (HIVE-6029) Add default authorization on database/table creation
[ https://issues.apache.org/jira/browse/HIVE-6029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-6029: -- Attachment: HIVE-6029-1.patch.txt Initial patch which presents the idea. Needs more work and clean up. Generated against Hive-0.10. Add default authorization on database/table creation Key: HIVE-6029 URL: https://issues.apache.org/jira/browse/HIVE-6029 Project: Hive Issue Type: Improvement Components: Authorization, Metastore Affects Versions: 0.10.0 Reporter: Chris Drome Assignee: Chris Drome Priority: Minor Attachments: HIVE-6029-1.patch.txt Default authorization privileges are not set when a database/table is created. This allows a user to create a database/table and not be able to access it through Sentry. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HIVE-4974) JDBC2 statements and result sets are not able to return their parents
[ https://issues.apache.org/jira/browse/HIVE-4974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804738#comment-13804738 ] Chris Drome commented on HIVE-4974: --- Thanks [~brocknoland] and [~vaibhavgumashta]. JDBC2 statements and result sets are not able to return their parents - Key: HIVE-4974 URL: https://issues.apache.org/jira/browse/HIVE-4974 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 0.11.0, 0.12.0 Reporter: Chris Drome Assignee: Chris Drome Priority: Minor Fix For: 0.13.0 Attachments: HIVE-4974.2-trunk.patch.txt, HIVE-4974-trunk-1.patch.txt, HIVE-4974-trunk-2.patch.txt, HIVE-4974-trunk.patch.txt The getConnection method of HiveStatement and HivePreparedStatement throw a not supported SQLException. The constructors should take the HiveConnection that creates them as an argument. Similarly, HiveBaseResultSet is not capable of returning the Statement that created it. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-4974) JDBC2 statements and result sets are not able to return their parents
[ https://issues.apache.org/jira/browse/HIVE-4974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13802070#comment-13802070 ] Chris Drome commented on HIVE-4974: --- [~brocknoland], I didn't add explicit tests to TestJdbcDriver2 as there is no new functionality per se. I did test against TestJdbcDriver2 to ensure that nothing broke though. Please advise as to whether specific tests should be added. JDBC2 statements and result sets are not able to return their parents - Key: HIVE-4974 URL: https://issues.apache.org/jira/browse/HIVE-4974 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 0.11.0, 0.12.0 Reporter: Chris Drome Assignee: Chris Drome Priority: Minor Fix For: 0.13.0 Attachments: HIVE-4974-trunk.patch.txt The getConnection method of HiveStatement and HivePreparedStatement throw a not supported SQLException. The constructors should take the HiveConnection that creates them as an argument. Similarly, HiveBaseResultSet is not capable of returning the Statement that created it. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-4974) JDBC2 statements and result sets are not able to return their parents
[ https://issues.apache.org/jira/browse/HIVE-4974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13802099#comment-13802099 ] Chris Drome commented on HIVE-4974: --- Not a problem. I will add some tests and should have an updated patch later today or this evening. JDBC2 statements and result sets are not able to return their parents - Key: HIVE-4974 URL: https://issues.apache.org/jira/browse/HIVE-4974 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 0.11.0, 0.12.0 Reporter: Chris Drome Assignee: Chris Drome Priority: Minor Fix For: 0.13.0 Attachments: HIVE-4974-trunk.patch.txt The getConnection method of HiveStatement and HivePreparedStatement throw a not supported SQLException. The constructors should take the HiveConnection that creates them as an argument. Similarly, HiveBaseResultSet is not capable of returning the Statement that created it. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-4974) JDBC2 statements and result sets are not able to return their parents
[ https://issues.apache.org/jira/browse/HIVE-4974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4974: -- Attachment: HIVE-4974-trunk-1.patch.txt Fixed minor issue and added unittest. JDBC2 statements and result sets are not able to return their parents - Key: HIVE-4974 URL: https://issues.apache.org/jira/browse/HIVE-4974 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 0.11.0, 0.12.0 Reporter: Chris Drome Assignee: Chris Drome Priority: Minor Fix For: 0.13.0 Attachments: HIVE-4974-trunk-1.patch.txt, HIVE-4974-trunk.patch.txt The getConnection method of HiveStatement and HivePreparedStatement throw a not supported SQLException. The constructors should take the HiveConnection that creates them as an argument. Similarly, HiveBaseResultSet is not capable of returning the Statement that created it. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-4974) JDBC2 statements and result sets are not able to return their parents
[ https://issues.apache.org/jira/browse/HIVE-4974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4974: -- Attachment: HIVE-4974-trunk-2.patch.txt Changes to address phabricator comments. JDBC2 statements and result sets are not able to return their parents - Key: HIVE-4974 URL: https://issues.apache.org/jira/browse/HIVE-4974 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 0.11.0, 0.12.0 Reporter: Chris Drome Assignee: Chris Drome Priority: Minor Fix For: 0.13.0 Attachments: HIVE-4974-trunk-1.patch.txt, HIVE-4974-trunk-2.patch.txt, HIVE-4974-trunk.patch.txt The getConnection method of HiveStatement and HivePreparedStatement throw a not supported SQLException. The constructors should take the HiveConnection that creates them as an argument. Similarly, HiveBaseResultSet is not capable of returning the Statement that created it. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-4974) JDBC2 statements and result sets are not able to return their parents
[ https://issues.apache.org/jira/browse/HIVE-4974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4974: -- Attachment: HIVE-4974-trunk.patch.txt Uploaded trunk patch. JDBC2 statements and result sets are not able to return their parents - Key: HIVE-4974 URL: https://issues.apache.org/jira/browse/HIVE-4974 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 0.10.0, 0.11.0, 0.12.0 Reporter: Chris Drome Assignee: Chris Drome Priority: Minor Attachments: HIVE-4974-trunk.patch.txt The getConnection method of HiveStatement and HivePreparedStatement throw a not supported SQLException. The constructors should take the HiveConnection that creates them as an argument. Similarly, HiveBaseResultSet is not capable of returning the Statement that created it. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-4974) JDBC2 statements and result sets are not able to return their parents
[ https://issues.apache.org/jira/browse/HIVE-4974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13801272#comment-13801272 ] Chris Drome commented on HIVE-4974: --- Created phabricator ticket: https://reviews.facebook.net/D13611 JDBC2 statements and result sets are not able to return their parents - Key: HIVE-4974 URL: https://issues.apache.org/jira/browse/HIVE-4974 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 0.10.0, 0.11.0, 0.12.0 Reporter: Chris Drome Assignee: Chris Drome Priority: Minor Fix For: 0.13.0 Attachments: HIVE-4974-trunk.patch.txt The getConnection method of HiveStatement and HivePreparedStatement throw a not supported SQLException. The constructors should take the HiveConnection that creates them as an argument. Similarly, HiveBaseResultSet is not capable of returning the Statement that created it. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-4974) JDBC2 statements and result sets are not able to return their parents
[ https://issues.apache.org/jira/browse/HIVE-4974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4974: -- Fix Version/s: 0.13.0 Status: Patch Available (was: Open) JDBC2 statements and result sets are not able to return their parents - Key: HIVE-4974 URL: https://issues.apache.org/jira/browse/HIVE-4974 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 0.12.0, 0.11.0, 0.10.0 Reporter: Chris Drome Assignee: Chris Drome Priority: Minor Fix For: 0.13.0 Attachments: HIVE-4974-trunk.patch.txt The getConnection method of HiveStatement and HivePreparedStatement throw a not supported SQLException. The constructors should take the HiveConnection that creates them as an argument. Similarly, HiveBaseResultSet is not capable of returning the Statement that created it. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HIVE-4974) JDBC2 statements and result sets are not able to return their parents
[ https://issues.apache.org/jira/browse/HIVE-4974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4974: -- Affects Version/s: (was: 0.10.0) JDBC2 statements and result sets are not able to return their parents - Key: HIVE-4974 URL: https://issues.apache.org/jira/browse/HIVE-4974 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 0.11.0, 0.12.0 Reporter: Chris Drome Assignee: Chris Drome Priority: Minor Fix For: 0.13.0 Attachments: HIVE-4974-trunk.patch.txt The getConnection method of HiveStatement and HivePreparedStatement throw a not supported SQLException. The constructors should take the HiveConnection that creates them as an argument. Similarly, HiveBaseResultSet is not capable of returning the Statement that created it. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-3630) udf_substr.q fails when using JDK7
[ https://issues.apache.org/jira/browse/HIVE-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13737467#comment-13737467 ] Chris Drome commented on HIVE-3630: --- Sorry for jumping into the discussion late. Feel free to close this if it is no longer reproducible ([~ashutoshc]] thought that would be the case after HIVE-3840). udf_substr.q fails when using JDK7 -- Key: HIVE-3630 URL: https://issues.apache.org/jira/browse/HIVE-3630 Project: Hive Issue Type: Bug Components: Tests Affects Versions: 0.9.1, 0.10.0, 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Attachments: HIVE-3630-0.10.patch, HIVE-3630-0.9.patch, HIVE-3630-trunk.patch Internal error: Cannot find ConstantObjectInspector for BINARY This exception has two causes. JDK7 iterators do not return values in the same order as JDK6, which selects a different implementation of this UDF when the first argument is null. With JDK7 this happens to be the binary version. The binary version is not implemented properly which ultimately causes the exception when the method is called. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3688) Various tests failing in TestNegativeCliDriver, TestParseNegative, TestParse when using JDK7
[ https://issues.apache.org/jira/browse/HIVE-3688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13737469#comment-13737469 ] Chris Drome commented on HIVE-3688: --- [~brocknoland] that would be great. I'll remove the TestParse parts of this patch and resubmit for the TestNegativeCliDriver cases only. Thanks. Various tests failing in TestNegativeCliDriver, TestParseNegative, TestParse when using JDK7 Key: HIVE-3688 URL: https://issues.apache.org/jira/browse/HIVE-3688 Project: Hive Issue Type: Bug Components: Query Processor, Tests Affects Versions: 0.9.1, 0.10.0 Reporter: Chris Drome Assignee: Chris Drome Attachments: HIVE-3688-0.9.patch, HIVE-3688-trunk.patch The following tests are failing when using JDK7. TestNegativeCliDriver: case_sensitivity.q cast1.q groupby1.q groupby2.q groupby3.q groupby4.q groupby5.q groupby6.q input1.q input2.q input20.q input3.q input4.q input5.q input6.q input7.q input8.q input9.q input_part1.q input_testsequencefile.q input_testxpath.q input_testxpath2.q join1.q join2.q join3.q join4.q join5.q join6.q join7.q join8.q sample1.q sample2.q sample3.q sample4.q sample5.q sample6.q sample7.q subq.q udf1.q udf4.q udf6.q udf_case.q udf_when.q union.q TestParseNegative: invalid_function_param2.q TestNegativeCliDriver: fs_default_name1.q.out_0.23_1.7 fs_default_name2.q.out_0.23_1.7 invalid_cast_from_binary_1.q.out_0.23_1.7 invalid_cast_from_binary_2.q.out_0.23_1.7 invalid_cast_from_binary_3.q.out_0.23_1.7 invalid_cast_from_binary_4.q.out_0.23_1.7 invalid_cast_from_binary_5.q.out_0.23_1.7 invalid_cast_from_binary_6.q.out_0.23_1.7 wrong_column_type.q.out_0.23_1.7 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4974) JDBC2 statements are not able to return the connection that created them
Chris Drome created HIVE-4974: - Summary: JDBC2 statements are not able to return the connection that created them Key: HIVE-4974 URL: https://issues.apache.org/jira/browse/HIVE-4974 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 0.11.0, 0.10.0, 0.12.0 Reporter: Chris Drome Assignee: Chris Drome Priority: Minor The getConnection method of HiveStatement and HivePreparedStatement throw a not supported SQLException. The constructors should take the HiveConnection that creates them as an argument. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4974) JDBC2 statements and result sets are not able to return their parents
[ https://issues.apache.org/jira/browse/HIVE-4974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4974: -- Summary: JDBC2 statements and result sets are not able to return their parents (was: JDBC2 statements are not able to return the connection that created them) JDBC2 statements and result sets are not able to return their parents - Key: HIVE-4974 URL: https://issues.apache.org/jira/browse/HIVE-4974 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 0.10.0, 0.11.0, 0.12.0 Reporter: Chris Drome Assignee: Chris Drome Priority: Minor The getConnection method of HiveStatement and HivePreparedStatement throw a not supported SQLException. The constructors should take the HiveConnection that creates them as an argument. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4974) JDBC2 statements and result sets are not able to return their parents
[ https://issues.apache.org/jira/browse/HIVE-4974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4974: -- Description: The getConnection method of HiveStatement and HivePreparedStatement throw a not supported SQLException. The constructors should take the HiveConnection that creates them as an argument. Similarly, HiveBaseResultSet is not capable of returning the Statement that created it. was:The getConnection method of HiveStatement and HivePreparedStatement throw a not supported SQLException. The constructors should take the HiveConnection that creates them as an argument. JDBC2 statements and result sets are not able to return their parents - Key: HIVE-4974 URL: https://issues.apache.org/jira/browse/HIVE-4974 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 0.10.0, 0.11.0, 0.12.0 Reporter: Chris Drome Assignee: Chris Drome Priority: Minor The getConnection method of HiveStatement and HivePreparedStatement throw a not supported SQLException. The constructors should take the HiveConnection that creates them as an argument. Similarly, HiveBaseResultSet is not capable of returning the Statement that created it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4977) HS2: support an alternate serialization protocol between client and server
Chris Drome created HIVE-4977: - Summary: HS2: support an alternate serialization protocol between client and server Key: HIVE-4977 URL: https://issues.apache.org/jira/browse/HIVE-4977 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.11.0, 0.10.0, 0.12.0 Reporter: Chris Drome Assignee: Chris Drome Current serialization protocol between client and server as defined in cli_service.thrift results in 2x (or more) throughput degradation compared to HS1. Initial proposal is to introduce HS1 serialization protocol as a negotiable alternative. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4574) XMLEncoder thread safety issues in openjdk7 causes HiveServer2 to be stuck
[ https://issues.apache.org/jira/browse/HIVE-4574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13724939#comment-13724939 ] Chris Drome commented on HIVE-4574: --- Thanks for your thoughts. We are concerned with 0.10 at this point, but your point is taken. Looking forward to HIVE-1511! XMLEncoder thread safety issues in openjdk7 causes HiveServer2 to be stuck -- Key: HIVE-4574 URL: https://issues.apache.org/jira/browse/HIVE-4574 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.11.0 Reporter: Thejas M Nair Assignee: Thejas M Nair Attachments: HIVE-4574.1.patch In open jdk7, XMLEncoder.writeObject call leads to calls to java.beans.MethodFinder.findMethod(). MethodFinder class not thread safe because it uses a static WeakHashMap that would get used from multiple threads. See - http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/7-b147/com/sun/beans/finder/MethodFinder.java#46 Concurrent access to HashMap implementation that are not thread safe can sometimes result in infinite-loops and other problems. If jdk7 is in use, it makes sense to synchronize calls to XMLEncoder.writeObject . -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4574) XMLEncoder thread safety issues in openjdk7 causes HiveServer2 to be stuck
[ https://issues.apache.org/jira/browse/HIVE-4574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13724653#comment-13724653 ] Chris Drome commented on HIVE-4574: --- [~thejas], I was wondering why none of the other methods which use XMLEncoder are not synchronized as well. Is there something specific about serializeExpression that makes it different? XMLEncoder thread safety issues in openjdk7 causes HiveServer2 to be stuck -- Key: HIVE-4574 URL: https://issues.apache.org/jira/browse/HIVE-4574 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.11.0 Reporter: Thejas M Nair Assignee: Thejas M Nair Attachments: HIVE-4574.1.patch In open jdk7, XMLEncoder.writeObject call leads to calls to java.beans.MethodFinder.findMethod(). MethodFinder class not thread safe because it uses a static WeakHashMap that would get used from multiple threads. See - http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/7-b147/com/sun/beans/finder/MethodFinder.java#46 Concurrent access to HashMap implementation that are not thread safe can sometimes result in infinite-loops and other problems. If jdk7 is in use, it makes sense to synchronize calls to XMLEncoder.writeObject . -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4574) XMLEncoder thread safety issues in openjdk7 causes HiveServer2 to be stuck
[ https://issues.apache.org/jira/browse/HIVE-4574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13724679#comment-13724679 ] Chris Drome commented on HIVE-4574: --- Specifically, it appears that serializeMapRedWork and serializeMapRedLocalWork may also be called by hive. XMLEncoder thread safety issues in openjdk7 causes HiveServer2 to be stuck -- Key: HIVE-4574 URL: https://issues.apache.org/jira/browse/HIVE-4574 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.11.0 Reporter: Thejas M Nair Assignee: Thejas M Nair Attachments: HIVE-4574.1.patch In open jdk7, XMLEncoder.writeObject call leads to calls to java.beans.MethodFinder.findMethod(). MethodFinder class not thread safe because it uses a static WeakHashMap that would get used from multiple threads. See - http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/7-b147/com/sun/beans/finder/MethodFinder.java#46 Concurrent access to HashMap implementation that are not thread safe can sometimes result in infinite-loops and other problems. If jdk7 is in use, it makes sense to synchronize calls to XMLEncoder.writeObject . -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4574) XMLEncoder thread safety issues in openjdk7 causes HiveServer2 to be stuck
[ https://issues.apache.org/jira/browse/HIVE-4574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13724840#comment-13724840 ] Chris Drome commented on HIVE-4574: --- [~thejas], sorry I was unclear in my previous comment. I was concerned about about other methods in the Utilities.java class that similarly call writeObject on XMLEncoder. It doesn't look like we need to worry about Utilities.serializeTasks or Utilities.serializeQueryPlan. However, it looks like Utilities.serializeMapRedWork or Utilities.serializeMapRedLocalWork. Additionally, I'm not sure if we have to be concerned about deserialization which calls readObject. XMLEncoder thread safety issues in openjdk7 causes HiveServer2 to be stuck -- Key: HIVE-4574 URL: https://issues.apache.org/jira/browse/HIVE-4574 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.11.0 Reporter: Thejas M Nair Assignee: Thejas M Nair Attachments: HIVE-4574.1.patch In open jdk7, XMLEncoder.writeObject call leads to calls to java.beans.MethodFinder.findMethod(). MethodFinder class not thread safe because it uses a static WeakHashMap that would get used from multiple threads. See - http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/7-b147/com/sun/beans/finder/MethodFinder.java#46 Concurrent access to HashMap implementation that are not thread safe can sometimes result in infinite-loops and other problems. If jdk7 is in use, it makes sense to synchronize calls to XMLEncoder.writeObject . -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4911) Enable QOP configuration for Hive Server 2 thrift transport
[ https://issues.apache.org/jira/browse/HIVE-4911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13716145#comment-13716145 ] Chris Drome commented on HIVE-4911: --- [~brocknoland], I marked this patch as superceding HIVE-4225. HIVE-4225 only addresses the fact that HS2 was ignoring the hadoop.rpc.protection setting. The major limitation of HIVE-4225 is that it applies the QOP setting to both external and internal connections. HIVE-4911 improves upon this by allowing separate configuration of external and internal connections. An example of where this is important is when the HS2 client connection must be encrypted, but the connection between HS2 and JT/NN does not require encryption. Enable QOP configuration for Hive Server 2 thrift transport --- Key: HIVE-4911 URL: https://issues.apache.org/jira/browse/HIVE-4911 Project: Hive Issue Type: New Feature Reporter: Arup Malakar Assignee: Arup Malakar Attachments: HIVE-4911-trunk-0.patch The QoP for hive server 2 should be configurable to enable encryption. A new configuration should be exposed hive.server2.thrift.rpc.protection. This would give greater control configuring hive server 2 service. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4496) JDBC2 won't compile with JDK7
[ https://issues.apache.org/jira/browse/HIVE-4496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4496: -- Attachment: HIVE-4496-1.patch Rebased trunk patch. JDBC2 won't compile with JDK7 - Key: HIVE-4496 URL: https://issues.apache.org/jira/browse/HIVE-4496 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 0.12.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.12.0 Attachments: HIVE-4496-1.patch, HIVE-4496.patch HiveServer2 related JDBC does not compile with JDK7. Related to HIVE-3384. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4496) JDBC2 won't compile with JDK7
[ https://issues.apache.org/jira/browse/HIVE-4496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4496: -- Attachment: HIVE-4496-2.patch Restored the SQLException signature to include the original Exception in openSession. JDBC2 won't compile with JDK7 - Key: HIVE-4496 URL: https://issues.apache.org/jira/browse/HIVE-4496 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 0.12.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.12.0 Attachments: HIVE-4496-1.patch, HIVE-4496-2.patch, HIVE-4496.patch HiveServer2 related JDBC does not compile with JDK7. Related to HIVE-3384. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4496) JDBC2 won't compile with JDK7
[ https://issues.apache.org/jira/browse/HIVE-4496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13692364#comment-13692364 ] Chris Drome commented on HIVE-4496: --- [~navis]: Please try the latest patch. Thanks for your help on this patch. JDBC2 won't compile with JDK7 - Key: HIVE-4496 URL: https://issues.apache.org/jira/browse/HIVE-4496 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 0.12.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.12.0 Attachments: HIVE-4496-1.patch, HIVE-4496-2.patch, HIVE-4496.patch HiveServer2 related JDBC does not compile with JDK7. Related to HIVE-3384. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4496) JDBC2 won't compile with JDK7
[ https://issues.apache.org/jira/browse/HIVE-4496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13665824#comment-13665824 ] Chris Drome commented on HIVE-4496: --- [~xuefuz] I don't think the problem is with the patch. RevisionManagerFactory.java:81 generates the javadoc warning. This file is not part of this patch, it was imported as part of HIVE-4264. BTW, I just finished patching trunk and running ant tar, which completed successfully. JDBC2 won't compile with JDK7 - Key: HIVE-4496 URL: https://issues.apache.org/jira/browse/HIVE-4496 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 0.12.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.12.0 Attachments: HIVE-4496.patch HiveServer2 related JDBC does not compile with JDK7. Related to HIVE-3384. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4508) Fix various release issues in 0.11.0rc1
[ https://issues.apache.org/jira/browse/HIVE-4508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13649934#comment-13649934 ] Chris Drome commented on HIVE-4508: --- HIVE-4496 fixes the build issue with JDBC2 and JDK1.7. Fix various release issues in 0.11.0rc1 --- Key: HIVE-4508 URL: https://issues.apache.org/jira/browse/HIVE-4508 Project: Hive Issue Type: Bug Reporter: Owen O'Malley Assignee: Owen O'Malley Fix For: 0.11.0 Carl described some non-code issues in the 0.11.0rc1 and I want to fix them. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4496) JDBC2 won't compile with JDK7
Chris Drome created HIVE-4496: - Summary: JDBC2 won't compile with JDK7 Key: HIVE-4496 URL: https://issues.apache.org/jira/browse/HIVE-4496 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 0.12.0 Reporter: Chris Drome Assignee: Chris Drome HiveServer2 related JDBC does not compile with JDK7. Related to HIVE-3384. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3384) HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification
[ https://issues.apache.org/jira/browse/HIVE-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13648913#comment-13648913 ] Chris Drome commented on HIVE-3384: --- The error is not related to this patch. Rather it is associated with new code added in 0.11. Please refer to HIVE-4496. HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification -- Key: HIVE-3384 URL: https://issues.apache.org/jira/browse/HIVE-3384 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 0.10.0 Reporter: Weidong Bian Assignee: Chris Drome Priority: Minor Fix For: 0.11.0 Attachments: D6873-0.9.1.patch, D6873.1.patch, D6873.2.patch, D6873.3.patch, D6873.4.patch, D6873.5.patch, D6873.6.patch, D6873.7.patch, HIVE-3384-0.10.patch, HIVE-3384-2012-12-02.patch, HIVE-3384-2012-12-04.patch, HIVE-3384.2.patch, HIVE-3384-branch-0.9.patch, HIVE-3384.patch, HIVE-JDK7-JDBC.patch jdbc module couldn't be compiled with jdk7 as it adds some abstract method in the JDBC specification some error info: error: HiveCallableStatement is not abstract and does not override abstract method TgetObject(String,ClassT) in CallableStatement . . . -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4496) JDBC2 won't compile with JDK7
[ https://issues.apache.org/jira/browse/HIVE-4496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4496: -- Attachment: HIVE-4496.patch Attached trunk patch. JDBC2 won't compile with JDK7 - Key: HIVE-4496 URL: https://issues.apache.org/jira/browse/HIVE-4496 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 0.12.0 Reporter: Chris Drome Assignee: Chris Drome Attachments: HIVE-4496.patch HiveServer2 related JDBC does not compile with JDK7. Related to HIVE-3384. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4496) JDBC2 won't compile with JDK7
[ https://issues.apache.org/jira/browse/HIVE-4496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13648953#comment-13648953 ] Chris Drome commented on HIVE-4496: --- Phabricator ticket: https://reviews.facebook.net/D10647 JDBC2 won't compile with JDK7 - Key: HIVE-4496 URL: https://issues.apache.org/jira/browse/HIVE-4496 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 0.12.0 Reporter: Chris Drome Assignee: Chris Drome Attachments: HIVE-4496.patch HiveServer2 related JDBC does not compile with JDK7. Related to HIVE-3384. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4496) JDBC2 won't compile with JDK7
[ https://issues.apache.org/jira/browse/HIVE-4496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4496: -- Fix Version/s: 0.12.0 Status: Patch Available (was: Open) Ported the HIVE-3384 patch to the HS2 JDBC code. JDBC2 won't compile with JDK7 - Key: HIVE-4496 URL: https://issues.apache.org/jira/browse/HIVE-4496 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 0.12.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.12.0 Attachments: HIVE-4496.patch HiveServer2 related JDBC does not compile with JDK7. Related to HIVE-3384. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4467) HiveConnection does not handle failures correctly
[ https://issues.apache.org/jira/browse/HIVE-4467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13647025#comment-13647025 ] Chris Drome commented on HIVE-4467: --- [~cwsteinbach], the issue is with the condition in the verifySuccess method: if ((status.getStatusCode() != TStatusCode.SUCCESS_STATUS) (withInfo (status.getStatusCode() != TStatusCode.SUCCESS_WITH_INFO_STATUS))) when withInfo is false this condition will always be false. We have seen the case where the status code is an error and because withInfo is false there is no exception thrown. Thiruvel's patch addresses this case. HiveConnection does not handle failures correctly - Key: HIVE-4467 URL: https://issues.apache.org/jira/browse/HIVE-4467 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0, 0.12.0 Reporter: Thiruvel Thirumoolan Assignee: Thiruvel Thirumoolan Fix For: 0.11.0, 0.12.0 Attachments: HIVE-4467.patch HiveConnection uses Utils.verifySuccess* routines to check if there is any error from the server side. This is not handled well. In Utils.verifySuccess() when withInfo is 'false', the condition evaluates to 'false' and no SQLexception is thrown even though there could be a problem on the server. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4467) HiveConnection does not handle failures correctly
[ https://issues.apache.org/jira/browse/HIVE-4467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13647046#comment-13647046 ] Chris Drome commented on HIVE-4467: --- My bad. I misunderstood the context of your comment. HiveConnection does not handle failures correctly - Key: HIVE-4467 URL: https://issues.apache.org/jira/browse/HIVE-4467 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0, 0.12.0 Reporter: Thiruvel Thirumoolan Assignee: Thiruvel Thirumoolan Fix For: 0.11.0, 0.12.0 Attachments: HIVE-4467.patch HiveConnection uses Utils.verifySuccess* routines to check if there is any error from the server side. This is not handled well. In Utils.verifySuccess() when withInfo is 'false', the condition evaluates to 'false' and no SQLexception is thrown even though there could be a problem on the server. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4232) JDBC2 HiveConnection has odd defaults
[ https://issues.apache.org/jira/browse/HIVE-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4232: -- Attachment: HIVE-4232-4-trunk.patch HIVE-4232-4-0.11.patch Fixed classpath problems with 20S build. JDBC2 HiveConnection has odd defaults - Key: HIVE-4232 URL: https://issues.apache.org/jira/browse/HIVE-4232 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0, 0.12.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.11.0, 0.12.0 Attachments: HIVE-4232-1.patch, HIVE-4232-2.patch, HIVE-4232-3-0.11.patch, HIVE-4232-3-trunk.patch, HIVE-4232-4-0.11.patch, HIVE-4232-4-trunk.patch, HIVE-4232.patch HiveConnection defaults to using a plain SASL transport if auth is not set. To get a raw transport auth must be set to noSasl; furthermore noSasl is case sensitive. Code tries to infer Kerberos or plain authentication based on the presence of principal. There is no provision for specifying QOP level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4232) JDBC2 HiveConnection has odd defaults
[ https://issues.apache.org/jira/browse/HIVE-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4232: -- Affects Version/s: 0.12.0 Fix Version/s: 0.12.0 JDBC2 HiveConnection has odd defaults - Key: HIVE-4232 URL: https://issues.apache.org/jira/browse/HIVE-4232 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0, 0.12.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.11.0, 0.12.0 Attachments: HIVE-4232-1.patch, HIVE-4232-2.patch, HIVE-4232.patch HiveConnection defaults to using a plain SASL transport if auth is not set. To get a raw transport auth must be set to noSasl; furthermore noSasl is case sensitive. Code tries to infer Kerberos or plain authentication based on the presence of principal. There is no provision for specifying QOP level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4232) JDBC2 HiveConnection has odd defaults
[ https://issues.apache.org/jira/browse/HIVE-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4232: -- Attachment: HIVE-4232-trunk-3.patch HIVE-4232-0.11-3.patch New patch fixes test failure. JDBC2 HiveConnection has odd defaults - Key: HIVE-4232 URL: https://issues.apache.org/jira/browse/HIVE-4232 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0, 0.12.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.11.0, 0.12.0 Attachments: HIVE-4232-0.11-3.patch, HIVE-4232-1.patch, HIVE-4232-2.patch, HIVE-4232.patch, HIVE-4232-trunk-3.patch HiveConnection defaults to using a plain SASL transport if auth is not set. To get a raw transport auth must be set to noSasl; furthermore noSasl is case sensitive. Code tries to infer Kerberos or plain authentication based on the presence of principal. There is no provision for specifying QOP level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4232) JDBC2 HiveConnection has odd defaults
[ https://issues.apache.org/jira/browse/HIVE-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4232: -- Attachment: (was: HIVE-4232-0.11-3.patch) JDBC2 HiveConnection has odd defaults - Key: HIVE-4232 URL: https://issues.apache.org/jira/browse/HIVE-4232 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0, 0.12.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.11.0, 0.12.0 Attachments: HIVE-4232-1.patch, HIVE-4232-2.patch, HIVE-4232.patch HiveConnection defaults to using a plain SASL transport if auth is not set. To get a raw transport auth must be set to noSasl; furthermore noSasl is case sensitive. Code tries to infer Kerberos or plain authentication based on the presence of principal. There is no provision for specifying QOP level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4232) JDBC2 HiveConnection has odd defaults
[ https://issues.apache.org/jira/browse/HIVE-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4232: -- Attachment: HIVE-4232-0.11-3.patch Missed a file. JDBC2 HiveConnection has odd defaults - Key: HIVE-4232 URL: https://issues.apache.org/jira/browse/HIVE-4232 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0, 0.12.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.11.0, 0.12.0 Attachments: HIVE-4232-1.patch, HIVE-4232-2.patch, HIVE-4232.patch HiveConnection defaults to using a plain SASL transport if auth is not set. To get a raw transport auth must be set to noSasl; furthermore noSasl is case sensitive. Code tries to infer Kerberos or plain authentication based on the presence of principal. There is no provision for specifying QOP level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4232) JDBC2 HiveConnection has odd defaults
[ https://issues.apache.org/jira/browse/HIVE-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4232: -- Attachment: (was: HIVE-4232-0.11-3.patch) JDBC2 HiveConnection has odd defaults - Key: HIVE-4232 URL: https://issues.apache.org/jira/browse/HIVE-4232 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0, 0.12.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.11.0, 0.12.0 Attachments: HIVE-4232-1.patch, HIVE-4232-2.patch, HIVE-4232.patch HiveConnection defaults to using a plain SASL transport if auth is not set. To get a raw transport auth must be set to noSasl; furthermore noSasl is case sensitive. Code tries to infer Kerberos or plain authentication based on the presence of principal. There is no provision for specifying QOP level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4232) JDBC2 HiveConnection has odd defaults
[ https://issues.apache.org/jira/browse/HIVE-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4232: -- Attachment: (was: HIVE-4232-trunk-3.patch) JDBC2 HiveConnection has odd defaults - Key: HIVE-4232 URL: https://issues.apache.org/jira/browse/HIVE-4232 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0, 0.12.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.11.0, 0.12.0 Attachments: HIVE-4232-1.patch, HIVE-4232-2.patch, HIVE-4232.patch HiveConnection defaults to using a plain SASL transport if auth is not set. To get a raw transport auth must be set to noSasl; furthermore noSasl is case sensitive. Code tries to infer Kerberos or plain authentication based on the presence of principal. There is no provision for specifying QOP level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4232) JDBC2 HiveConnection has odd defaults
[ https://issues.apache.org/jira/browse/HIVE-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4232: -- Attachment: HIVE-4232-3-trunk.patch HIVE-4232-3-0.11.patch Uploaded renamed files. JDBC2 HiveConnection has odd defaults - Key: HIVE-4232 URL: https://issues.apache.org/jira/browse/HIVE-4232 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0, 0.12.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.11.0, 0.12.0 Attachments: HIVE-4232-1.patch, HIVE-4232-2.patch, HIVE-4232-3-0.11.patch, HIVE-4232-3-trunk.patch, HIVE-4232.patch HiveConnection defaults to using a plain SASL transport if auth is not set. To get a raw transport auth must be set to noSasl; furthermore noSasl is case sensitive. Code tries to infer Kerberos or plain authentication based on the presence of principal. There is no provision for specifying QOP level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4232) JDBC2 HiveConnection has odd defaults
[ https://issues.apache.org/jira/browse/HIVE-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4232: -- Attachment: HIVE-4232-2.patch WIP patch incorporating the comments. NOSASL/NONE transport layer test failing with latest trunk code. JDBC2 HiveConnection has odd defaults - Key: HIVE-4232 URL: https://issues.apache.org/jira/browse/HIVE-4232 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.11.0 Attachments: HIVE-4232-1.patch, HIVE-4232-2.patch, HIVE-4232.patch HiveConnection defaults to using a plain SASL transport if auth is not set. To get a raw transport auth must be set to noSasl; furthermore noSasl is case sensitive. Code tries to infer Kerberos or plain authentication based on the presence of principal. There is no provision for specifying QOP level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4236) JDBC2 HivePreparedStatement does not release resources
[ https://issues.apache.org/jira/browse/HIVE-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634059#comment-13634059 ] Chris Drome commented on HIVE-4236: --- [~navis]: I will start addressing your comments, etc when I'm back in the office next week. JDBC2 HivePreparedStatement does not release resources -- Key: HIVE-4236 URL: https://issues.apache.org/jira/browse/HIVE-4236 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.11.0 Attachments: HIVE-4236.patch HivePreparedStatement does not close the associated server-side operation when close() is called. Nor does it call close() on the ResultSet. When execute() is called the current ResultSet is not closed first it is just set to null. Similarly, HiveStatement's close() does not call close() on the ResultSet, it just sets it to null. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4232) JDBC2 HiveConnection has odd defaults
[ https://issues.apache.org/jira/browse/HIVE-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4232: -- Attachment: HIVE-4232-2.patch Updated patch with your suggestions. Added some test cases. The test cases passed on Hadoop-0.23 and Hadoop-2.0. They failed on Hadoop-1.0 because of classpath issues that use conflicting versions of slf4j at runtime. I didn't have time to fix this. I think Thiruvel will be able to handle any blockers while I'm away. JDBC2 HiveConnection has odd defaults - Key: HIVE-4232 URL: https://issues.apache.org/jira/browse/HIVE-4232 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.11.0 Attachments: HIVE-4232-1.patch, HIVE-4232-2.patch, HIVE-4232.patch HiveConnection defaults to using a plain SASL transport if auth is not set. To get a raw transport auth must be set to noSasl; furthermore noSasl is case sensitive. Code tries to infer Kerberos or plain authentication based on the presence of principal. There is no provision for specifying QOP level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4232) JDBC2 HiveConnection has odd defaults
[ https://issues.apache.org/jira/browse/HIVE-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4232: -- Attachment: (was: HIVE-4232-2.patch) JDBC2 HiveConnection has odd defaults - Key: HIVE-4232 URL: https://issues.apache.org/jira/browse/HIVE-4232 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.11.0 Attachments: HIVE-4232-1.patch, HIVE-4232.patch HiveConnection defaults to using a plain SASL transport if auth is not set. To get a raw transport auth must be set to noSasl; furthermore noSasl is case sensitive. Code tries to infer Kerberos or plain authentication based on the presence of principal. There is no provision for specifying QOP level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4232) JDBC2 HiveConnection has odd defaults
[ https://issues.apache.org/jira/browse/HIVE-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13621923#comment-13621923 ] Chris Drome commented on HIVE-4232: --- [~prasadm][~cwsteinbach]: I think Prasad's point is fair. It explains why the defaults were set the way they were and should probably be documented somewhere in the code. We have a follow-up patch which adds functionality to set the QOP of the transport, so that data encryption can be enabled on the client-server transport. This will require some changes to the JDBC connection string and I wanted to get buy-in to modify the existing format of the connection string before posting that patch. I still feel the naming conventions are a little misleading and the JDBC connection string tries to infer the state. However, if everyone agrees that NOSASL should generally not be used, then I won't push that issue. Although it doesn't help in the case a client library adds auth=nosasl, resulting in the connection hanging. JDBC2 HiveConnection has odd defaults - Key: HIVE-4232 URL: https://issues.apache.org/jira/browse/HIVE-4232 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.11.0 Attachments: HIVE-4232-1.patch, HIVE-4232.patch HiveConnection defaults to using a plain SASL transport if auth is not set. To get a raw transport auth must be set to noSasl; furthermore noSasl is case sensitive. Code tries to infer Kerberos or plain authentication based on the presence of principal. There is no provision for specifying QOP level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4232) JDBC2 HiveConnection has odd defaults
[ https://issues.apache.org/jira/browse/HIVE-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13622931#comment-13622931 ] Chris Drome commented on HIVE-4232: --- I was thinking about this a little more and would like to summarize the current state: hive-site.xml - transport - JDBC connection string 1. hive.server2.authentication=NOSASL - raw transport - jdbc:hive2://host:port/dbname;auth=noSasl 2. hive.server2.authentication=NONE - plain SASL transport - jdbc:hive2://host:port/dbname (*DEFAULT*) 3. hive.server2.authentication=KERBEROS - Kerberos SASL transport - jdbc:hive2://host:port/dbname;principal=principal So we need to set auth=noSasl to disable SASL instead of auth=plainsasl|kerberos to enable SASL. Now if the server is set to Kerberos, then the client still has to know this because it if they exclude the principal parameter it will happily initialize a plain SASL transport because the code infers SASL/Kerberos based on the existence of that value. Granted, in this case the connection throws an exception. If it was a requirement to include say auth=plainsasl|kerberos in the connection string, then the code wouldn't have to guess what the intent was and could communicate an error if auth=kerberos and principal is not present. Furthermore, from what I can see, the plain SASL is really just a no-op as it is not doing anything when authentication=NONE. In the end, it comes down to the fact that the client still needs to know the authentication method of the server. That is why I asked whether we had data about the average use case. Do we know how often people are using raw vs plain vs kerberos? It seems that the real issue is that a SASL transport doesn't play nicely with a raw transport. If it would be possible to remove the need to support the raw transport, that would resolve this discussion. Even the raw transport much speak thrift, so that isn't an issue. Rather it is a question of whether client should be forced to implement a SASL transport. JDBC2 HiveConnection has odd defaults - Key: HIVE-4232 URL: https://issues.apache.org/jira/browse/HIVE-4232 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.11.0 Attachments: HIVE-4232-1.patch, HIVE-4232.patch HiveConnection defaults to using a plain SASL transport if auth is not set. To get a raw transport auth must be set to noSasl; furthermore noSasl is case sensitive. Code tries to infer Kerberos or plain authentication based on the presence of principal. There is no provision for specifying QOP level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4232) JDBC2 HiveConnection has odd defaults
[ https://issues.apache.org/jira/browse/HIVE-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13620702#comment-13620702 ] Chris Drome commented on HIVE-4232: --- In the process of writing some unit tests for the connection string changes I noticed that the connection type for HiveServer2 defaults to NONE, which uses the Plain SASL Transport Factory. So in this sense the original implementation of HiveConnection matches for the default case. hive.server2.authentication can take one of the following values: NOSASL NONE (default) LDAP KERBEROS CUSTOM I think NONE is misleading and should be renamed PLAIN (or something similar), and NOSASL should be labelled NONE. And that the default usage of Plain SASL Transport Factory seems odd. The default behavior of HiveConnection should match that of HiveServer2. I'd like to ask your opinion of what the default behavior should be. Should it default to the Plain SASL transport or no authentication at all? JDBC2 HiveConnection has odd defaults - Key: HIVE-4232 URL: https://issues.apache.org/jira/browse/HIVE-4232 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.11.0 Attachments: HIVE-4232.patch HiveConnection defaults to using a plain SASL transport if auth is not set. To get a raw transport auth must be set to noSasl; furthermore noSasl is case sensitive. Code tries to infer Kerberos or plain authentication based on the presence of principal. There is no provision for specifying QOP level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4232) JDBC2 HiveConnection has odd defaults
[ https://issues.apache.org/jira/browse/HIVE-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4232: -- Attachment: HIVE-4232-1.patch Changes associated with previous comment. JDBC2 HiveConnection has odd defaults - Key: HIVE-4232 URL: https://issues.apache.org/jira/browse/HIVE-4232 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.11.0 Attachments: HIVE-4232-1.patch, HIVE-4232.patch HiveConnection defaults to using a plain SASL transport if auth is not set. To get a raw transport auth must be set to noSasl; furthermore noSasl is case sensitive. Code tries to infer Kerberos or plain authentication based on the presence of principal. There is no provision for specifying QOP level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4232) JDBC2 HiveConnection has odd defaults
[ https://issues.apache.org/jira/browse/HIVE-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13620756#comment-13620756 ] Chris Drome commented on HIVE-4232: --- Working on some test cases, but need to resolve a classpath issue before I can include them. JDBC2 HiveConnection has odd defaults - Key: HIVE-4232 URL: https://issues.apache.org/jira/browse/HIVE-4232 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.11.0 Attachments: HIVE-4232-1.patch, HIVE-4232.patch HiveConnection defaults to using a plain SASL transport if auth is not set. To get a raw transport auth must be set to noSasl; furthermore noSasl is case sensitive. Code tries to infer Kerberos or plain authentication based on the presence of principal. There is no provision for specifying QOP level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4232) JDBC2 HiveConnection has odd defaults
[ https://issues.apache.org/jira/browse/HIVE-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619624#comment-13619624 ] Chris Drome commented on HIVE-4232: --- Phabricator link: https://reviews.facebook.net/D9879 JDBC2 HiveConnection has odd defaults - Key: HIVE-4232 URL: https://issues.apache.org/jira/browse/HIVE-4232 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.11.0 Attachments: HIVE-4232.patch HiveConnection defaults to using a plain SASL transport if auth is not set. To get a raw transport auth must be set to noSasl; furthermore noSasl is case sensitive. Code tries to infer Kerberos or plain authentication based on the presence of principal. There is no provision for specifying QOP level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4232) JDBC2 HiveConnection has odd defaults
[ https://issues.apache.org/jira/browse/HIVE-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13620118#comment-13620118 ] Chris Drome commented on HIVE-4232: --- Hi Ashutosh, I will be out of the office from Thursday, so I was wondering if I could get your feedback on the HIVE-4232 patch as soon as possible. That way I would be able to make changes or address concerns before I leave. I've created the fabricator ticket. https://reviews.facebook.net/D9879 Thanks, chris JDBC2 HiveConnection has odd defaults - Key: HIVE-4232 URL: https://issues.apache.org/jira/browse/HIVE-4232 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.11.0 Attachments: HIVE-4232.patch HiveConnection defaults to using a plain SASL transport if auth is not set. To get a raw transport auth must be set to noSasl; furthermore noSasl is case sensitive. Code tries to infer Kerberos or plain authentication based on the presence of principal. There is no provision for specifying QOP level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4166) closeAllForUGI causes failure in hiveserver2 when fetching large amount of data
[ https://issues.apache.org/jira/browse/HIVE-4166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4166: -- Attachment: (was: HIVE-4166-trunk.patch) closeAllForUGI causes failure in hiveserver2 when fetching large amount of data --- Key: HIVE-4166 URL: https://issues.apache.org/jira/browse/HIVE-4166 Project: Hive Issue Type: Bug Components: Security, Shims Affects Versions: 0.10.0, 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Attachments: HIVE-4166-trunk.patch HiveServer2 configured to use Kerberos authentication with doAs enabled throws an exception when fetching a large amount of data from a query. The exception is caused because FileSystem.closeAllForUGI is always called at the end of TUGIAssumingProcessor.process. This affects requests on the ResultSet for data from a SELECT query when the amount of data exceeds a certain size. At that point any subsequent calls to fetch more data throw an exception because the underlying DFSClient has been closed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4166) closeAllForUGI causes failure in hiveserver2 when fetching large amount of data
[ https://issues.apache.org/jira/browse/HIVE-4166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4166: -- Attachment: (was: HIVE-4166-0.10.patch) closeAllForUGI causes failure in hiveserver2 when fetching large amount of data --- Key: HIVE-4166 URL: https://issues.apache.org/jira/browse/HIVE-4166 Project: Hive Issue Type: Bug Components: Security, Shims Affects Versions: 0.10.0, 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Attachments: HIVE-4166-trunk.patch HiveServer2 configured to use Kerberos authentication with doAs enabled throws an exception when fetching a large amount of data from a query. The exception is caused because FileSystem.closeAllForUGI is always called at the end of TUGIAssumingProcessor.process. This affects requests on the ResultSet for data from a SELECT query when the amount of data exceeds a certain size. At that point any subsequent calls to fetch more data throw an exception because the underlying DFSClient has been closed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4166) closeAllForUGI causes failure in hiveserver2 when fetching large amount of data
[ https://issues.apache.org/jira/browse/HIVE-4166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4166: -- Attachment: HIVE-4166-trunk.patch Updated trunk patch. closeAllForUGI causes failure in hiveserver2 when fetching large amount of data --- Key: HIVE-4166 URL: https://issues.apache.org/jira/browse/HIVE-4166 Project: Hive Issue Type: Bug Components: Security, Shims Affects Versions: 0.10.0, 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Attachments: HIVE-4166-trunk.patch HiveServer2 configured to use Kerberos authentication with doAs enabled throws an exception when fetching a large amount of data from a query. The exception is caused because FileSystem.closeAllForUGI is always called at the end of TUGIAssumingProcessor.process. This affects requests on the ResultSet for data from a SELECT query when the amount of data exceeds a certain size. At that point any subsequent calls to fetch more data throw an exception because the underlying DFSClient has been closed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4166) closeAllForUGI causes failure in hiveserver2 when fetching large amount of data
[ https://issues.apache.org/jira/browse/HIVE-4166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4166: -- Component/s: HiveServer2 closeAllForUGI causes failure in hiveserver2 when fetching large amount of data --- Key: HIVE-4166 URL: https://issues.apache.org/jira/browse/HIVE-4166 Project: Hive Issue Type: Bug Components: HiveServer2, Security, Shims Affects Versions: 0.10.0, 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Attachments: HIVE-4166-0.10.patch, HIVE-4166-trunk.patch HiveServer2 configured to use Kerberos authentication with doAs enabled throws an exception when fetching a large amount of data from a query. The exception is caused because FileSystem.closeAllForUGI is always called at the end of TUGIAssumingProcessor.process. This affects requests on the ResultSet for data from a SELECT query when the amount of data exceeds a certain size. At that point any subsequent calls to fetch more data throw an exception because the underlying DFSClient has been closed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4166) closeAllForUGI causes failure in hiveserver2 when fetching large amount of data
[ https://issues.apache.org/jira/browse/HIVE-4166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4166: -- Status: Patch Available (was: Open) Fixes the problem in HiveServer2 which causes certain operations to fail by closing the FileSystem when the connection is closed. closeAllForUGI causes failure in hiveserver2 when fetching large amount of data --- Key: HIVE-4166 URL: https://issues.apache.org/jira/browse/HIVE-4166 Project: Hive Issue Type: Bug Components: HiveServer2, Security, Shims Affects Versions: 0.10.0, 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Attachments: HIVE-4166-0.10.patch, HIVE-4166-trunk.patch HiveServer2 configured to use Kerberos authentication with doAs enabled throws an exception when fetching a large amount of data from a query. The exception is caused because FileSystem.closeAllForUGI is always called at the end of TUGIAssumingProcessor.process. This affects requests on the ResultSet for data from a SELECT query when the amount of data exceeds a certain size. At that point any subsequent calls to fetch more data throw an exception because the underlying DFSClient has been closed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4166) closeAllForUGI causes failure in hiveserver2 when fetching large amount of data
[ https://issues.apache.org/jira/browse/HIVE-4166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4166: -- Fix Version/s: 0.10.0 0.11.0 closeAllForUGI causes failure in hiveserver2 when fetching large amount of data --- Key: HIVE-4166 URL: https://issues.apache.org/jira/browse/HIVE-4166 Project: Hive Issue Type: Bug Components: HiveServer2, Security, Shims Affects Versions: 0.10.0, 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.10.0, 0.11.0 Attachments: HIVE-4166-0.10.patch, HIVE-4166-trunk.patch HiveServer2 configured to use Kerberos authentication with doAs enabled throws an exception when fetching a large amount of data from a query. The exception is caused because FileSystem.closeAllForUGI is always called at the end of TUGIAssumingProcessor.process. This affects requests on the ResultSet for data from a SELECT query when the amount of data exceeds a certain size. At that point any subsequent calls to fetch more data throw an exception because the underlying DFSClient has been closed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4256) JDBC2 HiveConnection does not use the specified database
Chris Drome created HIVE-4256: - Summary: JDBC2 HiveConnection does not use the specified database Key: HIVE-4256 URL: https://issues.apache.org/jira/browse/HIVE-4256 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0 Reporter: Chris Drome Assignee: Chris Drome HiveConnection ignores the database specified in the connection string when configuring the connection. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4236) JDBC2 HivePreparedStatement does not release resources
Chris Drome created HIVE-4236: - Summary: JDBC2 HivePreparedStatement does not release resources Key: HIVE-4236 URL: https://issues.apache.org/jira/browse/HIVE-4236 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0 Reporter: Chris Drome HivePreparedStatement does not close the associated server-side operation when close() is called. Nor does it call close() on the ResultSet. When execute() is called the current ResultSet is not closed first it is just set to null. Similarly, HiveStatement's close() does not call close() on the ResultSet, it just sets it to null. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4236) JDBC2 HivePreparedStatement does not release resources
[ https://issues.apache.org/jira/browse/HIVE-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4236: -- Assignee: Chris Drome JDBC2 HivePreparedStatement does not release resources -- Key: HIVE-4236 URL: https://issues.apache.org/jira/browse/HIVE-4236 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0 Reporter: Chris Drome Assignee: Chris Drome HivePreparedStatement does not close the associated server-side operation when close() is called. Nor does it call close() on the ResultSet. When execute() is called the current ResultSet is not closed first it is just set to null. Similarly, HiveStatement's close() does not call close() on the ResultSet, it just sets it to null. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4236) JDBC2 HivePreparedStatement does not release resources
[ https://issues.apache.org/jira/browse/HIVE-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4236: -- Attachment: HIVE-4236.patch Uploading patch which closes ResultSet and frees server-side resources. JDBC2 HivePreparedStatement does not release resources -- Key: HIVE-4236 URL: https://issues.apache.org/jira/browse/HIVE-4236 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Attachments: HIVE-4236.patch HivePreparedStatement does not close the associated server-side operation when close() is called. Nor does it call close() on the ResultSet. When execute() is called the current ResultSet is not closed first it is just set to null. Similarly, HiveStatement's close() does not call close() on the ResultSet, it just sets it to null. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4236) JDBC2 HivePreparedStatement does not release resources
[ https://issues.apache.org/jira/browse/HIVE-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4236: -- Fix Version/s: 0.11.0 Status: Patch Available (was: Open) JDBC2 HivePreparedStatement does not release resources -- Key: HIVE-4236 URL: https://issues.apache.org/jira/browse/HIVE-4236 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.11.0 Attachments: HIVE-4236.patch HivePreparedStatement does not close the associated server-side operation when close() is called. Nor does it call close() on the ResultSet. When execute() is called the current ResultSet is not closed first it is just set to null. Similarly, HiveStatement's close() does not call close() on the ResultSet, it just sets it to null. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4232) JDBC2 HiveConnection has odd defaults
Chris Drome created HIVE-4232: - Summary: JDBC2 HiveConnection has odd defaults Key: HIVE-4232 URL: https://issues.apache.org/jira/browse/HIVE-4232 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0 Reporter: Chris Drome HiveConnection defaults to using a plain SASL transport if auth is not set. To get a raw transport auth must be set to noSasl; furthermore noSasl is case sensitive. Code tries to infer Kerberos or plain authentication based on the presence of principal. There is no provision for specifying QOP level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4232) JDBC2 HiveConnection has odd defaults
[ https://issues.apache.org/jira/browse/HIVE-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4232: -- Assignee: Chris Drome JDBC2 HiveConnection has odd defaults - Key: HIVE-4232 URL: https://issues.apache.org/jira/browse/HIVE-4232 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0 Reporter: Chris Drome Assignee: Chris Drome HiveConnection defaults to using a plain SASL transport if auth is not set. To get a raw transport auth must be set to noSasl; furthermore noSasl is case sensitive. Code tries to infer Kerberos or plain authentication based on the presence of principal. There is no provision for specifying QOP level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4232) JDBC2 HiveConnection has odd defaults
[ https://issues.apache.org/jira/browse/HIVE-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13613592#comment-13613592 ] Chris Drome commented on HIVE-4232: --- The patch proposes that if auth is not specified or auth=none the transport will default to TSocket. If auth=kerberos|plain then the Kerberos SASL transport or the Plain SASL transport will be used. If auth=kerberos then principal must also be specified. We propose controlling the QOP with another parameter qop=auth|auth-int|auth-conf. The patch also takes care of case-sensitive value comparisons. I feel that these changes result in a more reasonable set of defaults and don't rely upon inferring Kerberos or plain based on the existence of other parameters. I will rebase HIVE-4225 if these proposed changes are accepted. JDBC2 HiveConnection has odd defaults - Key: HIVE-4232 URL: https://issues.apache.org/jira/browse/HIVE-4232 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Attachments: HIVE-4232.patch HiveConnection defaults to using a plain SASL transport if auth is not set. To get a raw transport auth must be set to noSasl; furthermore noSasl is case sensitive. Code tries to infer Kerberos or plain authentication based on the presence of principal. There is no provision for specifying QOP level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4232) JDBC2 HiveConnection has odd defaults
[ https://issues.apache.org/jira/browse/HIVE-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4232: -- Attachment: HIVE-4232.patch Uploaded patch which implements the proposed changes. JDBC2 HiveConnection has odd defaults - Key: HIVE-4232 URL: https://issues.apache.org/jira/browse/HIVE-4232 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Attachments: HIVE-4232.patch HiveConnection defaults to using a plain SASL transport if auth is not set. To get a raw transport auth must be set to noSasl; furthermore noSasl is case sensitive. Code tries to infer Kerberos or plain authentication based on the presence of principal. There is no provision for specifying QOP level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4225) HiveServer2 does not support SASL QOP
[ https://issues.apache.org/jira/browse/HIVE-4225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13613595#comment-13613595 ] Chris Drome commented on HIVE-4225: --- I will rebase this patch if the changes in HIVE-4232 are acceptable. HiveServer2 does not support SASL QOP - Key: HIVE-4225 URL: https://issues.apache.org/jira/browse/HIVE-4225 Project: Hive Issue Type: Bug Components: HiveServer2, Shims Affects Versions: 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.11.0 Attachments: HIVE-4225.patch HiveServer2 implements Kerberos authentication through SASL framework, but does not support setting QOP. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4225) HiveServer2 does not support SASL QOP
Chris Drome created HIVE-4225: - Summary: HiveServer2 does not support SASL QOP Key: HIVE-4225 URL: https://issues.apache.org/jira/browse/HIVE-4225 Project: Hive Issue Type: Bug Components: HiveServer2, Shims Affects Versions: 0.11.0 Reporter: Chris Drome Assignee: Chris Drome HiveServer2 implements Kerberos authentication through SASL framework, but does not support setting QOP. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4225) HiveServer2 does not support SASL QOP
[ https://issues.apache.org/jira/browse/HIVE-4225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4225: -- Attachment: HIVE-4225.patch Added trunk patch which honors the QOP setting of hadoop.rpc.protection variable when a Configuration is used to initialize the client/server connection. HiveServer2 does not support SASL QOP - Key: HIVE-4225 URL: https://issues.apache.org/jira/browse/HIVE-4225 Project: Hive Issue Type: Bug Components: HiveServer2, Shims Affects Versions: 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Attachments: HIVE-4225.patch HiveServer2 implements Kerberos authentication through SASL framework, but does not support setting QOP. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4225) HiveServer2 does not support SASL QOP
[ https://issues.apache.org/jira/browse/HIVE-4225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4225: -- Fix Version/s: 0.11.0 Status: Patch Available (was: Open) HiveServer2 does not support SASL QOP - Key: HIVE-4225 URL: https://issues.apache.org/jira/browse/HIVE-4225 Project: Hive Issue Type: Bug Components: HiveServer2, Shims Affects Versions: 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.11.0 Attachments: HIVE-4225.patch HiveServer2 implements Kerberos authentication through SASL framework, but does not support setting QOP. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4166) closeAllForUGI causes failure in hiveserver2 when fetching large amount of data
[ https://issues.apache.org/jira/browse/HIVE-4166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4166: -- Attachment: (was: HIVE-4166-0.10.patch) closeAllForUGI causes failure in hiveserver2 when fetching large amount of data --- Key: HIVE-4166 URL: https://issues.apache.org/jira/browse/HIVE-4166 Project: Hive Issue Type: Bug Components: Security, Shims Affects Versions: 0.10.0, 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Attachments: HIVE-4166-trunk.patch HiveServer2 configured to use Kerberos authentication with doAs enabled throws an exception when fetching a large amount of data from a query. The exception is caused because FileSystem.closeAllForUGI is always called at the end of TUGIAssumingProcessor.process. This affects requests on the ResultSet for data from a SELECT query when the amount of data exceeds a certain size. At that point any subsequent calls to fetch more data throw an exception because the underlying DFSClient has been closed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4166) closeAllForUGI causes failure in hiveserver2 when fetching large amount of data
[ https://issues.apache.org/jira/browse/HIVE-4166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4166: -- Attachment: HIVE-4166-0.10.patch Fixed patch. closeAllForUGI causes failure in hiveserver2 when fetching large amount of data --- Key: HIVE-4166 URL: https://issues.apache.org/jira/browse/HIVE-4166 Project: Hive Issue Type: Bug Components: Security, Shims Affects Versions: 0.10.0, 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Attachments: HIVE-4166-0.10.patch, HIVE-4166-trunk.patch HiveServer2 configured to use Kerberos authentication with doAs enabled throws an exception when fetching a large amount of data from a query. The exception is caused because FileSystem.closeAllForUGI is always called at the end of TUGIAssumingProcessor.process. This affects requests on the ResultSet for data from a SELECT query when the amount of data exceeds a certain size. At that point any subsequent calls to fetch more data throw an exception because the underlying DFSClient has been closed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4166) closeAllForUGI causes failure in hiveserver2 when fetching large amount of data
Chris Drome created HIVE-4166: - Summary: closeAllForUGI causes failure in hiveserver2 when fetching large amount of data Key: HIVE-4166 URL: https://issues.apache.org/jira/browse/HIVE-4166 Project: Hive Issue Type: Bug Components: Security, Shims Affects Versions: 0.10.0, 0.11.0 Reporter: Chris Drome Assignee: Chris Drome HiveServer2 configured to use Kerberos authentication with doAs enabled throws an exception when fetching a large amount of data from a query. The exception is caused because FileSystem.closeAllForUGI is always called at the end of TUGIAssumingProcessor.process. This affects requests on the ResultSet for data from a SELECT query when the amount of data exceeds a certain size. At that point any subsequent calls to fetch more data throw an exception because the underlying DFSClient has been closed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4166) closeAllForUGI causes failure in hiveserver2 when fetching large amount of data
[ https://issues.apache.org/jira/browse/HIVE-4166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-4166: -- Attachment: HIVE-4166-0.10.patch HIVE-4166-trunk.patch Initial patch that fixes this failure. Will continue to test. closeAllForUGI causes failure in hiveserver2 when fetching large amount of data --- Key: HIVE-4166 URL: https://issues.apache.org/jira/browse/HIVE-4166 Project: Hive Issue Type: Bug Components: Security, Shims Affects Versions: 0.10.0, 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Attachments: HIVE-4166-0.10.patch, HIVE-4166-trunk.patch HiveServer2 configured to use Kerberos authentication with doAs enabled throws an exception when fetching a large amount of data from a query. The exception is caused because FileSystem.closeAllForUGI is always called at the end of TUGIAssumingProcessor.process. This affects requests on the ResultSet for data from a SELECT query when the amount of data exceeds a certain size. At that point any subsequent calls to fetch more data throw an exception because the underlying DFSClient has been closed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4166) closeAllForUGI causes failure in hiveserver2 when fetching large amount of data
[ https://issues.apache.org/jira/browse/HIVE-4166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13601912#comment-13601912 ] Chris Drome commented on HIVE-4166: --- [~prasadm] thanks for the comment. We are working with a patched version of 0.10, but I will keep this in mind. The original code assumed it was okay to close all FileSystem resources at the end of each call to process, with HiveServer2 this assumption no longer holds. So it should be fixed to ensure that doAs works properly or doAs should be deprecated in favor of impersonation. closeAllForUGI causes failure in hiveserver2 when fetching large amount of data --- Key: HIVE-4166 URL: https://issues.apache.org/jira/browse/HIVE-4166 Project: Hive Issue Type: Bug Components: Security, Shims Affects Versions: 0.10.0, 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Attachments: HIVE-4166-0.10.patch, HIVE-4166-trunk.patch HiveServer2 configured to use Kerberos authentication with doAs enabled throws an exception when fetching a large amount of data from a query. The exception is caused because FileSystem.closeAllForUGI is always called at the end of TUGIAssumingProcessor.process. This affects requests on the ResultSet for data from a SELECT query when the amount of data exceeds a certain size. At that point any subsequent calls to fetch more data throw an exception because the underlying DFSClient has been closed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
