[jira] [Created] (HIVE-15723) Hive should report a warning about missing table/column statistics to user.
Remus Rusanu created HIVE-15723: --- Summary: Hive should report a warning about missing table/column statistics to user. Key: HIVE-15723 URL: https://issues.apache.org/jira/browse/HIVE-15723 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Remus Rusanu Assignee: Remus Rusanu Priority: Minor Many Hive performance issues are due to missing statistics. Either all, table or column statistics are missing. Potentially a new partition has been added and customer forgot to gather stats for that partition. A simple warning about a table or column missing statistics can be very helpful and makes hive more user friendly. Hive already has this information, its a matter of printing it out. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-15724) getPrimaryKeys and getForeignKeys in metastore does not normalize db and table name
Daniel Dai created HIVE-15724: - Summary: getPrimaryKeys and getForeignKeys in metastore does not normalize db and table name Key: HIVE-15724 URL: https://issues.apache.org/jira/browse/HIVE-15724 Project: Hive Issue Type: Bug Components: Metastore Reporter: Daniel Dai Assignee: Daniel Dai In db, everything is lower case. When we retrieve constraints back, we need to normalize dbname/tablename. Otherwise, the following sample script fail: alter table Table9 add constraint pk1 primary key (a) disable novalidate; ALTER TABLE Table9 drop constraint pk1; Error message: InvalidObjectException(message:The constraint: pk1 does not exist for the associated table: default.Table9 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 55846: HIVE-15604: Improve contents of qfile test output
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55846/ --- (Updated Jan. 25, 2017, 1:12 p.m.) Review request for hive. Changes --- Thank you for the review and for spotting the issues. I change the throwables to exceptions in the catch clauses. Repository: hive-git Description --- HIVE-15604: Improve contents of qfile test output Diffs (updated) - itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/TestLocationQueries.java 06888467ec70eb1780e11969dafe2bff2c432d9d itests/util/src/main/java/org/apache/hadoop/hive/cli/control/AbstractCoreBlobstoreCliDriver.java 9c97c318ee89f598515d7b2e14f7f8184dc1119d itests/util/src/main/java/org/apache/hadoop/hive/cli/control/CoreAccumuloCliDriver.java 3e4b3732cf49c1ac14baa58b65731f407f2b1dcd itests/util/src/main/java/org/apache/hadoop/hive/cli/control/CoreCliDriver.java a735346c062684eb6069565e7354b875cde44bd7 itests/util/src/main/java/org/apache/hadoop/hive/cli/control/CoreCompareCliDriver.java 71a02bca0b416c264beb00e95d2c3ab8da30b5c1 itests/util/src/main/java/org/apache/hadoop/hive/cli/control/CoreHBaseCliDriver.java 956a42d2711296061aa09f79596885af475ce6e6 itests/util/src/main/java/org/apache/hadoop/hive/cli/control/CoreHBaseNegativeCliDriver.java 6225180a0735e41f5eee2c32e5b05c674e2edbdd itests/util/src/main/java/org/apache/hadoop/hive/cli/control/CoreNegativeCliDriver.java 65b2ce71d3797823c6c14002384857cb2a61749a itests/util/src/main/java/org/apache/hadoop/hive/cli/control/CorePerfCliDriver.java 8620cde37894aa5f15a4f7850c6f62c9ac8bf6d1 itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestProcessExecResult.java PRE-CREATION itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java 1eaf7af5381a164f442a1094fb2543a08dc6c253 itests/util/src/main/java/org/apache/hadoop/hive/ql/parse/CoreParseNegative.java 8dba0bb9d5288240414a59f9468d3adae4dd8ef1 Diff: https://reviews.apache.org/r/55846/diff/ Testing --- Ran qtests manually and checked the error messages. Checked that for diff failures we now have the output captured from the diff process in the error message of the failed test. Checkout how multiple tests, all with failures, look on the console to make sure that the results are still readeable/easy to follow. Thanks, Barna Zsombor Klara
Review Request 55932: HIVE-15387: NPE in HiveServer2 webUI Historical SQL Operations section
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55932/ --- Review request for hive. Repository: hive-git Description --- HIVE-15387: NPE in HiveServer2 webUI Historical SQL Operations section There is a race condition in the handling of the SQLOperationDisplay classes. Once an operation is complete/cancelled/timed out we move the SQLOperationDisplay from the live stack to the historical one. But the runtime is only updated once the state is set, after the SQLOperationDisplay is already among the historical queries. With this change the runtime will be "n/a" instead of an NPE being thrown. Diffs - service/src/resources/hive-webapps/hiveserver2/hiveserver2.jsp 6e899819cf33e149cbb2dbffb201348408330bbc Diff: https://reviews.apache.org/r/55932/diff/ Testing --- Ran multiple queries and checked that they are still displayed correctly. Thanks, Barna Zsombor Klara
[jira] [Created] (HIVE-15725) Make it possible to run checkstyle for a specific module
Peter Vary created HIVE-15725: - Summary: Make it possible to run checkstyle for a specific module Key: HIVE-15725 URL: https://issues.apache.org/jira/browse/HIVE-15725 Project: Hive Issue Type: Sub-task Affects Versions: 2.2.0 Reporter: Peter Vary Assignee: Peter Vary Priority: Minor Remove relative paths from the checkstyle.xml, so it could be run for every module, not just from the root. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-15726) Reenable indentation checks to checkstyle
Peter Vary created HIVE-15726: - Summary: Reenable indentation checks to checkstyle Key: HIVE-15726 URL: https://issues.apache.org/jira/browse/HIVE-15726 Project: Hive Issue Type: Improvement Reporter: Peter Vary Assignee: Peter Vary Priority: Minor Fix For: 2.2.0 The Indentation check is commented out because at that time there were no possibility to check the throws indentation. There is a possibility now, so we can reenable it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 55932: HIVE-15387: NPE in HiveServer2 webUI Historical SQL Operations section
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55932/#review162964 --- Ship it! Ship It! - Peter Vary On Jan. 25, 2017, 3:03 p.m., Barna Zsombor Klara wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/55932/ > --- > > (Updated Jan. 25, 2017, 3:03 p.m.) > > > Review request for hive. > > > Repository: hive-git > > > Description > --- > > HIVE-15387: NPE in HiveServer2 webUI Historical SQL Operations section > There is a race condition in the handling of the SQLOperationDisplay classes. > Once an operation is complete/cancelled/timed out we move the > SQLOperationDisplay from the live stack to the historical one. But the > runtime is only updated once the state is set, after the SQLOperationDisplay > is already among the historical queries. > With this change the runtime will be "n/a" instead of an NPE being thrown. > > > Diffs > - > > service/src/resources/hive-webapps/hiveserver2/hiveserver2.jsp > 6e899819cf33e149cbb2dbffb201348408330bbc > > Diff: https://reviews.apache.org/r/55932/diff/ > > > Testing > --- > > Ran multiple queries and checked that they are still displayed correctly. > > > Thanks, > > Barna Zsombor Klara > >
[jira] [Created] (HIVE-15727) Add pre insert work to give storage handler the possibility to perform pre insert checking
slim bouguerra created HIVE-15727: - Summary: Add pre insert work to give storage handler the possibility to perform pre insert checking Key: HIVE-15727 URL: https://issues.apache.org/jira/browse/HIVE-15727 Project: Hive Issue Type: Sub-task Components: Druid integration Reporter: slim bouguerra Assignee: slim bouguerra Fix For: 2.2.0 Add pre insert work stage to give storage handler the possibility to perform pre insert checking. For instance for the druid storage handler this will block the statement INSERT INTO statement. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] hive pull request #139: Adding pre insert work and block insert intot statem...
GitHub user b-slim opened a pull request: https://github.com/apache/hive/pull/139 Adding pre insert work and block insert intot statement You can merge this pull request into a Git repository by running: $ git pull https://github.com/b-slim/hive HIVE-15727-block-insert-into Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hive/pull/139.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #139 commit 68be75b3089a1439dedbe5a7a838e7b1cd19190d Author: Slim Bouguerra Date: 2017-01-21T00:01:43Z Adding pre insert work and block insert intot statement --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] hive pull request #135: [HIVE-15586] adding check loaded segments on coordin...
Github user b-slim closed the pull request at: https://github.com/apache/hive/pull/135 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] hive pull request #119: HIVE-15273 adding confing to http client
Github user b-slim closed the pull request at: https://github.com/apache/hive/pull/119 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] hive pull request #120: HIVE-15277 Druid stograge handler
Github user b-slim closed the pull request at: https://github.com/apache/hive/pull/120 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] hive pull request #124: [HIVE-15439] adding support for insert overwite
Github user b-slim closed the pull request at: https://github.com/apache/hive/pull/124 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
Review Request 55952: HIVE-15721 Allow IN/NOT IN correlated subquery with aggregates
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55952/ --- Review request for hive and Ashutosh Chauhan. Bugs: HIVE-15721 https://issues.apache.org/jira/browse/HIVE-15721 Repository: hive-git Description --- This patch lifts restriction on correlated IN/NOT IN subqueries with aggregate. See jira for more detailed description Diffs - ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSubQueryRemoveRule.java 564ef7a ql/src/java/org/apache/hadoop/hive/ql/parse/QBSubQuery.java 7ca722a ql/src/java/org/apache/hadoop/hive/ql/parse/SubQueryUtils.java bd771f9 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSQCountCheck.java 53e6231 ql/src/test/queries/clientnegative/subquery_in_implicit_gby.q 338747e ql/src/test/queries/clientnegative/subquery_notin_implicit_gby.q PRE-CREATION ql/src/test/queries/clientpositive/subquery_in.q 7293c77 ql/src/test/queries/clientpositive/subquery_notin.q 0883c89 ql/src/test/results/clientnegative/subquery_exists_implicit_gby.q.out bab6138 ql/src/test/results/clientnegative/subquery_in_implicit_gby.q.out a882fbc ql/src/test/results/clientnegative/subquery_notexists_implicit_gby.q.out bab6138 ql/src/test/results/clientnegative/subquery_notin_implicit_gby.q.out PRE-CREATION ql/src/test/results/clientpositive/llap/subquery_in.q.out eae6f1d ql/src/test/results/clientpositive/llap/subquery_notin.q.out e8ba117 ql/src/test/results/clientpositive/perf/query23.q.out c8f3b85 ql/src/test/results/clientpositive/spark/subquery_in.q.out 37a6617 Diff: https://reviews.apache.org/r/55952/diff/ Testing --- * Added positive and negative tests. * Pre-commit testing Thanks, Vineet Garg
Re: Review Request 55952: HIVE-15721 Allow IN/NOT IN correlated subquery with aggregates
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55952/ --- (Updated Jan. 25, 2017, 9:12 p.m.) Review request for hive and Ashutosh Chauhan. Changes --- Pre-commit test failure fix Bugs: HIVE-15721 https://issues.apache.org/jira/browse/HIVE-15721 Repository: hive-git Description --- This patch lifts restriction on correlated IN/NOT IN subqueries with aggregate. See jira for more detailed description Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSubQueryRemoveRule.java 564ef7a ql/src/java/org/apache/hadoop/hive/ql/parse/QBSubQuery.java 7ca722a ql/src/java/org/apache/hadoop/hive/ql/parse/SubQueryUtils.java bd771f9 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSQCountCheck.java 53e6231 ql/src/test/org/apache/hadoop/hive/ql/parse/TestQBSubQuery.java f9db2c8 ql/src/test/queries/clientnegative/subquery_in_implicit_gby.q 338747e ql/src/test/queries/clientnegative/subquery_notin_implicit_gby.q PRE-CREATION ql/src/test/queries/clientpositive/subquery_in.q 7293c77 ql/src/test/queries/clientpositive/subquery_notin.q 0883c89 ql/src/test/results/clientnegative/subquery_exists_implicit_gby.q.out bab6138 ql/src/test/results/clientnegative/subquery_in_implicit_gby.q.out a882fbc ql/src/test/results/clientnegative/subquery_notexists_implicit_gby.q.out bab6138 ql/src/test/results/clientnegative/subquery_notin_implicit_gby.q.out PRE-CREATION ql/src/test/results/clientpositive/llap/subquery_in.q.out eae6f1d ql/src/test/results/clientpositive/llap/subquery_notin.q.out e8ba117 ql/src/test/results/clientpositive/spark/subquery_in.q.out 37a6617 Diff: https://reviews.apache.org/r/55952/diff/ Testing --- * Added positive and negative tests. * Pre-commit testing Thanks, Vineet Garg
[jira] [Created] (HIVE-15728) Empty table returns result when querying partitioned field with a function
Jared Leable created HIVE-15728: --- Summary: Empty table returns result when querying partitioned field with a function Key: HIVE-15728 URL: https://issues.apache.org/jira/browse/HIVE-15728 Project: Hive Issue Type: Bug Components: Hive Affects Versions: 1.1.0 Reporter: Jared Leable If a partitioned table contained data and is then truncated a query will still return a result when using a function on a partitioned field. create table test1 ( field string ) partitioned by(dt string); set hive.exec.dynamic.partition.mode=nonstrict; insert into test1 partition(dt) select 'a','2017-01-01'; -- to view inserted records select * from test1; -- to delete all records from the table truncate table test1; -- to view 0 records in the table select * from test1; -- still returns a result for the partitioned field select max(dt) from test1; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-15729) Temporary table with LOCATION clause not removed after session ends
Marcus T created HIVE-15729:
---
Summary: Temporary table with LOCATION clause not removed after
session ends
Key: HIVE-15729
URL: https://issues.apache.org/jira/browse/HIVE-15729
Project: Hive
Issue Type: Bug
Reporter: Marcus T
Priority: Minor
When using CREATE TEMPORARY TABLE with a LOCATION clause, Hive doesn't remove
the created files when the session ends.
Steps to reproduce:
beeline
CREATE TEMPORARY TABLE this_gets_removed AS SELECT 'OK';
CREATE TEMPORARY TABLE this_stays LOCATION '/tmp/this_stays' AS SELECT
'BAD';
!quit
hdfs dfs -ls -R "/tmp/hive/${USER}" "/tmp/this_stays"
Expected result: files and directories for `this_gets_removed` and `this_stays`
are removed.
Actual result: files and directories for `this_gets_removed` are removed, files
and directories for `this_stays` are still present.
Ideally, it would be great if the LOCATION clause worked with the TEMPORARY
clause. If they can't be made to work together, perhaps the LOCATION clause
should be illegal when using TEMPORARY.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-15730) JDBC should use SQLFeatureNotSupportedException where appropriate instead of SQLException
Thejas M Nair created HIVE-15730:
Summary: JDBC should use SQLFeatureNotSupportedException where
appropriate instead of SQLException
Key: HIVE-15730
URL: https://issues.apache.org/jira/browse/HIVE-15730
Project: Hive
Issue Type: Bug
Components: JDBC
Reporter: Thejas M Nair
An example is HiveBaseResultSet.rowDeleted. It throws SQLException("Method not
supported") instead of SQLFeatureNotSupportedException.
For that optional method, the use of SQLFeatureNotSupportedException is more
appropriate.
See
http://docs.oracle.com/javase/7/docs/api/java/sql/ResultSet.html#rowDeleted()
http://docs.oracle.com/javase/7/docs/api/java/sql/SQLFeatureNotSupportedException.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-15731) sessions are not returned to the sessionPool in case of an interrupt
Siddharth Seth created HIVE-15731: - Summary: sessions are not returned to the sessionPool in case of an interrupt Key: HIVE-15731 URL: https://issues.apache.org/jira/browse/HIVE-15731 Project: Hive Issue Type: Bug Reporter: Siddharth Seth Assignee: Siddharth Seth Priority: Critical While returning a session to the pool, the interrupt status on the thread seems to be set, which causes the pool return to fail. The session slot is useless at this point. A HS2 instance configured for a single session will stop running queries. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-15732) add the ability to restrict configuration for the queries submitted to HS2 (Tez pool)
Sergey Shelukhin created HIVE-15732: --- Summary: add the ability to restrict configuration for the queries submitted to HS2 (Tez pool) Key: HIVE-15732 URL: https://issues.apache.org/jira/browse/HIVE-15732 Project: Hive Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: HIVE-15732.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Review Request 55977: HIVE-15672 LLAP text cache: improve first query perf II
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55977/ --- Review request for hive, Gopal V and Prasanth_J. Repository: hive-git Description --- see jira Diffs - common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 291ca7d llap-server/src/java/org/apache/hadoop/hive/llap/cache/SerDeLowLevelCacheImpl.java 85fae9a llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java d0e70d1 llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/SerDeEncodedDataReader.java 9ab26e6 llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/VertorDeserializeOrcWriter.java 98fc9df orc/src/java/org/apache/orc/impl/TreeReaderFactory.java 8b097ba orc/src/java/org/apache/orc/impl/WriterImpl.java b1f3cfb ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 6281edd ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java 0dba1a0 ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/Reader.java 1c5f0e6 storage-api/src/java/org/apache/hadoop/hive/common/io/encoded/EncodedColumnBatch.java b894c11e storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.java 552982c storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/ColumnVector.java 6f090a1 storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DecimalColumnVector.java e4f8d82 storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.java bd421f4 storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/IntervalDayTimeColumnVector.java c4a6c0f storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.java 80d4731 storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/MultiValuedColumnVector.java 1aeff83 storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/StructColumnVector.java cf07bca storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/TimestampColumnVector.java 28997a0 storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/UnionColumnVector.java 0c61243 Diff: https://reviews.apache.org/r/55977/diff/ Testing --- Thanks, Sergey Shelukhin
