[jira] [Created] (HIVE-10656) Beeline set var=value not carrying over to queries
Reuben Kuhnert created HIVE-10656:
-
Summary: Beeline set var=value not carrying over to queries
Key: HIVE-10656
URL: https://issues.apache.org/jira/browse/HIVE-10656
Project: Hive
Issue Type: Bug
Reporter: Reuben Kuhnert
Priority: Minor
After performing a {{set name=value}} I would expect that the variable name
would carry over to all locations within the session. It appears to work when
querying the value via {{set;}}, but not when trying to do actual sql
statements.
Example:
{code}
0: jdbc:hive2://localhost:1> set foo;
+--+--+
| set|
+--+--+
| foo=bar |
+--+--+
1 row selected (0.932 seconds)
0: jdbc:hive2://localhost:1> select * from ${foo};
Error: Error while compiling statement: FAILED: SemanticException [Error
10001]: Line 1:14 Table not found 'bar' (state=42S02,code=10001)
0: jdbc:hive2://localhost:1> show tables;
++--+
| tab_name |
++--+
| my |
| purchases |
++--+
2 rows selected (0.437 seconds)
0: jdbc:hive2://localhost:1> set foo=my;
No rows affected (0.017 seconds)
0: jdbc:hive2://localhost:1> set foo;
+-+--+
| set |
+-+--+
| foo=my |
+-+--+
1 row selected (0.02 seconds)
0: jdbc:hive2://localhost:1> select * from ${foo};
select * from ${foo};
Error: Error while compiling statement: FAILED: SemanticException [Error
10001]: Line 1:14 Table not found 'bar' (state=42S02,code=10001)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Review Request 33927: HIVE-10639 create SHA1 UDF
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33927/ --- (Updated May 8, 2015, 5:33 p.m.) Review request for hive and Jason Dere. Changes --- performance optimization. removed unnecessary copyBytes operation. Bugs: HIVE-10639 https://issues.apache.org/jira/browse/HIVE-10639 Repository: hive-git Description --- HIVE-10639 create SHA1 UDF Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 02a604ff0a4ed92dfd94b199e8b539f636b66f77 ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSha1.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/udf/TestUDFSha1.java PRE-CREATION ql/src/test/queries/clientpositive/udf_sha1.q PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out a422760400c62d026324dd667e4a632bfbe01b82 ql/src/test/results/clientpositive/udf_sha1.q.out PRE-CREATION Diff: https://reviews.apache.org/r/33927/diff/ Testing --- Thanks, Alexander Pivovarov
Re: [VOTE] Apache Hive 1.2.0 release candidate 1
1. Build src with hadoop-1, and hadoop-2. No issue. 2. Tried simple DDL queries with downloaded bin and built-bin from step 1). No issue. Didn't try DML query though. 3. Checked README, RELEASE_NOTES, etc. README contains @Version@ in the first line - not sure if that's an issue. RELEASE_NOTES is for 1.1.0 - doesn't look right. 4. Verified asc and md5. No issue. Best, Chao On Thu, May 7, 2015 at 6:07 PM, Sushanth Sowmyan wrote: > Hi Folks, > > We've cleared all the blockers listed for 1.2.0 release, either > committing them, or deferring out to an eventual 1.2.1 stabilization > release. (Any deferrals were a result of discussion between myself and > the committer responsible for the issue.) More details are available > here : > https://cwiki.apache.org/confluence/display/Hive/Hive+1.2+Release+Status > > Apache Hive 1.2.0 Release Candidate 1 is available here: > > https://people.apache.org/~khorgath/releases/1.2.0_RC1/artifacts/ > > My public key used for signing is as in : > > http://people.apache.org/~khorgath/releases/1.2.0_RC1/artifacts/khorgath.public_key > > Maven artifacts are available here: > > https://repository.apache.org/content/repositories/orgapachehive-1030 > > Source tag for RC0 is up on the apache git repo as tag > "release-1.2.0-rc1" (Browseable view over at > > https://git-wip-us.apache.org/repos/asf?p=hive.git;a=tag;h=42d6d588c4fb8b2d9f231af2f8bd09d592d0 > ) > > Voting will conclude in 72 hours. > > Hive PMC Members: Please test and vote. > > Thanks, > -Sushanth >
[jira] [Created] (HIVE-10657) Remove copyBytes operation from MD5 UDF
Alexander Pivovarov created HIVE-10657: -- Summary: Remove copyBytes operation from MD5 UDF Key: HIVE-10657 URL: https://issues.apache.org/jira/browse/HIVE-10657 Project: Hive Issue Type: Improvement Components: UDF Reporter: Alexander Pivovarov Assignee: Alexander Pivovarov Priority: Minor Current implementation uses Apache Commons DigestUtils.md5Hex method to get md5 hex. DigestUtils does not provide md5Hex method with signature (byte[], start, length). This is why copyBytes method was added to get bytes[] from BytesWritable. To avoid copying bytes from BytesWritable to new byte array we can use java MessageDigest API directly. MessageDigest has method update(byte[], start, length) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Review Request 33992: HIVE-10657 Remove copyBytes operation from MD5 UDF
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33992/ --- Review request for hive and Jason Dere. Bugs: HIVE-10657 https://issues.apache.org/jira/browse/HIVE-10657 Repository: hive-git Description --- HIVE-10657 Remove copyBytes operation from MD5 UDF Diffs - ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMd5.java 62c16c23375eec96def5553404945dd963459850 Diff: https://reviews.apache.org/r/33992/diff/ Testing --- Thanks, Alexander Pivovarov
[jira] [Created] (HIVE-10658) ACID operation expose encrypted data
Eugene Koifman created HIVE-10658: - Summary: ACID operation expose encrypted data Key: HIVE-10658 URL: https://issues.apache.org/jira/browse/HIVE-10658 Project: Hive Issue Type: Sub-task Reporter: Eugene Koifman Insert/Update/Delete operations all use temporary tables. the data in temp tables is stored under the hive.exec.scratchdir which is not usually encrypted. This is a similar issue to using scratchdir for staging query results -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Precommit queued up patch missing
Hi experts I submitted patch HIVE-10591 yesterday to queue up again for precommit tests. Was keeping an eye on the queue for once in a while yesterday and it was progressing forward from 20th to 16th item in the queue. But today morning it was missing from the queue. I had to reupload the patch again today but the earlier held position is lost (back to 18th). What can trigger the patch to just drop off from the queue? Thanks Prasanth
Re: Precommit queued up patch missing
I haven't seen anything similar like that. When the ptest is restarted due to changes on the configuration, this removes the patches from the queue, and it does not notify the jira about it. There was a restart in early this week, but there were not patches on the queue. Not sure what happened. On Fri, May 8, 2015 at 2:48 PM, Prasanth J wrote: > Hi experts > > I submitted patch HIVE-10591 yesterday to queue up again for precommit > tests. Was keeping an eye on the queue for once in a while yesterday and it > was progressing forward from 20th to 16th item in the queue. But today > morning it was missing from the queue. I had to reupload the patch again > today but the earlier held position is lost (back to 18th). What can > trigger the patch to just drop off from the queue? > > Thanks > Prasanth
[jira] [Created] (HIVE-10659) Beeline commands which contains semi-colon as a non-command terminator will fail
Hari Sankar Sivarama Subramaniyan created HIVE-10659: Summary: Beeline commands which contains semi-colon as a non-command terminator will fail Key: HIVE-10659 URL: https://issues.apache.org/jira/browse/HIVE-10659 Project: Hive Issue Type: Bug Reporter: Hari Sankar Sivarama Subramaniyan Assignee: Hari Sankar Sivarama Subramaniyan Consider beeline for connecting to mysql and creating commands involving stored procedures. MySQL stored procedures have semi-colon as the statement terminator. Since this coincides with beeline's only available command terminator , semi-colon, beeline will not able to execute the original command successfully. The above scenario can happen when Hive SchemaTool is used to upgrade a mysql metastore db which contains stored procedure in the script(as the one introduced initially by HIVE-7018). As of now, we cannot have any stored procedures as part of MySQL scripts because schemaTool uses beeline as the jdbc client to connect to MySQL. This is a serious limitation and needs to be fixed by providing an option to beeline to not use ";" as the command delimiter and process the entire line send to it as a single command. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Precommit queued up patch missing
Yea restart may cause it, but I dont think there was a restart or any activity yesterday, so not sure what happened. Thanks Szehon On Fri, May 8, 2015 at 1:43 PM, Sergio Pena wrote: > I haven't seen anything similar like that. > > When the ptest is restarted due to changes on the configuration, this > removes the patches from the queue, and it does not notify the jira about > it. > There was a restart in early this week, but there were not patches on the > queue. > > Not sure what happened. > > On Fri, May 8, 2015 at 2:48 PM, Prasanth J wrote: > > > Hi experts > > > > I submitted patch HIVE-10591 yesterday to queue up again for precommit > > tests. Was keeping an eye on the queue for once in a while yesterday and > it > > was progressing forward from 20th to 16th item in the queue. But today > > morning it was missing from the queue. I had to reupload the patch again > > today but the earlier held position is lost (back to 18th). What can > > trigger the patch to just drop off from the queue? > > > > Thanks > > Prasanth >
Re: [VOTE] Apache Hive 1.2.0 release candidate 1
Ugh, true, I missed updating RELEASE_NOTES. I will spin another RC with that fixed. In the meanwhile, since I intend to change no code bits between this RC and the next (unless anyone else discovers a code issue), I would much appreciate other eyes also on this RC to see if there are any other problems. Thanks, Chao! As to @VERSION@, does anyone know if that's supposed to be auto-updated by some tool, or is it intended to be edited in by the release manager? Also, Szehon, would it be possible to run a ptest run on branch-1.2 for a full unit test run as a one-time test? Thanks, -Sushanth On Fri, May 8, 2015 at 11:01 AM, Chao Sun wrote: > 1. Build src with hadoop-1, and hadoop-2. No issue. > 2. Tried simple DDL queries with downloaded bin and built-bin from step 1). > No issue. Didn't try DML query though. > 3. Checked README, RELEASE_NOTES, etc. README contains @Version@ in the > first line - not sure if that's an issue. > RELEASE_NOTES is for 1.1.0 - doesn't look right. > 4. Verified asc and md5. No issue. > > Best, > Chao > > On Thu, May 7, 2015 at 6:07 PM, Sushanth Sowmyan wrote: > >> Hi Folks, >> >> We've cleared all the blockers listed for 1.2.0 release, either >> committing them, or deferring out to an eventual 1.2.1 stabilization >> release. (Any deferrals were a result of discussion between myself and >> the committer responsible for the issue.) More details are available >> here : >> https://cwiki.apache.org/confluence/display/Hive/Hive+1.2+Release+Status >> >> Apache Hive 1.2.0 Release Candidate 1 is available here: >> >> https://people.apache.org/~khorgath/releases/1.2.0_RC1/artifacts/ >> >> My public key used for signing is as in : >> >> http://people.apache.org/~khorgath/releases/1.2.0_RC1/artifacts/khorgath.public_key >> >> Maven artifacts are available here: >> >> https://repository.apache.org/content/repositories/orgapachehive-1030 >> >> Source tag for RC0 is up on the apache git repo as tag >> "release-1.2.0-rc1" (Browseable view over at >> >> https://git-wip-us.apache.org/repos/asf?p=hive.git;a=tag;h=42d6d588c4fb8b2d9f231af2f8bd09d592d0 >> ) >> >> Voting will conclude in 72 hours. >> >> Hive PMC Members: Please test and vote. >> >> Thanks, >> -Sushanth >>
[jira] [Created] (HIVE-10660) Fix typo in Type.getType(TTypeId) exception message
Keegan Witt created HIVE-10660:
--
Summary: Fix typo in Type.getType(TTypeId) exception message
Key: HIVE-10660
URL: https://issues.apache.org/jira/browse/HIVE-10660
Project: Hive
Issue Type: Bug
Reporter: Keegan Witt
Assignee: Keegan Witt
Priority: Trivial
{{org.apache.hive.service.cli.Type.getType(org.apache.hive.service.cli.thrift.TTypeId)}}
throws an _IllegalArgumentException_ with 'Unrecognized' misspelled as
'Unrecognized'.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Review Request 34001: [Refactor] Removes ExprNodeNullDesc.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34001/ --- Review request for hive and Gopal V. Bugs: HIVE-10327 https://issues.apache.org/jira/browse/HIVE-10327 Repository: hive-git Description --- Removes ExprNodeNullDesc. Use ExprNodeConstantDesc in its place, which also contains type info additionally. Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/ExprNodeEvaluatorFactory.java ff0ddc8 ql/src/java/org/apache/hadoop/hive/ql/exec/ExprNodeGenericFuncEvaluator.java b695bef ql/src/java/org/apache/hadoop/hive/ql/exec/ExprNodeNullEvaluator.java 3aaf17c ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java df39218 ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConstantPropagateProcFactory.java f536ef6 ql/src/java/org/apache/hadoop/hive/ql/optimizer/GroupByOptimizer.java 1e47fcb ql/src/java/org/apache/hadoop/hive/ql/optimizer/PrunerExpressionOperatorFactory.java e633fdc ql/src/java/org/apache/hadoop/hive/ql/optimizer/SimpleFetchOptimizer.java 0328007 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/RexNodeConverter.java abd7afd ql/src/java/org/apache/hadoop/hive/ql/optimizer/lineage/ExprProcFactory.java 86d221d ql/src/java/org/apache/hadoop/hive/ql/optimizer/pcr/PcrExprProcFactory.java cbd4e6c ql/src/java/org/apache/hadoop/hive/ql/parse/TableAccessAnalyzer.java 01398f0 ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java a38511a ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeConstantDesc.java b15df0f ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeNullDesc.java 25b16da ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java 508d880 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFCoalesce.java 8890e69 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFGreatest.java e919345 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFInstr.java 0a13ac9 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLocate.java 094f280 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFPrintf.java cb6dd62 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTranslate.java 4ac542f ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFUtils.java 09d2d1f ql/src/test/results/clientpositive/annotate_stats_filter.q.out e8cd06d ql/src/test/results/clientpositive/input6.q.out 38c9fe1 ql/src/test/results/clientpositive/join_nullsafe.q.out 9bdfcbd ql/src/test/results/clientpositive/subquery_notin_having.q.java1.7.out 2d5b486 ql/src/test/results/clientpositive/vector_coalesce.q.out c63f2d1 serde/src/java/org/apache/hadoop/hive/serde2/io/HiveVarcharWritable.java a165b84 serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/AbstractPrimitiveObjectInspector.java baa4a94 serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableVoidObjectInspector.java f3f4838 Diff: https://reviews.apache.org/r/34001/diff/ Testing --- Refactoring patch. Existing regression tests will suffice. Thanks, Ashutosh Chauhan
Re: Review Request 33882: Speed-up count(distinct c1)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33882/ --- (Updated May 8, 2015, 9:30 p.m.) Review request for hive and John Pullokkaran. Changes --- Addresses John's comments. Bugs: HIVE-10568 https://issues.apache.org/jira/browse/HIVE-10568 Repository: hive-git Description --- Speed-up count(distinct c1) Diffs (updated) - common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 5d5a928 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveExpandDistinctAggregatesRule.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 48f488f ql/src/test/results/clientpositive/tez/limit_pushdown.q.out 7038b4d ql/src/test/results/clientpositive/tez/mrr.q.out ac5ac5c ql/src/test/results/clientpositive/tez/vector_count_distinct.q.out f1da471 ql/src/test/results/clientpositive/tez/vectorization_limit.q.out d815938 ql/src/test/results/clientpositive/tez/vectorized_distinct_gby.q.out 90c9934 Diff: https://reviews.apache.org/r/33882/diff/ Testing --- Existing tests. Thanks, Ashutosh Chauhan
Re: Review Request 33882: Speed-up count(distinct c1)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33882/#review83105 --- Ship it! Ship It! - John Pullokkaran On May 8, 2015, 9:30 p.m., Ashutosh Chauhan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/33882/ > --- > > (Updated May 8, 2015, 9:30 p.m.) > > > Review request for hive and John Pullokkaran. > > > Bugs: HIVE-10568 > https://issues.apache.org/jira/browse/HIVE-10568 > > > Repository: hive-git > > > Description > --- > > Speed-up count(distinct c1) > > > Diffs > - > > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 5d5a928 > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveExpandDistinctAggregatesRule.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 48f488f > ql/src/test/results/clientpositive/tez/limit_pushdown.q.out 7038b4d > ql/src/test/results/clientpositive/tez/mrr.q.out ac5ac5c > ql/src/test/results/clientpositive/tez/vector_count_distinct.q.out f1da471 > ql/src/test/results/clientpositive/tez/vectorization_limit.q.out d815938 > ql/src/test/results/clientpositive/tez/vectorized_distinct_gby.q.out > 90c9934 > > Diff: https://reviews.apache.org/r/33882/diff/ > > > Testing > --- > > Existing tests. > > > Thanks, > > Ashutosh Chauhan > >
[jira] [Created] (HIVE-10661) LLAP: investigate why GC with IO elevator disabled is so bad
Sergey Shelukhin created HIVE-10661:
---
Summary: LLAP: investigate why GC with IO elevator disabled is so
bad
Key: HIVE-10661
URL: https://issues.apache.org/jira/browse/HIVE-10661
Project: Hive
Issue Type: Sub-task
Reporter: Sergey Shelukhin
Assignee: Prasanth Jayachandran
Examples of running same query (Q1) on experimental setup, with Parallel GC, 12
times.
Time, DAG name, DAG time, GC time counter.
GC time counter on LLAP seems relatively reliable.
Note that non-IO jobs are also much slower during some time. It may not be
explained entirely by GC, I am investigating it now.
I may look at this later, after main GC tuning, but for now I decided to give
up on this since elevator will be on by default when using LLAP.
{noformat}
$ cat io-dag.csv
2015-05-08 12:10:57,695,dag_1429683757595_0843_1,71142,953216
2015-05-08 12:11:41,769,dag_1429683757595_0843_2,43144,844430
2015-05-08 12:12:22,335,dag_1429683757595_0843_3,39828,866538
2015-05-08 12:13:01,327,dag_1429683757595_0843_4,38213,822179
2015-05-08 12:13:39,610,dag_1429683757595_0843_5,37513,863968
2015-05-08 12:14:19,293,dag_1429683757595_0843_6,38320,913591
2015-05-08 12:14:58,500,dag_1429683757595_0843_7,38587,972450
2015-05-08 12:15:39,017,dag_1429683757595_0843_8,39845,1085598
2015-05-08 12:16:19,708,dag_1429683757595_0843_9,39979,1165559
2015-05-08 12:17:03,174,dag_1429683757595_0843_10,42713,1447033
2015-05-08 12:17:47,557,dag_1429683757595_0843_11,43670,1454114
2015-05-08 12:18:31,440,dag_1429683757595_0843_12,43178,1380477
$ cat noio-dag.csv
2015-05-08 11:44:05,846,dag_1429683757595_0841_1,60740,1643276
2015-05-08 11:44:55,761,dag_1429683757595_0841_2,48984,1590546
2015-05-08 11:45:48,978,dag_1429683757595_0841_3,52353,1765823
2015-05-08 11:46:44,810,dag_1429683757595_0841_4,54930,1831224
2015-05-08 11:47:47,368,dag_1429683757595_0841_5,61677,2068089
2015-05-08 11:49:05,235,dag_1429683757595_0841_6,76725,2416709
2015-05-08 11:51:56,998,dag_1429683757595_0841_7,170575,3250698
2015-05-08 11:58:16,728,dag_1429683757595_0841_8,377732,5541900
2015-05-08 12:03:17,344,dag_1429683757595_0841_9,298682,1844769
2015-05-08 12:05:23,267,dag_1429683757595_0841_10,124954,1331763
2015-05-08 12:06:35,650,dag_1429683757595_0841_11,71350,1703387
2015-05-08 12:07:42,599,dag_1429683757595_0841_12,66143,1724482
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-10662) LLAP: Wait queue pre-emption
Prasanth Jayachandran created HIVE-10662: Summary: LLAP: Wait queue pre-emption Key: HIVE-10662 URL: https://issues.apache.org/jira/browse/HIVE-10662 Project: Hive Issue Type: Sub-task Affects Versions: llap Reporter: Prasanth Jayachandran Assignee: Prasanth Jayachandran Fix For: llap If the wait queue is full, currently the task scheduler throws RejectedExecutionException. Instead it should kick out the lowest priority task and notify the AM as task killed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-10663) LLAP: Update tez dependency
Siddharth Seth created HIVE-10663: - Summary: LLAP: Update tez dependency Key: HIVE-10663 URL: https://issues.apache.org/jira/browse/HIVE-10663 Project: Hive Issue Type: Sub-task Reporter: Siddharth Seth Assignee: Siddharth Seth Fix For: llap -- This message was sent by Atlassian JIRA (v6.3.4#6332)
