[jira] [Created] (HIVE-15939) Make cast expressions comply more to sql2011
Zoltan Haindrich created HIVE-15939:
---
Summary: Make cast expressions comply more to sql2011
Key: HIVE-15939
URL: https://issues.apache.org/jira/browse/HIVE-15939
Project: Hive
Issue Type: Sub-task
Reporter: Zoltan Haindrich
in HIVE-6590 Jason have uncovered the fact that UDFToBoolean treats all
non-empty strings as true.
It would be great to have the cast expressions closer to the standard...at
least when there is an expected behaviour from the user;
like {{cast('false' as boolean)}} should be true.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[GitHub] hive pull request #150: mysql schema script should add a comma.
GitHub user hellopower opened a pull request: https://github.com/apache/hive/pull/150 mysql schema script should add a comma. when I run command 'schematool -dbType mysql -initSchema -verbose'. error: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ') ENGINE=InnoDB DEFAULT CHARSET=latin1' at line 1 You can merge this pull request into a Git repository by running: $ git pull https://github.com/hellopower/hive master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hive/pull/150.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #150 commit 700701e1fb0b0314e7e856954a35fa0b94fceb5a Author: wubiao Date: 2017-02-16T03:39:55Z mysql schema script should add a comma. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (HIVE-15938) position alias in order by fails for union queries
Sergey Shelukhin created HIVE-15938: --- Summary: position alias in order by fails for union queries Key: HIVE-15938 URL: https://issues.apache.org/jira/browse/HIVE-15938 Project: Hive Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HIVE-15937) RetryingMetaStoreClient.invoke() should retry Trift exception even if not wrapped in MetaException
Eugene Koifman created HIVE-15937: - Summary: RetryingMetaStoreClient.invoke() should retry Trift exception even if not wrapped in MetaException Key: HIVE-15937 URL: https://issues.apache.org/jira/browse/HIVE-15937 Project: Hive Issue Type: Bug Components: Metastore, Transactions Reporter: Eugene Koifman Assignee: Eugene Koifman -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HIVE-15936) ConcurrentModificationException in ATSHook
Daniel Dai created HIVE-15936:
-
Summary: ConcurrentModificationException in ATSHook
Key: HIVE-15936
URL: https://issues.apache.org/jira/browse/HIVE-15936
Project: Hive
Issue Type: Bug
Reporter: Daniel Dai
Assignee: Daniel Dai
Attachments: HIVE-15936.1.patch
See ATSHook error:
{noformat}
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextNode(HashMap.java:1437)
~[?:1.8.0_112]
at java.util.HashMap$EntryIterator.next(HashMap.java:1471)
~[?:1.8.0_112]
at java.util.HashMap$EntryIterator.next(HashMap.java:1469)
~[?:1.8.0_112]
at java.util.AbstractCollection.toArray(AbstractCollection.java:196)
~[?:1.8.0_112]
at com.google.common.collect.ImmutableMap.copyOf(ImmutableMap.java:290)
~[guava-14.0.1.jar:?]
at
org.apache.hadoop.hive.ql.log.PerfLogger.getEndTimes(PerfLogger.java:219)
~[hive-common-2.1.0.2.6.0.0-457.jar:2.1.0.2.6.0.0-457]
at
org.apache.hadoop.hive.ql.hooks.ATSHook.createPostHookEvent(ATSHook.java:347)
~[hive-exec-2.1.0.2.6.0.0-457.jar:2.1.0.2.6.0.0-457]
at org.apache.hadoop.hive.ql.hooks.ATSHook$2.run(ATSHook.java:206)
[hive-exec-2.1.0.2.6.0.0-457.jar:2.1.0.2.6.0.0-457]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[?:1.8.0_112]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[?:1.8.0_112]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[?:1.8.0_112]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[?:1.8.0_112]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
{noformat}
According to [~jdere], ATSHook is currently accessing the PerfLogger on a
separate thread, which means the main query thread can potentially write to the
PerfLogger at the same time.
The ATSHook should access the PerfLogger on the main query thread, before it
sends the execution to the ATS Logger thread.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (HIVE-15935) ACL is not set in ATS data
Daniel Dai created HIVE-15935: - Summary: ACL is not set in ATS data Key: HIVE-15935 URL: https://issues.apache.org/jira/browse/HIVE-15935 Project: Hive Issue Type: Bug Reporter: Daniel Dai Assignee: Daniel Dai When publishing ATS info, Hive does not set ACL, that make Hive ATS entries visible to all users. On the other hand, Tez ATS entires is using Tez DAG ACL which limit both view/modify ACL to end user only. We shall make them consistent. In the Jira, I am going to limit ACL to end user for both Tez ATS and Hive ATS, also provide config "hive.view.acls" and "hive.modify.acls" if user need to overridden. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HIVE-15934) Downgrade Maven surefire plugin from 2.19.1 to 2.18.1
Wei Zheng created HIVE-15934: Summary: Downgrade Maven surefire plugin from 2.19.1 to 2.18.1 Key: HIVE-15934 URL: https://issues.apache.org/jira/browse/HIVE-15934 Project: Hive Issue Type: Bug Affects Versions: 2.2.0 Reporter: Wei Zheng Assignee: Wei Zheng Surefire 2.19.1 has some issue (https://issues.apache.org/jira/browse/SUREFIRE-1255) which caused debugging session to abort after a short period of time. Many IntelliJ users have seen this, although it looks fine for Eclipse users. Version 2.18.1 works fine. We'd better make the change to not impact the development for IntelliJ guys. We can upgrade again once the root cause is figured out. cc [~kgyrtkirk] [~ashutoshc] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HIVE-15933) Improve plans for correlated subquery with join and predicate
Vineet Garg created HIVE-15933:
--
Summary: Improve plans for correlated subquery with join and
predicate
Key: HIVE-15933
URL: https://issues.apache.org/jira/browse/HIVE-15933
Project: Hive
Issue Type: Sub-task
Components: Query Planning
Reporter: Vineet Garg
Assignee: Vineet Garg
This is a continuation of HIVE-15905
for queries such as:
{code:SQL}
explain select
cd_gender,
cd_marital_status,
cd_education_status,
count(*) cnt1,
cd_purchase_estimate,
count(*) cnt2,
cd_credit_rating,
count(*) cnt3,
cd_dep_count,
count(*) cnt4,
cd_dep_employed_count,
count(*) cnt5,
cd_dep_college_count,
count(*) cnt6
from
customer c,customer_address ca,customer_demographics
where
c.c_current_addr_sk = ca.ca_address_sk and
ca_county in ('Walker County','Richland County','Gaines County','Douglas
County','Dona Ana County') and
cd_demo_sk = c.c_current_cdemo_sk and
exists (select *
from store_sales,date_dim
where c.c_customer_sk = ss_customer_sk and
ss_sold_date_sk = d_date_sk and
d_year = 2002 and
d_moy between 4 and 4+3)
group by cd_gender,
cd_marital_status,
cd_education_status,
cd_purchase_estimate,
cd_credit_rating,
cd_dep_count,
cd_dep_employed_count,
cd_dep_college_count
order by cd_gender,
cd_marital_status,
cd_education_status,
cd_purchase_estimate,
cd_credit_rating,
cd_dep_count,
cd_dep_employed_count,
cd_dep_college_count
limit 100;
{code}
HIVE generates un-necessary joins to produce value for correlated columns.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[GitHub] hive pull request #149: HIVE-15929. Fix HiveDecimalWritable compatibility wi...
GitHub user omalley opened a pull request: https://github.com/apache/hive/pull/149 HIVE-15929. Fix HiveDecimalWritable compatibility with Hive 2.1. This allows Hive 2.1 to work without getting an exception. You can merge this pull request into a Git repository by running: $ git pull https://github.com/omalley/hive hive-15929 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hive/pull/149.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #149 commit aeaf8dc2d8ef0adf0c6fdf216da89453a807459a Author: Owen O'Malley Date: 2017-02-15T01:25:46Z HIVE-15929. Fix HiveDecimalWritable compatibility with Hive 2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (HIVE-15932) add "explain ast"
Sergey Shelukhin created HIVE-15932: --- Summary: add "explain ast" Key: HIVE-15932 URL: https://issues.apache.org/jira/browse/HIVE-15932 Project: Hive Issue Type: Bug Reporter: Sergey Shelukhin AST was removed in explain extended in HIVE-13533; that makes sense from users perspective for the common case, but it would be useful for Hive developers and advanced users to see the AST to diagnose issues. "Explain ast" command can be added to dump the AST. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HIVE-15931) JDBC: Improve logging when using ZooKeeper
Vaibhav Gumashta created HIVE-15931: --- Summary: JDBC: Improve logging when using ZooKeeper Key: HIVE-15931 URL: https://issues.apache.org/jira/browse/HIVE-15931 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 2.2.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HIVE-15930) Syntax error in the MySQL metastore schema definition
Deepesh Khandelwal created HIVE-15930:
-
Summary: Syntax error in the MySQL metastore schema definition
Key: HIVE-15930
URL: https://issues.apache.org/jira/browse/HIVE-15930
Project: Hive
Issue Type: Bug
Components: Metastore
Reporter: Deepesh Khandelwal
Assignee: Deepesh Khandelwal
Priority: Critical
The MySQL metastore definition
metastore/scripts/upgrade/mysql/hive-schema-2.2.0.mysql.sql contains a syntax
error for the table NOTIFICATION_LOG:
{noformat}
CREATE TABLE IF NOT EXISTS `NOTIFICATION_LOG`
(
`NL_ID` BIGINT(20) NOT NULL,
`EVENT_ID` BIGINT(20) NOT NULL,
`EVENT_TIME` INT(11) NOT NULL,
`EVENT_TYPE` varchar(32) NOT NULL,
`DB_NAME` varchar(128),
`TBL_NAME` varchar(128),
`MESSAGE` longtext,
`MESSAGE_FORMAT` varchar(16)
PRIMARY KEY (`NL_ID`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
{noformat}
Notice a comma is missing after {{`MESSAGE_FORMAT` varchar(16)}}.
This got introduced through HIVE-10562.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (HIVE-15929) Fix HiveDecimalWritable
Owen O'Malley created HIVE-15929: Summary: Fix HiveDecimalWritable Key: HIVE-15929 URL: https://issues.apache.org/jira/browse/HIVE-15929 Project: Hive Issue Type: Bug Reporter: Owen O'Malley HIVE-15335 broke compatibility with Hive 2.1 by making HiveDecimalWritable.getInternalStorate() throw an exception when called on an unset value. It is easy to instead return an empty array, which will allow the old code to allocate a new array. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HIVE-15928) Parallelization of Select queries in Druid handler
Jesus Camacho Rodriguez created HIVE-15928: -- Summary: Parallelization of Select queries in Druid handler Key: HIVE-15928 URL: https://issues.apache.org/jira/browse/HIVE-15928 Project: Hive Issue Type: Sub-task Components: Druid integration Affects Versions: 2.2.0 Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Even if we split a Select query along its time dimension, parallelization is limited as all queries will hit the broker node. Instead, we can interrogate the broker to get the Druid nodes that contain the data, and query those nodes directly. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HIVE-15927) HIve does not insert UTF-8 characters properly when insert query is triggered.
JaySenSharma created HIVE-15927:
---
Summary: HIve does not insert UTF-8 characters properly when
insert query is triggered.
Key: HIVE-15927
URL: https://issues.apache.org/jira/browse/HIVE-15927
Project: Hive
Issue Type: Bug
Components: Hive
Affects Versions: 1.2.1, 1.2.0
Environment: All
Reporter: JaySenSharma
-
If we create a table as following from Ambari Hive View (or using any other
approach)
{code}
CREATE TABLE IF NOT EXISTS currency2 (id int, name string) ROW FORMAT SERDE
'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' WITH
SERDEPROPERTIES("serialization.encoding"='UTF-8','store.charset'='UTF-8',
'retrieve.charset'='UTF-8');
INSERT INTO currency2 VALUES (1, "£€₡");
SELECT * FROM currency2;
{code}
- Output from ambari side is attached as "Result.png"
Here we see that the queries were sent to the "hive" correctly as we see the
following entries appear in the "hiveserver2.log"
{code}
ssh root@hiveserevr2host
cd /var/log/hive
tail -f hiveserver2.log
{code}
Output of the log shows that hiveserver2 received the proper insert query
statements from Ambari Hive View.
{code}
2017-01-25 09:25:27,589 INFO [HiveServer2-Handler-Pool: Thread-60]:
parse.ParseDriver (ParseDriver.java:parse(185)) - Parsing command: INSERT INTO
currency2 VALUES (1, "£€₡")
2017-01-25 09:25:28,337 INFO [HiveServer2-Background-Pool: Thread-244]:
exec.Task (TezTask.java:build(316)) - Dag name: INSERT INTO currency2 VALUES
(1, "£€₡")(Stage-1)
2017-01-25 09:25:28,694 INFO [HiveServer2-Background-Pool: Thread-244]:
client.TezClient (TezClient.java:submitDAGSession(577)) - Submitted dag to
TezSession, sessionName=HIVE-bf038f08-853a-4850-80bc-7a6eaded3c2d,
applicationId=application_1485333705258_0007, dagName=INSERT INTO c
urrency1 VALUES (1, "£€₡")(Stage-1)
{code}
- Even the beeline fails to show the correct entry:
{code}
[root@erie3 ~]# su - hive
[hive@erie3 root]$ beeline
Beeline version 1.2.1000.2.5.3.0-37 by Apache Hive
beeline> !connect
jdbc:hive2://erie3.example.com:2181,erie1.example.com:2181,erie4.example.com:2181,erie2.example.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
0: jdbc:hive2://erie3.example.com:2181,erie1.> SELECT * FROM currency2;
+---+-+--+
| currency2.id | currency2.name |
+---+-+--+
| 1 | ��� |
+---+-+--+
{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (HIVE-15926) Hive 2.1.1 is not supporting any version of Spark
Hrishieksh created HIVE-15926: - Summary: Hive 2.1.1 is not supporting any version of Spark Key: HIVE-15926 URL: https://issues.apache.org/jira/browse/HIVE-15926 Project: Hive Issue Type: Bug Components: CLI Affects Versions: 2.1.1 Reporter: Hrishieksh I have Hive 2.1.1 and trying to integrate with spark so that i can use spark instead of mr as engine. i opened hive cli and run a sql query ,now when using Spark version 1.6.1 and 1.6.3 i am geting error Exception in thread "main" java.lang.NoSuchFieldError: SPARK_RPC_SERVER_ADDRESS at org.apache.hive.spark.client.rpc.RpcConfiguration.(RpcConfiguration.java:45) and whn using spark version 2.0.0 or 2.1.0 i am getting error org/apache/spark/JavaSparkListener class not found excpetion. Now tell me what to do with such problem. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
