Re: Review Request 19599: HiveServer2 secure thrift/http authentication needs to support SPNego

[Prasad Mujumdar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19599/#review38701
---


Overall looks fine to me. A couple minor of questions/comments below. Don't 
have to be address as part of this patch. Thanks!


service/src/java/org/apache/hive/service/cli/CLIService.java


Just curious, do the two principals need to be different ? Can't the same 
user run the service as well as authenticate with Knox ?




service/src/java/org/apache/hive/service/cli/CLIService.java


Should this throw an exception instead or warning ?


- Prasad Mujumdar


On March 26, 2014, 2:38 a.m., dilli dorai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19599/
> ---
> 
> (Updated March 26, 2014, 2:38 a.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan, Thejas Nair, and Vaibhav Gumashta.
> 
> 
> Bugs: HIVE-6697
> https://issues.apache.org/jira/browse/HIVE-6697
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> See JIra for description
> https://issues.apache.org/jira/browse/HIVE-6697
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java affcbb4 
>   conf/hive-default.xml.template 3c3df43 
>   service/src/java/org/apache/hive/service/auth/HiveAuthFactory.java 6e6a47d 
>   service/src/java/org/apache/hive/service/cli/CLIService.java e31a74e 
>   
> service/src/java/org/apache/hive/service/cli/thrift/ThriftHttpCLIService.java 
> cb01cfd 
>   service/src/java/org/apache/hive/service/cli/thrift/ThriftHttpServlet.java 
> 255a165 
>   shims/0.20/src/main/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java 
> 9aa555a 
>   
> shims/common-secure/src/main/java/org/apache/hadoop/hive/shims/HadoopShimsSecure.java
>  d4cddda 
>   shims/common/src/main/java/org/apache/hadoop/hive/shims/HadoopShims.java 
> ed951f1 
> 
> Diff: https://reviews.apache.org/r/19599/diff/
> 
> 
> Testing
> ---
> 
> ## Verification of enhancement with Beeline/JDBC 
> 
> ### Verified the following calls succeeded getting connection, and listig 
> tables, 
> when valid spnego.principal and spengo.keytab are specified in hive-site.xml,
> and the client has KINITed and has a valid kerberos ticket in cache
> 
> 
> !connect 
> jdbc:hive2://hdps.example.com:10001/default;principal=hive/hdps.example@example.com?hive.server2.transport.mode=http;hive.server2.thrift.http.path=cliservice
>   dummy dummy-pass org.apache.hive.jdbc.HiveDriver 
> 
> 
> !connect 
> jdbc:hive2://hdps.example.com:10001/default;principal=HTTP/hdps.example@example.com?hive.server2.transport.mode=http;hive.server2.thrift.http.path=cliservice
>   dummy dummy-pass org.apache.hive.jdbc.HiveDriver 
> 
> ### Verified the following call succeeded getting connection, and listig 
> tables, 
> even if valid spnego.principal or valid spengo.keytab is not  specified in 
> hive-site.xml,
> as long as valid hive server2 kerberos principal and keytab are specified in 
> hive-site.xml,
> and the client has KINITed and has a valid kerberos ticket in cache
> 
> !connect 
> jdbc:hive2://hdps.example.com:10001/default;principal=hive/hdps.example@example.com?hive.server2.transport.mode=http;hive.server2.thrift.http.path=cliservice
>   dummy dummy-pass org.apache.hive.jdbc.HiveDriver 
> 
> ### Verified the following call failed  getting connection, 
> when valid  spnego.principal or valid spengo.keytab is not specified in 
> hive-site.xml
> 
> !connect 
> jdbc:hive2://hdps.example.com:10001/default;principal=HTTP/hdps.example@example.com?hive.server2.transport.mode=http;hive.server2.thrift.http.path=cliservice
>   dummy dummy-pass org.apache.hive.jdbc.HiveDriver 
> 
> ## Verification of enhancement with Apache Knox
> 
> Apache Knox was able to authenticate to hive server 2 as SPNego client using 
> Apache HttpClient,
> and list tables, when correct spnego.principal and spengo.keytab are 
> specified in hive-site.xml
> 
> Apache Knox was not able to authenticate to hive server 2 as SPNego client 
> using Apache HttpClient,
> when valid spnego.principal or spengo.keytab is not specified in hive-site.xml
> 
> ## Verification of enhancement with curl
> 
> ### when valid spnego.principal and spengo.keytab are specified in 
> hive-site.xml
> and the client has KINITed and has a valid kerberos ticket in cache
> 
> curl -i --negotiate -u : http://hdps.example.com:10001/cliservice
> 
> SPNego authentication succeeded and got a HTTP status code 500,
> since we did not end Thrift body content
> 
> ### when valid spnego.principal and spengo.keytab are specified in 
> hive-site.xml
> and the client has not KINITed and does not have a  valid k

[jira] [Updated] (HIVE-6638) Hive needs to implement recovery for Application Master restart

[Mohammad Kamrul Islam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mohammad Kamrul Islam updated HIVE-6638:


Status: Patch Available  (was: Open)

> Hive needs to implement recovery for Application Master restart 
> 
>
> Key: HIVE-6638
> URL: https://issues.apache.org/jira/browse/HIVE-6638
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Affects Versions: 0.12.0, 0.11.0, 0.13.0
>Reporter: Ashutosh Chauhan
>Assignee: Mohammad Kamrul Islam
> Attachments: HIVE-6638.1.patch
>
>
> Currently, if AM restarts, whole job is restarted. Although, job and 
> subsequently query would still finish to completion, it would be nice if Hive 
> don't need to redo all the work done under previous AM.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6743) Allow specifying the log level for Tez tasks

[Lefty Leverenz (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948945#comment-13948945
 ] 

Lefty Leverenz commented on HIVE-6743:
--

For the record:  this adds *hive.tez.log.level* to HiveConf.java and 
hive-default.xml.template.

After HIVE-6037 gets committed, the description in hive-default.xml.template 
can be merged into the new HiveConf.java (via HIVE-6586).

> Allow specifying the log level for Tez tasks
> 
>
> Key: HIVE-6743
> URL: https://issues.apache.org/jira/browse/HIVE-6743
> Project: Hive
>  Issue Type: Improvement
>Reporter: Siddharth Seth
>Assignee: Siddharth Seth
> Attachments: HIVE-6743.1.patch, HIVE-6743.2.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6697) HiveServer2 secure thrift/http authentication needs to support SPNego

[Vaibhav Gumashta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vaibhav Gumashta updated HIVE-6697:
---

Attachment: HIVE-6697.4.patch

[~darumugam] The v3 failed to apply on trunk. I'm attaching your patch rebased 
on trunk.

> HiveServer2 secure thrift/http authentication needs to support SPNego 
> --
>
> Key: HIVE-6697
> URL: https://issues.apache.org/jira/browse/HIVE-6697
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Reporter: Dilli Arumugam
>Assignee: Dilli Arumugam
> Attachments: HIVE-6697.1.patch, HIVE-6697.2.patch, HIVE-6697.3.patch, 
> HIVE-6697.4.patch, hive-6697-req-impl-verify.md
>
>
> Looking to integrating Apache Knox to work with HiveServer2 secure 
> thrift/http.
> Found that thrift/http uses some form of Kerberos authentication that is not 
> SPNego. Considering it is going over http protocol, expected it to use SPNego 
> protocol.
> Apache Knox is already integrated with WebHDFS, WebHCat, Oozie and HBase 
> Stargate using SPNego for authentication.
> Requesting that HiveServer2 secure thrift/http authentication support SPNego.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6697) HiveServer2 secure thrift/http authentication needs to support SPNego

[Vaibhav Gumashta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vaibhav Gumashta updated HIVE-6697:
---

Status: Open  (was: Patch Available)

> HiveServer2 secure thrift/http authentication needs to support SPNego 
> --
>
> Key: HIVE-6697
> URL: https://issues.apache.org/jira/browse/HIVE-6697
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Reporter: Dilli Arumugam
>Assignee: Dilli Arumugam
> Attachments: HIVE-6697.1.patch, HIVE-6697.2.patch, HIVE-6697.3.patch, 
> HIVE-6697.4.patch, hive-6697-req-impl-verify.md
>
>
> Looking to integrating Apache Knox to work with HiveServer2 secure 
> thrift/http.
> Found that thrift/http uses some form of Kerberos authentication that is not 
> SPNego. Considering it is going over http protocol, expected it to use SPNego 
> protocol.
> Apache Knox is already integrated with WebHDFS, WebHCat, Oozie and HBase 
> Stargate using SPNego for authentication.
> Requesting that HiveServer2 secure thrift/http authentication support SPNego.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6697) HiveServer2 secure thrift/http authentication needs to support SPNego

[Vaibhav Gumashta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vaibhav Gumashta updated HIVE-6697:
---

Status: Patch Available  (was: Open)

> HiveServer2 secure thrift/http authentication needs to support SPNego 
> --
>
> Key: HIVE-6697
> URL: https://issues.apache.org/jira/browse/HIVE-6697
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Reporter: Dilli Arumugam
>Assignee: Dilli Arumugam
> Attachments: HIVE-6697.1.patch, HIVE-6697.2.patch, HIVE-6697.3.patch, 
> HIVE-6697.4.patch, hive-6697-req-impl-verify.md
>
>
> Looking to integrating Apache Knox to work with HiveServer2 secure 
> thrift/http.
> Found that thrift/http uses some form of Kerberos authentication that is not 
> SPNego. Considering it is going over http protocol, expected it to use SPNego 
> protocol.
> Apache Knox is already integrated with WebHDFS, WebHCat, Oozie and HBase 
> Stargate using SPNego for authentication.
> Requesting that HiveServer2 secure thrift/http authentication support SPNego.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6492) limit partition number involved in a table scan

[Lefty Leverenz (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948934#comment-13948934
 ] 

Lefty Leverenz commented on HIVE-6492:
--

This adds *hive.limit.query.max.table.partition* to HiveConf.java but it needs 
a description.  There's plenty of description in the comments, but a release 
note would be helpful.  Then I could put it in the wiki, and make sure the 
description goes into the new HiveConf.java (via HIVE-6586) after HIVE-6037 
gets committed.

> limit partition number involved in a table scan
> ---
>
> Key: HIVE-6492
> URL: https://issues.apache.org/jira/browse/HIVE-6492
> Project: Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Affects Versions: 0.12.0
>Reporter: Selina Zhang
>Assignee: Selina Zhang
> Fix For: 0.13.0
>
> Attachments: HIVE-6492.1.patch.txt, HIVE-6492.2.patch.txt, 
> HIVE-6492.3.patch.txt, HIVE-6492.4.patch.txt, HIVE-6492.4.patch_suggestion, 
> HIVE-6492.5.patch.txt, HIVE-6492.6.patch.txt, HIVE-6492.7.parch.txt
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> To protect the cluster, a new configure variable 
> "hive.limit.query.max.table.partition" is added to hive configuration to
> limit the table partitions involved in a table scan. 
> The default value will be set to -1 which means there is no limit by default. 
> This variable will not affect "metadata only" query.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6764) when set hive.security.authorization.enabled=true,hive start up with errors.

[haitangfan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

haitangfan updated HIVE-6764:
-

Description: 
1、set hive.security.authorization.enabled=true;
2、startup hive,find some errors in the log:
notice: /Stage[2]/Hdp-hcat::Hcat::Service_check/Exec[hcatSmoke.sh 
prepare]/returns: FAILED:AuthorizationException No privilege 'Create' found for 
inputs { database:default,
table:hcatsmokeida8c00f0b_date432614}
err: /Stage[2]/Hdp-hcat::Hcat::Service_check/Exec[hcatSmoke.sh 
prepare]/returns: change from notrun to 0
failed: su - ambari-qa -c 'sh /tmp/hcatSmoke.sh hcatsmokeida8c00f0b_date432614 
prepare' returned 64 instead of
one of [0] at 
/var/lib/ambari-agent/puppet/modules/hdp-hcat/manifests/hcat/service_check.pp:54
notice: 
/Stage[2]/Hdp-hive::Hive::Service_check/Exec[/tmp/hiveserver2Smoke.sh]/returns: 
Smoke test of
hiveserver2 passed
notice: 
/Stage[2]/Hdp-hive::Hive::Service_check/Exec[/tmp/hiveserver2Smoke.sh]/returns: 
executed successfully

3、the /tmp/hcatSmoke.sh contents
export tablename=$1case "$2" in
prepare)
  hcat -e "show tables"
  hcat -e "drop table IF EXISTS ${tablename}"
  hcat -e "create table ${tablename} ( id INT, name string ) stored as rcfile ;"
;;
cleanup)
  hcat -e "drop table IF EXISTS ${tablename}"
;;
esac

4、try to grant to user ambari-qa,faild.
hive> grant all on database default to user ambari-qa;FAILED: ParseException 
line 1:44 missing EOF at '-' near 'ambari'

How to fix it?

> when set hive.security.authorization.enabled=true,hive start up with errors.
> 
>
> Key: HIVE-6764
> URL: https://issues.apache.org/jira/browse/HIVE-6764
> Project: Hive
>  Issue Type: Bug
>  Components: Authorization
>Reporter: haitangfan
>
> 1、set hive.security.authorization.enabled=true;
> 2、startup hive,find some errors in the log:
> notice: /Stage[2]/Hdp-hcat::Hcat::Service_check/Exec[hcatSmoke.sh 
> prepare]/returns: FAILED:AuthorizationException No privilege 'Create' found 
> for inputs { database:default,
> table:hcatsmokeida8c00f0b_date432614}
> err: /Stage[2]/Hdp-hcat::Hcat::Service_check/Exec[hcatSmoke.sh 
> prepare]/returns: change from notrun to 0
> failed: su - ambari-qa -c 'sh /tmp/hcatSmoke.sh 
> hcatsmokeida8c00f0b_date432614 prepare' returned 64 instead of
> one of [0] at 
> /var/lib/ambari-agent/puppet/modules/hdp-hcat/manifests/hcat/service_check.pp:54
> notice: 
> /Stage[2]/Hdp-hive::Hive::Service_check/Exec[/tmp/hiveserver2Smoke.sh]/returns:
>  Smoke test of
> hiveserver2 passed
> notice: 
> /Stage[2]/Hdp-hive::Hive::Service_check/Exec[/tmp/hiveserver2Smoke.sh]/returns:
>  executed successfully
> 3、the /tmp/hcatSmoke.sh contents
> export tablename=$1case "$2" in
> prepare)
>   hcat -e "show tables"
>   hcat -e "drop table IF EXISTS ${tablename}"
>   hcat -e "create table ${tablename} ( id INT, name string ) stored as rcfile 
> ;"
> ;;
> cleanup)
>   hcat -e "drop table IF EXISTS ${tablename}"
> ;;
> esac
> 4、try to grant to user ambari-qa,faild.
> hive> grant all on database default to user ambari-qa;FAILED: ParseException 
> line 1:44 missing EOF at '-' near 'ambari'
> How to fix it?



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HIVE-6764) when set hive.security.authorization.enabled=true,hive start up with errors.

[haitangfan (JIRA)]

haitangfan created HIVE-6764:


 Summary: when set hive.security.authorization.enabled=true,hive 
start up with errors.
 Key: HIVE-6764
 URL: https://issues.apache.org/jira/browse/HIVE-6764
 Project: Hive
  Issue Type: Bug
  Components: Authorization
Reporter: haitangfan






--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6447) Bucket map joins in hive-tez

[Vikram Dixit K (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948917#comment-13948917
 ] 

Vikram Dixit K commented on HIVE-6447:
--

Failures unrelated. They pass locally.

> Bucket map joins in hive-tez
> 
>
> Key: HIVE-6447
> URL: https://issues.apache.org/jira/browse/HIVE-6447
> Project: Hive
>  Issue Type: Bug
>  Components: Tez
>Affects Versions: tez-branch
>Reporter: Vikram Dixit K
>Assignee: Vikram Dixit K
> Attachments: HIVE-6447.1.patch, HIVE-6447.10.patch, 
> HIVE-6447.11.patch, HIVE-6447.12.patch, HIVE-6447.2.patch, HIVE-6447.3.patch, 
> HIVE-6447.4.patch, HIVE-6447.5.patch, HIVE-6447.6.patch, HIVE-6447.7.patch, 
> HIVE-6447.8.patch, HIVE-6447.9.patch, HIVE-6447.WIP.patch
>
>
> Support bucket map joins in tez.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6642) Query fails to vectorize when a non string partition column is part of the query expression

[Alan Gates (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948885#comment-13948885
 ] 

Alan Gates commented on HIVE-6642:
--

Saw the following failures:
{code}
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_partition_coltype
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_part
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_1
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_11
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_12
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_2
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_3
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_4
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_7
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_8
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket3
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_1
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_2
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_3
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_4
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_6
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_7
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_8
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin1
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin10
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin11
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin12
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin13
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin2
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin3
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin5
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin8
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin9
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin_negative
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin_negative2
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_columnstats_partlvl
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_combine2_hadoop20
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_dynamic_partition_skip_default
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_filter_join_breaktask
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_filter_numeric
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_map_ppr
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_map_ppr_multi_distinct
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_ppr
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_ppr_multi_distinct
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_6
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input23
2014-03-27 02:49:33 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input42
2014-

[jira] [Commented] (HIVE-6734) DDL locking too course grained in new db txn manager

[Alan Gates (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948884#comment-13948884
 ] 

Alan Gates commented on HIVE-6734:
--

Ran tests locally, all looks good.

> DDL locking too course grained in new db txn manager
> 
>
> Key: HIVE-6734
> URL: https://issues.apache.org/jira/browse/HIVE-6734
> Project: Hive
>  Issue Type: Bug
>  Components: Locking
>Affects Versions: 0.13.0
>Reporter: Alan Gates
>Assignee: Alan Gates
> Fix For: 0.13.0
>
> Attachments: HIVE-6734.patch
>
>
> All DDL operations currently acquire an exclusive lock.  This is too course 
> grained, as some operations like alter table add partition shouldn't get an 
> exclusive lock on the entire table.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6643) Add a check for cross products in plans and output a warning

[Alan Gates (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948872#comment-13948872
 ] 

Alan Gates commented on HIVE-6643:
--

I still see the following failures when running the tests on the latest patch:
2014-03-27 01:20:45 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input23
2014-03-27 01:20:45 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join0
2014-03-27 01:20:45 WARN  PTest:205 - 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join23

> Add a check for cross products in plans and output a warning
> 
>
> Key: HIVE-6643
> URL: https://issues.apache.org/jira/browse/HIVE-6643
> Project: Hive
>  Issue Type: Bug
>Reporter: Harish Butani
>Assignee: Harish Butani
> Fix For: 0.13.0
>
> Attachments: HIVE-6643.1.patch, HIVE-6643.2.patch, HIVE-6643.3.patch, 
> HIVE-6643.4.patch, HIVE-6643.5.patch
>
>
> Now that we support old style join syntax, it is easy to write queries that 
> generate a plan with a cross product.
> For e.g. say you have A join B join C join D on A.x = B.x and A.y = D.y and 
> C.z = D.z
> So the JoinTree is:
> A — B
> |__  D — C
> Since we don't reorder join graphs, we will end up with a cross product 
> between (A join B) and C



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6710) Deadlocks seen in transaction handler using mysql

[Alan Gates (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948873#comment-13948873
 ] 

Alan Gates commented on HIVE-6710:
--

Ran tests locally, all looks good.

> Deadlocks seen in transaction handler using mysql
> -
>
> Key: HIVE-6710
> URL: https://issues.apache.org/jira/browse/HIVE-6710
> Project: Hive
>  Issue Type: Bug
>  Components: Locking
>Affects Versions: 0.13.0
>Reporter: Alan Gates
>Assignee: Alan Gates
> Fix For: 0.13.0
>
> Attachments: HIVE-6710.patch
>
>
> When multiple clients attempt to obtain locks a deadlock on the mysql 
> database occasionally occurs.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6694) Beeline should provide a way to execute shell command as Hive CLI does

[Xuefu Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-6694:
--

Affects Version/s: 0.13.0

> Beeline should provide a way to execute shell command as Hive CLI does
> --
>
> Key: HIVE-6694
> URL: https://issues.apache.org/jira/browse/HIVE-6694
> Project: Hive
>  Issue Type: Improvement
>  Components: CLI, Clients
>Affects Versions: 0.11.0, 0.12.0, 0.13.0
>Reporter: Xuefu Zhang
>Assignee: Xuefu Zhang
> Attachments: HIVE-6694.patch
>
>
> Hive CLI allows a user to execute a shell command using ! notation. For 
> instance, !cat myfile.txt. Being able to execute shell command may be 
> important for some users. As a replacement, however, Beeline provides no such 
> capability, possibly because ! notation is reserved for SQLLine commands. 
> It's possible to provide this using a slightly syntactic variation such as 
> !sh cat myfilie.txt.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6694) Beeline should provide a way to execute shell command as Hive CLI does

[Xuefu Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-6694:
--

Attachment: HIVE-6694.patch

> Beeline should provide a way to execute shell command as Hive CLI does
> --
>
> Key: HIVE-6694
> URL: https://issues.apache.org/jira/browse/HIVE-6694
> Project: Hive
>  Issue Type: Improvement
>  Components: CLI, Clients
>Affects Versions: 0.11.0, 0.12.0
>Reporter: Xuefu Zhang
>Assignee: Xuefu Zhang
> Attachments: HIVE-6694.patch
>
>
> Hive CLI allows a user to execute a shell command using ! notation. For 
> instance, !cat myfile.txt. Being able to execute shell command may be 
> important for some users. As a replacement, however, Beeline provides no such 
> capability, possibly because ! notation is reserved for SQLLine commands. 
> It's possible to provide this using a slightly syntactic variation such as 
> !sh cat myfilie.txt.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6694) Beeline should provide a way to execute shell command as Hive CLI does

[Xuefu Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-6694:
--

Status: Patch Available  (was: Open)

> Beeline should provide a way to execute shell command as Hive CLI does
> --
>
> Key: HIVE-6694
> URL: https://issues.apache.org/jira/browse/HIVE-6694
> Project: Hive
>  Issue Type: Improvement
>  Components: CLI, Clients
>Affects Versions: 0.12.0, 0.11.0
>Reporter: Xuefu Zhang
>Assignee: Xuefu Zhang
> Attachments: HIVE-6694.patch
>
>
> Hive CLI allows a user to execute a shell command using ! notation. For 
> instance, !cat myfile.txt. Being able to execute shell command may be 
> important for some users. As a replacement, however, Beeline provides no such 
> capability, possibly because ! notation is reserved for SQLLine commands. 
> It's possible to provide this using a slightly syntactic variation such as 
> !sh cat myfilie.txt.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6447) Bucket map joins in hive-tez

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948862#comment-13948862
 ] 

Hive QA commented on HIVE-6447:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12637022/HIVE-6447.12.patch

{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 5491 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_parallel_orderby
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_mapreduce_stack_trace_hadoop20
{noformat}

Test results: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1974/testReport
Console output: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1974/console

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12637022

> Bucket map joins in hive-tez
> 
>
> Key: HIVE-6447
> URL: https://issues.apache.org/jira/browse/HIVE-6447
> Project: Hive
>  Issue Type: Bug
>  Components: Tez
>Affects Versions: tez-branch
>Reporter: Vikram Dixit K
>Assignee: Vikram Dixit K
> Attachments: HIVE-6447.1.patch, HIVE-6447.10.patch, 
> HIVE-6447.11.patch, HIVE-6447.12.patch, HIVE-6447.2.patch, HIVE-6447.3.patch, 
> HIVE-6447.4.patch, HIVE-6447.5.patch, HIVE-6447.6.patch, HIVE-6447.7.patch, 
> HIVE-6447.8.patch, HIVE-6447.9.patch, HIVE-6447.WIP.patch
>
>
> Support bucket map joins in tez.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6685) Beeline throws ArrayIndexOutOfBoundsException for mismatched arguments

[Xuefu Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948822#comment-13948822
 ] 

Xuefu Zhang commented on HIVE-6685:
---

+1

> Beeline throws ArrayIndexOutOfBoundsException for mismatched arguments
> --
>
> Key: HIVE-6685
> URL: https://issues.apache.org/jira/browse/HIVE-6685
> Project: Hive
>  Issue Type: Bug
>  Components: CLI
>Affects Versions: 0.12.0
>Reporter: Szehon Ho
>Assignee: Szehon Ho
> Attachments: HIVE-6685.2.patch, HIVE-6685.3.patch, HIVE-6685.4.patch, 
> HIVE-6685.patch
>
>
> Noticed that there is an ugly ArrayIndexOutOfBoundsException for mismatched 
> arguments in beeline prompt.  It would be nice to cleanup.
> Example:
> {noformat}
> beeline -u szehon -p
> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 3
>   at org.apache.hive.beeline.BeeLine.initArgs(BeeLine.java:560)
>   at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:628)
>   at 
> org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:366)
>   at org.apache.hive.beeline.BeeLine.main(BeeLine.java:349)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:606)
>   at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: Review Request 19322: HIVE-6685 Beeline throws ArrayIndexOutOfBoundsException for mismatched arguments

[Xuefu Zhang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19322/#review38693
---

Ship it!


Ship It!

- Xuefu Zhang


On March 26, 2014, 8:32 p.m., Szehon Ho wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19322/
> ---
> 
> (Updated March 26, 2014, 8:32 p.m.)
> 
> 
> Review request for hive and Xuefu Zhang.
> 
> 
> Bugs: HIVE-6685
> https://issues.apache.org/jira/browse/HIVE-6685
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Improving the error-handling in ArrayIndexOutOfBoundsException of Beeline.
> 
> 
> Diffs
> -
> 
>   beeline/src/java/org/apache/hive/beeline/BeeLine.java 1bd7af3 
>   beeline/src/test/org/apache/hive/beeline/TestBeelineArgParsing.java 
> PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/19322/diff/
> 
> 
> Testing
> ---
> 
> Manual test.  Now, in this scenario it will display the usage like:
> 
> beeline -u
> Usage: java org.apache.hive.cli.beeline.BeeLine 
>-uthe JDBC URL to connect to
>-nthe username to connect as
>-pthe password to connect as
> ...
> 
> 
> Thanks,
> 
> Szehon Ho
> 
>

[jira] [Commented] (HIVE-6757) Remove deprecated parquet classes from outside of org.apache package

[Xuefu Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948811#comment-13948811
 ] 

Xuefu Zhang commented on HIVE-6757:
---

Yeah, I think it makes more sense to remove these deprecated parquet classes in 
certain future release. 

> Remove deprecated parquet classes from outside of org.apache package
> 
>
> Key: HIVE-6757
> URL: https://issues.apache.org/jira/browse/HIVE-6757
> Project: Hive
>  Issue Type: Bug
>Reporter: Owen O'Malley
>Assignee: Owen O'Malley
> Fix For: 0.13.0
>
>
> Apache shouldn't release projects with files outside of the org.apache 
> namespace.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6638) Hive needs to implement recovery for Application Master restart

[Mohammad Kamrul Islam (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948801#comment-13948801
 ] 

Mohammad Kamrul Islam commented on HIVE-6638:
-

In case, anyone is interested. The testing is an involved process and 
choreographed. I tested it as follows:

set mapred.map.tasks.speculative.execution=false;
set mapred.job.map.memory.mb=4096;
set hive.merge.mapfiles=false;
set hive.input.format=org.apache.hadoop.hive.ql.io.HiveInputFormat;
create table load_overwrite (key string, value string) stored as textfile;
load data  local inpath '/tmp/data/' into table load_overwrite;
select key from load_overwrite where length(key) > 0 ;

Assuming /tmp/data has four copies of kv1.txt.

Tested against Hadoop 2.3 in single node Mac machine. The four tasks will run 
kind of sequentially.
Important:  When to kill MRAM? I killed the MRAM when the second one finished. 
It could be anytime before the last one finished. Command used: "jps |grep 
MRAppMaster |cut -d' ' -f1|xargs kill"


I was monitoring in two ways:
1. cd HADOOP_LOG_DIR/userlogs/ and ran "grep  -R "New Final Path" *". 
This will show what tasks are completed with file written to  HDFS.
2. run hadoop fs -lsr hdfs://localhost:9000/tmp/hive-/. It will show all 
the tasks' output during the execution. At the end , it is cleaned up.


Anyway, if you can kill MRAM during the execution, you should see there are 
only 4 output files . More importantly, you will see the completed (before MRAM 
was killed) task never rerun. Also you get the correct result.








> Hive needs to implement recovery for Application Master restart 
> 
>
> Key: HIVE-6638
> URL: https://issues.apache.org/jira/browse/HIVE-6638
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Affects Versions: 0.11.0, 0.12.0, 0.13.0
>Reporter: Ashutosh Chauhan
>Assignee: Mohammad Kamrul Islam
> Attachments: HIVE-6638.1.patch
>
>
> Currently, if AM restarts, whole job is restarted. Although, job and 
> subsequently query would still finish to completion, it would be nice if Hive 
> don't need to redo all the work done under previous AM.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Resolved] (HIVE-5537) Add a test for ThriftCLIService async execution to ThriftCLIServiceTest

[Vaibhav Gumashta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-5537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vaibhav Gumashta resolved HIVE-5537.


Resolution: Fixed

A test has already been added.

> Add a test for ThriftCLIService async execution to ThriftCLIServiceTest
> ---
>
> Key: HIVE-5537
> URL: https://issues.apache.org/jira/browse/HIVE-5537
> Project: Hive
>  Issue Type: Test
>  Components: HiveServer2
>Affects Versions: 0.12.0, 0.13.0
>Reporter: Vaibhav Gumashta
> Fix For: 0.13.0
>
>
> Currently there is no unit test to test the async execution behaviour.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6599) Document new jdbc url connection parameters for HiveServer2

[Vaibhav Gumashta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vaibhav Gumashta updated HIVE-6599:
---

Priority: Major  (was: Critical)

> Document new jdbc url connection parameters for HiveServer2
> ---
>
> Key: HIVE-6599
> URL: https://issues.apache.org/jira/browse/HIVE-6599
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, JDBC
>Affects Versions: 0.13.0
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
> Fix For: 0.13.0
>
>
> Need to document the parameters for http mode, ssl mode and secure proxy 
> access.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Resolved] (HIVE-6503) document pluggable authentication modules (PAM) in template config, wiki

[Vaibhav Gumashta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vaibhav Gumashta resolved HIVE-6503.


Resolution: Fixed

Added release notes to original jira

> document pluggable authentication modules (PAM) in template config, wiki
> 
>
> Key: HIVE-6503
> URL: https://issues.apache.org/jira/browse/HIVE-6503
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 0.13.0
>Reporter: Thejas M Nair
>Assignee: Vaibhav Gumashta
>Priority: Blocker
> Fix For: 0.13.0
>
> Attachments: HIVE-6503.1.patch
>
>
> HIVE-6466 adds support for "PAM" as a supported value for 
> hive.server2.authentication. 
> It also adds a config parameter hive.server2.authentication.pam.services.
> The default template file needs to be updated to document these. The wiki 
> docs should also document the support for pluggable authentication modules.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6763) HiveServer2 in http mode might send same kerberos client ticket in case of concurrent requests resulting in server throwing a replay exception

[Vaibhav Gumashta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vaibhav Gumashta updated HIVE-6763:
---

Status: Patch Available  (was: Open)

> HiveServer2 in http mode might send same kerberos client ticket in case of 
> concurrent requests resulting in server throwing a replay exception
> --
>
> Key: HIVE-6763
> URL: https://issues.apache.org/jira/browse/HIVE-6763
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 0.13.0
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
> Fix For: 0.13.0
>
> Attachments: HIVE-6763.1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6763) HiveServer2 in http mode might send same kerberos client ticket in case of concurrent requests resulting in server throwing a replay exception

[Vaibhav Gumashta (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948781#comment-13948781
 ] 

Vaibhav Gumashta commented on HIVE-6763:


cc [~thejas] [~rhbutani] This is a bug for 13. Thanks!


> HiveServer2 in http mode might send same kerberos client ticket in case of 
> concurrent requests resulting in server throwing a replay exception
> --
>
> Key: HIVE-6763
> URL: https://issues.apache.org/jira/browse/HIVE-6763
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 0.13.0
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
> Fix For: 0.13.0
>
> Attachments: HIVE-6763.1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6763) HiveServer2 in http mode might send same kerberos client ticket in case of concurrent requests resulting in server throwing a replay exception

[Vaibhav Gumashta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vaibhav Gumashta updated HIVE-6763:
---

Attachment: HIVE-6763.1.patch

> HiveServer2 in http mode might send same kerberos client ticket in case of 
> concurrent requests resulting in server throwing a replay exception
> --
>
> Key: HIVE-6763
> URL: https://issues.apache.org/jira/browse/HIVE-6763
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 0.13.0
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
> Fix For: 0.13.0
>
> Attachments: HIVE-6763.1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6638) Hive needs to implement recovery for Application Master restart

[Mohammad Kamrul Islam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mohammad Kamrul Islam updated HIVE-6638:


Attachment: HIVE-6638.1.patch

Initial patch.

> Hive needs to implement recovery for Application Master restart 
> 
>
> Key: HIVE-6638
> URL: https://issues.apache.org/jira/browse/HIVE-6638
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Affects Versions: 0.11.0, 0.12.0, 0.13.0
>Reporter: Ashutosh Chauhan
>Assignee: Mohammad Kamrul Islam
> Attachments: HIVE-6638.1.patch
>
>
> Currently, if AM restarts, whole job is restarted. Although, job and 
> subsequently query would still finish to completion, it would be nice if Hive 
> don't need to redo all the work done under previous AM.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6696) Implement DBMD.getIndexInfo()

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948777#comment-13948777
 ] 

Hive QA commented on HIVE-6696:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12636839/HIVE-6696.patch

{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 5492 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketizedhiveinputformat
{noformat}

Test results: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1972/testReport
Console output: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1972/console

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12636839

> Implement DBMD.getIndexInfo()
> -
>
> Key: HIVE-6696
> URL: https://issues.apache.org/jira/browse/HIVE-6696
> Project: Hive
>  Issue Type: Improvement
>  Components: JDBC
>Affects Versions: 0.12.0
>Reporter: Jonathan Seidman
>Assignee: Szehon Ho
>Priority: Minor
> Attachments: HIVE-6696.patch
>
>
> HiveDatabaseMetaData.getIndexInfo() currently throws a "not supported" 
> exception. There seems to be no technical obstacle to implementing this to 
> return index info for tables with indexes defined, and probably an empty 
> ResultSet for tables with no indexes.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: Timeline for the Hive 0.13 release?

[Prasanth Jayachandran]

Hi Harish

Can we have the following bugs for 0.13? These bugs are related to feature 
HIVE-6455 added as part of 0.13.
https://issues.apache.org/jira/browse/HIVE-6748 (Resource leak bug)
https://issues.apache.org/jira/browse/HIVE-6760 (Bug in handling list bucketing)
https://issues.apache.org/jira/browse/HIVE-6761 (Bug with hashcodes generation)

Thanks
Prasanth Jayachandran

On Mar 26, 2014, at 1:22 PM, Hari Subramaniyan  
wrote:

> Hi Harish
> Can you include HIVE-6708. It covers quite a number of issues associated
> with Vectorization(including some correctness issues and exceptions).
> 
> Thanks
> Hari
> 
> 
> On Tue, Mar 25, 2014 at 12:01 PM, Xuefu Zhang  wrote:
> 
>> Harish,
>> 
>> Could we include HIVE-6740?
>> 
>> Thanks,
>> Xuefu
>> 
>> 
>> On Thu, Mar 20, 2014 at 7:27 PM, Prasanth Jayachandran <
>> pjayachand...@hortonworks.com> wrote:
>> 
>>> Harish,
>>> 
>>> Could you add the following bugs as well?
>>> Following are related to LazyMap bug
>>> https://issues.apache.org/jira/browse/HIVE-6707
>>> https://issues.apache.org/jira/browse/HIVE-6714
>>> https://issues.apache.org/jira/browse/HIVE-6711
>>> 
>>> Following is NPE bug with orc struct
>>> https://issues.apache.org/jira/browse/HIVE-6716
>>> 
>>> Thanks
>>> Prasanth Jayachandran
>>> 
>>> On Mar 14, 2014, at 6:26 PM, Eugene Koifman 
>>> wrote:
>>> 
 could you add https://issues.apache.org/jira/browse/HIVE-6676 please.
>>> It's
 a blocker as well.
 
 Thanks,
 Eugene
 
 
 On Fri, Mar 14, 2014 at 5:30 PM, Vaibhav Gumashta <
>>> vgumas...@hortonworks.com
> wrote:
 
> Harish,
> 
> Can we have this in as well:
> https://issues.apache.org/jira/browse/HIVE-6660.
> Blocker bug in my opinion.
> 
> Thanks,
> --Vaibhav
> 
> 
> On Fri, Mar 14, 2014 at 2:21 PM, Thejas Nair 
> wrote:
> 
>> Harish,
>> Can you also include  HIVE-6673<
>> https://issues.apache.org/jira/browse/HIVE-6673>
>> -  show grant statement for all principals throws NPE
>> This variant of 'show grant' is very useful, and the fix for NPE is
>> straightforward. It is patch available now.
>> 
>> 
>> 
>> On Fri, Mar 14, 2014 at 10:25 AM, Yin Huai 
> wrote:
>> 
>>> Guys,
>>> 
>>> Seems ConditionalResolverCommonJoin is not working correctly? I
>>> created
>>> https://issues.apache.org/jira/browse/HIVE-6668 and set it as a
> blocker.
>>> 
>>> thanks,
>>> 
>>> Yin
>>> 
>>> 
>>> On Fri, Mar 14, 2014 at 11:34 AM, Thejas Nair <
>> the...@hortonworks.com
 wrote:
>>> 
 Can you also add HIVE-6647 <
 https://issues.apache.org/jira/browse/HIVE-6647> to
 the list? It is marked as a blocker for 0.13.
 It has a necessary version number upgrade for HS2. It is ready to
>> be
 committed.
 
 
 On Fri, Mar 14, 2014 at 12:38 AM, Prasanth Jayachandran <
 pjayachand...@hortonworks.com> wrote:
 
> Harish
> 
> Can you please make the following changes to my earlier request?
> 
> HIVE-4177 is not required.. instead the same work is tracked under
> HIVE-6578.
> 
> Can you also consider HIVE-6656?
> HIVE-6656 is bug fix for ORC reader when reading timestamp
>> nanoseconds.
> This bug exists in earlier versions as well, so it will be good
> have
>>> this
> fixed in 0.13.0
> 
> Thanks
> Prasanth Jayachandran
> 
> On Mar 13, 2014, at 8:52 AM, Thejas Nair 
>>> wrote:
> 
>> Harish,
>> I think we should include the following -
>> HIVE-6547 - This is a cleanup of metastore api changes introduced
>> in
> 0.13 .
>> This can't be done post release. I will get a patch out in few
>> hours.
>> HIVE-6567 -  fixes a NPE in 'show grant .. on all"
>> HIVE-6629 - change in syntax for 'set role none' . marked as a
>>> blocker
> bug.
>> 
>> 
>> On Tue, Mar 11, 2014 at 8:39 AM, Harish Butani <
 hbut...@hortonworks.com
>> wrote:
>> 
>>> yes sure.
>>> 
>>> 
>>> On Mar 10, 2014, at 3:55 PM, Gopal V  wrote:
>>> 
 Can I add HIVE-6518 as well to the merge queue on
 
 
>>> 
> 
 
>>> 
>> 
> 
>>> 
>> https://cwiki.apache.org/confluence/display/Hive/Hive+0.13+release+status
 
 It is a relatively simple OOM safety patch to vectorized
>> group-by.
 
 Tests pass locally for vec group-by, but the pre-commit tests
>>> haven't
>>> fired eventhough it's been PA for a while now.
 
 Cheers,
 Gopal
>>> 
>>> 
>>> --
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message i

[jira] [Created] (HIVE-6763) HiveServer2 in http mode might send same kerberos client ticket in case of concurrent requests resulting in server throwing a replay exception

[Vaibhav Gumashta (JIRA)]

Vaibhav Gumashta created HIVE-6763:
--

 Summary: HiveServer2 in http mode might send same kerberos client 
ticket in case of concurrent requests resulting in server throwing a replay 
exception
 Key: HIVE-6763
 URL: https://issues.apache.org/jira/browse/HIVE-6763
 Project: Hive
  Issue Type: Bug
  Components: HiveServer2
Affects Versions: 0.13.0
Reporter: Vaibhav Gumashta
Assignee: Vaibhav Gumashta
 Fix For: 0.13.0






--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6642) Query fails to vectorize when a non string partition column is part of the query expression

[Hari Sankar Sivarama Subramaniyan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hari Sankar Sivarama Subramaniyan updated HIVE-6642:


Fix Version/s: 0.13.0

> Query fails to vectorize when a non string partition column is part of the 
> query expression
> ---
>
> Key: HIVE-6642
> URL: https://issues.apache.org/jira/browse/HIVE-6642
> Project: Hive
>  Issue Type: Bug
>Reporter: Hari Sankar Sivarama Subramaniyan
>Assignee: Hari Sankar Sivarama Subramaniyan
> Fix For: 0.13.0
>
> Attachments: HIVE-6642-2.patch, HIVE-6642-3.patch, HIVE-6642-4.patch, 
> HIVE-6642.1.patch
>
>
> drop table if exists alltypesorc_part;
> CREATE TABLE alltypesorc_part (
> ctinyint tinyint,
> csmallint smallint,
> cint int,
> cbigint bigint,
> cfloat float,
> cdouble double,
> cstring1 string,
> cstring2 string,
> ctimestamp1 timestamp,
> ctimestamp2 timestamp,
> cboolean1 boolean,
> cboolean2 boolean) partitioned by (ds int) STORED AS ORC;
> insert overwrite table alltypesorc_part partition (ds=2011) select * from 
> alltypesorc limit 100;
> insert overwrite table alltypesorc_part partition (ds=2012) select * from 
> alltypesorc limit 200;
> explain select *
> from (select ds from alltypesorc_part) t1,
>  alltypesorc t2
> where t1.ds = t2.cint
> order by t2.ctimestamp1
> limit 100;
> The above query fails to vectorize because (select ds from alltypesorc_part) 
> t1 returns a string column and the join equality on t2 is performed on an int 
> column. The correct output when vectorization is turned on should be:
> STAGE DEPENDENCIES:
>   Stage-5 is a root stage
>   Stage-2 depends on stages: Stage-5
>   Stage-0 is a root stage
> STAGE PLANS:
>   Stage: Stage-5
> Map Reduce Local Work
>   Alias -> Map Local Tables:
> t1:alltypesorc_part
>   Fetch Operator
> limit: -1
>   Alias -> Map Local Operator Tree:
> t1:alltypesorc_part
>   TableScan
> alias: alltypesorc_part
> Statistics: Num rows: 300 Data size: 62328 Basic stats: COMPLETE 
> Column stats: COMPLETE
> Select Operator
>   expressions: ds (type: int)
>   outputColumnNames: _col0
>   Statistics: Num rows: 300 Data size: 1200 Basic stats: COMPLETE 
> Column stats: COMPLETE
>   HashTable Sink Operator
> condition expressions:
>   0 {_col0}
>   1 {ctinyint} {csmallint} {cint} {cbigint} {cfloat} 
> {cdouble} {cstring1} {cstring2} {ctimestamp1} {ctimestamp2} {cboolean1} 
> {cboolean2}
> keys:
>   0 _col0 (type: int)
>   1 cint (type: int)
>   Stage: Stage-2
> Map Reduce
>   Map Operator Tree:
>   TableScan
> alias: t2
> Statistics: Num rows: 3536 Data size: 1131711 Basic stats: 
> COMPLETE Column stats: NONE
> Map Join Operator
>   condition map:
>Inner Join 0 to 1
>   condition expressions:
> 0 {_col0}
> 1 {ctinyint} {csmallint} {cint} {cbigint} {cfloat} {cdouble} 
> {cstring1} {cstring2} {ctimestamp1} {ctimestamp2} {cboolean1} {cboolean2}
>   keys:
> 0 _col0 (type: int)
> 1 cint (type: int)
>   outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, 
> _col6, _col7, _col8, _col9, _col10, _col11, _col12
>   Statistics: Num rows: 3889 Data size: 1244882 Basic stats: 
> COMPLETE Column stats: NONE
>   Filter Operator
> predicate: (_col0 = _col3) (type: boolean)
> Statistics: Num rows: 1944 Data size: 622280 Basic stats: 
> COMPLETE Column stats: NONE
> Select Operator
>   expressions: _col0 (type: int), _col1 (type: tinyint), 
> _col2 (type: smallint), _col3 (type: int), _col4 (type: bigint), _col5 (type: 
> float), _col6 (type: double), _col7 (type: string), _col8 (type: string), 
> _col\
> 9 (type: timestamp), _col10 (type: timestamp), _col11 (type: boolean), _col12 
> (type: boolean)
>   outputColumnNames: _col0, _col1, _col2, _col3, _col4, 
> _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12
>   Statistics: Num rows: 1944 Data size: 622280 Basic stats: 
> COMPLETE Column stats: NONE
>   Reduce Output Operator
> key expressions: _col9 (type: timestamp)
> sort order: +
> Statistics: Num rows: 1944 Data size: 622280 Basic stats: 
> COMPLETE Column stats: NONE
> value expressions: _col0 (type: int), _col1 (type: 
> tinyint), _col2 (type: smallint), _col3 (type: int)

[jira] [Updated] (HIVE-5836) Improve logging in ExecReducer.reduce()

[Harish Butani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-5836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Harish Butani updated HIVE-5836:


Fix Version/s: (was: 0.13.0)

> Improve logging in ExecReducer.reduce()
> ---
>
> Key: HIVE-5836
> URL: https://issues.apache.org/jira/browse/HIVE-5836
> Project: Hive
>  Issue Type: Bug
>Reporter: Hari Sankar Sivarama Subramaniyan
>Assignee: Hari Sankar Sivarama Subramaniyan
>
> Although ExecReducer.reduce() logs the used JVM heap memory, we need to log 
> the committed, max and used memory 
> (http://docs.oracle.com/javase/6/docs/api/java/lang/management/MemoryUsage.html)
>  to debug any future outofmemory issues.
> l4j.info("ExecReducer: processing " + cntr
> + " rows: used memory = " + used_memory);
> i.e. the above log info should include max_memory as well as committed_memory 
> for a propery analysis of memory usage at runtime.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-5617) Add webhcat e2e tests using 1. jobs (GET) 2. jobs/:jobid (GET) 3. jobs/:jobid (DELETE) apis

[Harish Butani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-5617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Harish Butani updated HIVE-5617:


Fix Version/s: (was: 0.13.0)

> Add webhcat e2e tests using 1. jobs (GET) 2. jobs/:jobid (GET) 3. jobs/:jobid 
> (DELETE) apis
> ---
>
> Key: HIVE-5617
> URL: https://issues.apache.org/jira/browse/HIVE-5617
> Project: Hive
>  Issue Type: Bug
>  Components: WebHCat
>Reporter: Hari Sankar Sivarama Subramaniyan
>Assignee: Hari Sankar Sivarama Subramaniyan
> Attachments: HIVE-5617.1.patch.txt
>
>
> The current e2e test driver module(TestDriverCurl.pm) uses the deprecated API 
> for hive 0.12. Use the jobs api introduced in Hive 0.12 for killing a job, 
> getting the status of a job, etc. The reference is 
> https://cwiki.apache.org/confluence/display/Hive/WebHCat+Reference



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-5796) percentjobcomplete returned by webhcat is null

[Harish Butani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Harish Butani updated HIVE-5796:


Fix Version/s: (was: 0.13.0)

> percentjobcomplete returned by webhcat is null
> --
>
> Key: HIVE-5796
> URL: https://issues.apache.org/jira/browse/HIVE-5796
> Project: Hive
>  Issue Type: Bug
>  Components: WebHCat
>Affects Versions: 0.12.0
>Reporter: Hari Sankar Sivarama Subramaniyan
>Assignee: Hari Sankar Sivarama Subramaniyan
> Attachments: HIVE-5796.1.patch.txt
>
>
> Currently the WebHCat E2E test TestMapReduce_1 fails when comparing the job 
> status field percentComplete which is returned as null, the expected value is 
> "map 100% reduce 100%".
> In the templeton.log I see the following message:
> {noformat}
> org.apache.hive.hcatalog.templeton.tool.TempletonControllerJob | Using Hadoop 
> Version: 0.23
> INFO  | 07 Nov 2013 17:14:08,106 | 
> org.apache.hive.hcatalog.templeton.tool.TempletonControllerJob | 
> WEBHCAT_CONF_DIR=null
> WARN  | 07 Nov 2013 17:14:08,106 | 
> org.apache.hive.hcatalog.templeton.tool.TempletonControllerJob | Could not 
> find 
> D:\hdp\hcatalog-0.12.0.2.0.6.0-1007\bin\null\override-container-log4j.properties.
>  Monitoring of Hadoop jobs submitted through WebHCat may be affected.
> {noformat}
> To get past this problem I explicitly set the WEBHCAT_CONF_DIR environment 
> variable to HCATALOG_HOME/etc/webhcat where 
> override-container-log4j.properties is present.
> After fixing the above and rerunning, I still see the following error:
> {noformat}
> INFO  | 07 Nov 2013 18:29:39,517 | 
> org.apache.hive.hcatalog.templeton.tool.TempletonControllerJob | 
> WEBHCAT_CONF_DIR=D:\hdp\\hcatalog-0.12.0.2.0.6.0-1007\etc\webhcat
> INFO  | 07 Nov 2013 18:29:39,517 | 
> org.apache.hive.hcatalog.templeton.tool.TempletonControllerJob | Found 
> D:\hdp\hcatalog-0.12.0.2.0.6.0-1007\etc\webhcat\override-container-log4j.properties
>  to use for job submission.
> ERROR | 07 Nov 2013 18:29:39,519 | 
> org.apache.hive.hcatalog.templeton.tool.TempletonControllerJob | 
> org.apache.hive.hcatalog.templeton.tool.TempletonControllerJob is not 
> properly initialized. Monitoring of Hadoop jobs submitted through WebHCat may 
> be affected.
> java.lang.IllegalArgumentException: Pathname /d:/hadoop/hdfs/tmp from 
> d:/hadoop/hdfs/tmp is not a valid DFS filename.
>   at 
> org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:184)
>   at 
> org.apache.hadoop.hdfs.DistributedFileSystem.access$000(DistributedFileSystem.java:92)
>   at 
> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1106)
>   at 
> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1102)
>   at 
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
>   at 
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1102)
>   at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1397)
>   at 
> org.apache.hive.hcatalog.templeton.tool.TempletonControllerJob$1.run(TempletonControllerJob.java:93)
>   at 
> org.apache.hive.hcatalog.templeton.tool.TempletonControllerJob$1.run(TempletonControllerJob.java:82)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Unknown Source)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
>   at 
> org.apache.hive.hcatalog.templeton.tool.TempletonControllerJob.copyLog4JtoFileSystem(TempletonControllerJob.java:82)
>   at 
> org.apache.hive.hcatalog.templeton.tool.TempletonControllerJob.(TempletonControllerJob.java:126)
>   at 
> org.apache.hive.hcatalog.templeton.LauncherDelegator$1.run(LauncherDelegator.java:104)
>   at 
> org.apache.hive.hcatalog.templeton.LauncherDelegator$1.run(LauncherDelegator.java:101)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Unknown Source)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
>   at 
> org.apache.hive.hcatalog.templeton.LauncherDelegator.queueAsUser(LauncherDelegator.java:101)
>   at 
> org.apache.hive.hcatalog.templeton.LauncherDelegator.enqueueController(LauncherDelegator.java:82)
>   at 
> org.apache.hive.hcatalog.templeton.JarDelegator.run(JarDelegator.java:55)
>   at 
> org.apache.hive.hcatalog.templeton.Server.mapReduceJar(Server.java:690)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
>   at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>   at java.lang.refle

[jira] [Commented] (HIVE-6757) Remove deprecated parquet classes from outside of org.apache package

[Thejas M Nair (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948760#comment-13948760
 ] 

Thejas M Nair commented on HIVE-6757:
-

Regarding hcat, it was added in 0.11 , there is a patch out there to remove the 
old deprecated classes in 0.14 .


> Remove deprecated parquet classes from outside of org.apache package
> 
>
> Key: HIVE-6757
> URL: https://issues.apache.org/jira/browse/HIVE-6757
> Project: Hive
>  Issue Type: Bug
>Reporter: Owen O'Malley
>Assignee: Owen O'Malley
> Fix For: 0.13.0
>
>
> Apache shouldn't release projects with files outside of the org.apache 
> namespace.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6223) Webhcat e2e test fail due to Pig lib directory change

[Harish Butani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Harish Butani updated HIVE-6223:


Fix Version/s: (was: 0.13.0)

> Webhcat e2e test fail due to Pig lib directory change
> -
>
> Key: HIVE-6223
> URL: https://issues.apache.org/jira/browse/HIVE-6223
> Project: Hive
>  Issue Type: Bug
>  Components: WebHCat
>Reporter: Daniel Dai
>Assignee: Deepesh Khandelwal
>
> In PIG-3676, we will change piggybank.jar to $PIG_HOME/lib, this will result 
> some webhcat test fail, which need to be fixed.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6760) Scalable dynamic partitioning should bail out properly for list bucketing

[Gunther Hagleitner (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948748#comment-13948748
 ] 

Gunther Hagleitner commented on HIVE-6760:
--

+1

> Scalable dynamic partitioning should bail out properly for list bucketing
> -
>
> Key: HIVE-6760
> URL: https://issues.apache.org/jira/browse/HIVE-6760
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor
>Affects Versions: 0.13.0, 0.14.0
>Reporter: Prasanth J
>Assignee: Prasanth J
> Fix For: 0.13.0, 0.14.0
>
> Attachments: HIVE-6760.1.patch, HIVE-6760.2.patch
>
>
> In case of list bucketing HIVE-6455 looks only at this config 
> "hive.optimize.listbucketing" to bail out. There are cases when this config 
> ("hive.optimize.listbucketing") is not set but list bucketing is enabled 
> using SKEWED BY in CREATE TABLE statement.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6129) alter exchange is implemented in inverted manner

[Harish Butani (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948747#comment-13948747
 ] 

Harish Butani commented on HIVE-6129:
-

+1

> alter exchange is implemented in inverted manner
> 
>
> Key: HIVE-6129
> URL: https://issues.apache.org/jira/browse/HIVE-6129
> Project: Hive
>  Issue Type: Bug
>Reporter: Navis
>Assignee: Navis
>Priority: Critical
> Attachments: HIVE-6129.1.patch.txt, HIVE-6129.2.patch
>
>
> see 
> https://issues.apache.org/jira/browse/HIVE-4095?focusedCommentId=13819885&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13819885
> alter exchange should be implemented accord to document in 
> https://cwiki.apache.org/confluence/display/Hive/Exchange+Partition. i.e 
> {code}
> alter table T1 exchange partition (ds='1') with table T2" 
> {code}
> should be (after creating T1@ds=1) 
> {quote}
> moves the data from T2 to T1@ds=1 
> {quote}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6761) Hashcode computation does not use maximum parallelism for scalable dynamic partitioning

[Gunther Hagleitner (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948741#comment-13948741
 ] 

Gunther Hagleitner commented on HIVE-6761:
--

Looks good to me. +1

> Hashcode computation does not use maximum parallelism for scalable dynamic 
> partitioning
> ---
>
> Key: HIVE-6761
> URL: https://issues.apache.org/jira/browse/HIVE-6761
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor
>Affects Versions: 0.13.0, 0.14.0
>Reporter: Prasanth J
>Assignee: Prasanth J
> Fix For: 0.13.0, 0.14.0
>
> Attachments: HIVE-6761.1.patch
>
>
> Hashcode computation for HIVE-6455 should consider all the partitioning 
> columns and bucket number to distribute the rows. The following code 
> {code}
> for (int i = 0; i < partitionEval.length - 1; i++) {
> {code}
> ignores the last partition column thereby generating lesser hashcodes.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6757) Remove deprecated parquet classes from outside of org.apache package

[Thejas M Nair (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948738#comment-13948738
 ] 

Thejas M Nair commented on HIVE-6757:
-

I am not sure if we need to have by-laws for things like this. For example we 
don't have by-law for indentation in code, or adding tests with patches.



> Remove deprecated parquet classes from outside of org.apache package
> 
>
> Key: HIVE-6757
> URL: https://issues.apache.org/jira/browse/HIVE-6757
> Project: Hive
>  Issue Type: Bug
>Reporter: Owen O'Malley
>Assignee: Owen O'Malley
> Fix For: 0.13.0
>
>
> Apache shouldn't release projects with files outside of the org.apache 
> namespace.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6744) Permanent UDF lookup fails when current DB has uppercase letters

[Thejas M Nair (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948732#comment-13948732
 ] 

Thejas M Nair commented on HIVE-6744:
-

+1 .
Lets just remove that commented line in test. We don't need to run tests again 
for that.


> Permanent UDF lookup fails when current DB has uppercase letters
> 
>
> Key: HIVE-6744
> URL: https://issues.apache.org/jira/browse/HIVE-6744
> Project: Hive
>  Issue Type: Bug
>  Components: UDF
>Affects Versions: 0.13.0
>Reporter: Jason Dere
>Assignee: Jason Dere
> Attachments: HIVE-6744.1.patch, HIVE-6744.1.patch
>
>
> If defaulting to current DB/schema name for resolving UDF name, the DB name 
> should be lowercased before doing the UDF lookup.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: Review Request 19623: HIVE-6744: Permanent UDF lookup fails when current DB has uppercase letters

[Thejas Nair]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19623/#review38688
---



itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcWithMiniMr.java


This comment is not related to this patch. But I would expect the right 
behavior for 'describe function ' on a function that does not exist to result 
in a query failure. ie a SQLException .




itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcWithMiniMr.java


lets remove this commented line.



- Thejas Nair


On March 25, 2014, 6:02 p.m., Jason Dere wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19623/
> ---
> 
> (Updated March 25, 2014, 6:02 p.m.)
> 
> 
> Review request for hive and Thejas Nair.
> 
> 
> Bugs: HIVE-6744
> https://issues.apache.org/jira/browse/HIVE-6744
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Make sure DB name is lowercased when doing lookups for permanent UDFs.
> 
> 
> Diffs
> -
> 
>   itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcWithMiniMr.java 
> 885803b 
>   itests/hive-unit/src/test/java/org/apache/hive/jdbc/miniHS2/MiniHS2.java 
> d76750c 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java b7da95e 
> 
> Diff: https://reviews.apache.org/r/19623/diff/
> 
> 
> Testing
> ---
> 
> test added
> 
> 
> Thanks,
> 
> Jason Dere
> 
>

[jira] [Updated] (HIVE-6662) Vector Join operations with DATE columns fail

[Gopal V (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gopal V updated HIVE-6662:
--

Release Note: Add VectorColumnAssignFactory methods to assign DATE into 
LongColumn vectors
  Status: Patch Available  (was: Open)

> Vector Join operations with DATE columns fail
> -
>
> Key: HIVE-6662
> URL: https://issues.apache.org/jira/browse/HIVE-6662
> Project: Hive
>  Issue Type: Bug
>Reporter: Gopal V
>Assignee: Gopal V
> Attachments: HIVE-6662.1.patch, HIVE-6662.2.patch
>
>
> Trying to generate a DATE column as part of a JOIN's output throws an 
> exception
> {code}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Incompatible 
> Long vector column and primitive category DATE
> at 
> org.apache.hadoop.hive.ql.exec.vector.VectorColumnAssignFactory.buildObjectAssign(VectorColumnAssignFactory.java:306)
> at 
> org.apache.hadoop.hive.ql.exec.vector.VectorColumnAssignFactory.buildAssigners(VectorColumnAssignFactory.java:414)
> at 
> org.apache.hadoop.hive.ql.exec.vector.VectorMapJoinOperator.internalForward(VectorMapJoinOperator.java:235)
> at 
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genAllOneUniqueJoinObject(CommonJoinOperator.java:670)
> at 
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.checkAndGenObject(CommonJoinOperator.java:754)
> at 
> org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:229)
> at 
> org.apache.hadoop.hive.ql.exec.vector.VectorMapJoinOperator.processOp(VectorMapJoinOperator.java:292)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:790)
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Assigned] (HIVE-6492) limit partition number involved in a table scan

[Xuefu Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang reassigned HIVE-6492:
-

Assignee: Selina Zhang  (was: Ashutosh Chauhan)

It seemed Selina provided the patch. Assigned to her accordingly.

Thanks to Selina for the contribution.

> limit partition number involved in a table scan
> ---
>
> Key: HIVE-6492
> URL: https://issues.apache.org/jira/browse/HIVE-6492
> Project: Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Affects Versions: 0.12.0
>Reporter: Selina Zhang
>Assignee: Selina Zhang
> Fix For: 0.13.0
>
> Attachments: HIVE-6492.1.patch.txt, HIVE-6492.2.patch.txt, 
> HIVE-6492.3.patch.txt, HIVE-6492.4.patch.txt, HIVE-6492.4.patch_suggestion, 
> HIVE-6492.5.patch.txt, HIVE-6492.6.patch.txt, HIVE-6492.7.parch.txt
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> To protect the cluster, a new configure variable 
> "hive.limit.query.max.table.partition" is added to hive configuration to
> limit the table partitions involved in a table scan. 
> The default value will be set to -1 which means there is no limit by default. 
> This variable will not affect "metadata only" query.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6757) Remove deprecated parquet classes from outside of org.apache package

[Xuefu Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948719#comment-13948719
 ] 

Xuefu Zhang commented on HIVE-6757:
---

I'm not sure what the motivation is. The inclusion of the code violates no 
bylaws of either Hive or Apache. For those who don't use Parquet, they probably 
never deal with these classes, but for those who do, these are important to 
them. I'm not sure what we gain by removing them.

If we think this is important for Hive, I'd be open to consider amending the 
current by-laws to apply going forward. In absence of such bylaw, I don't see a 
strong argument for the removal.

As to the case of HCat, I'm wondering how many releases have passed before it 
lands on the current package structure.

> Remove deprecated parquet classes from outside of org.apache package
> 
>
> Key: HIVE-6757
> URL: https://issues.apache.org/jira/browse/HIVE-6757
> Project: Hive
>  Issue Type: Bug
>Reporter: Owen O'Malley
>Assignee: Owen O'Malley
> Fix For: 0.13.0
>
>
> Apache shouldn't release projects with files outside of the org.apache 
> namespace.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6697) HiveServer2 secure thrift/http authentication needs to support SPNego

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948700#comment-13948700
 ] 

Hive QA commented on HIVE-6697:
---



{color:red}Overall{color}: -1 no tests executed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12636850/HIVE-6697.3.patch

Test results: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1971/testReport
Console output: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1971/console

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit 
status 1 and output '+ [[ -n '' ]]
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost 
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost 
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-Build-1971/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ svn = \s\v\n ]]
+ [[ -n '' ]]
+ [[ -d apache-svn-trunk-source ]]
+ [[ ! -d apache-svn-trunk-source/.svn ]]
+ [[ ! -d apache-svn-trunk-source ]]
+ cd apache-svn-trunk-source
+ svn revert -R .
Reverted 'common/src/java/org/apache/hadoop/hive/common/JavaUtils.java'
++ egrep -v '^X|^Performing status on external'
++ awk '{print $2}'
++ svn status --no-ignore
+ rm -rf target datanucleus.log ant/target shims/target shims/0.20/target 
shims/0.20S/target shims/0.23/target shims/aggregator/target 
shims/common/target shims/common-secure/target packaging/target 
hbase-handler/target testutils/target jdbc/target metastore/target 
itests/target itests/hcatalog-unit/target itests/test-serde/target 
itests/qtest/target itests/hive-unit/target itests/custom-serde/target 
itests/util/target hcatalog/target hcatalog/storage-handlers/hbase/target 
hcatalog/server-extensions/target hcatalog/core/target 
hcatalog/webhcat/svr/target hcatalog/webhcat/java-client/target 
hcatalog/hcatalog-pig-adapter/target hwi/target common/target common/src/gen 
contrib/target service/target serde/target beeline/target odbc/target 
cli/target ql/dependency-reduced-pom.xml ql/target
+ svn update
Ucommon/src/java/org/apache/hadoop/hive/common/JavaUtils.java
Ucommon/src/java/org/apache/hadoop/hive/conf/HiveConf.java
Umetastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
Aql/src/test/queries/clientnegative/limit_partition_stats.q
Aql/src/test/queries/clientnegative/limit_partition.q
Uql/src/test/queries/clientpositive/authorization_9.q
Aql/src/test/queries/clientpositive/limit_partition_metadataonly.q
Aql/src/test/results/clientnegative/limit_partition_stats.q.out
Aql/src/test/results/clientnegative/limit_partition.q.out
Aql/src/test/results/clientpositive/limit_partition_metadataonly.q.out
Uql/src/test/results/clientpositive/authorization_9.q.out
Uql/src/java/org/apache/hadoop/hive/ql/plan/TableScanDesc.java
Uql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
Uql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java
U
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/MetadataOnlyOptimizer.java

Fetching external item into 'hcatalog/src/test/e2e/harness'
Updated external to revision 1582137.

Updated to revision 1582137.
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh 
/data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12636850

> HiveServer2 secure thrift/http authentication needs to support SPNego 
> --
>
> Key: HIVE-6697
> URL: https://issues.apache.org/jira/browse/HIVE-6697
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Reporter: Dilli Arumugam
>Assignee: Dilli Arumugam
> Attachments: HIVE-6697.1.patch, HIVE-6697.2.patch, HIVE-6697.3.patch, 
> hive-6697-req-impl-verify.md
>
>
> Looking to integrating Apache Knox to work with HiveServer2 secure 
> thrift/http.
> Found that thrift/http uses some form of Kerberos authentication that is not 
> SPNego. Considering it is going over http protocol, expected it to use SPNego 
> protocol.
> Apache Knox is already integrated with WebHDFS, WebHCat, Oozie and HBase 
> Stargate using SPNego for authentication.
>

[jira] [Commented] (HIVE-6750) Hive printing debug information in stdout after the end of CLI session

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948698#comment-13948698
 ] 

Hive QA commented on HIVE-6750:
---



{color:green}Overall{color}: +1 all checks pass

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12636843/HIVE-6750.2.patch

{color:green}SUCCESS:{color} +1 5488 tests passed

Test results: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1970/testReport
Console output: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1970/console

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12636843

> Hive printing debug information in stdout after the end of CLI session
> --
>
> Key: HIVE-6750
> URL: https://issues.apache.org/jira/browse/HIVE-6750
> Project: Hive
>  Issue Type: Bug
>  Components: CLI
>Affects Versions: 0.14.0
> Environment: Open JDK6
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
> Fix For: 0.13.0
>
> Attachments: HIVE-6750.1.patch, HIVE-6750.2.patch
>
>
> {noformat}
> $ hive -e "show tables;"
> OK
> table1
> table2
> table3
> Time taken: 0.694 seconds, Fetched: 3 row(s)
> classLoader = java.net.URLClassLoader@29978933
> SharedSecrets.getJavaNetAccess()=java.net.URLClassLoader$7@4b7d03c5
> {noformat}
> Observe the last two lines that are printed upon the end of Hive CLI session. 
> JavaUtils#closeClassLoader uses reflection to invoke 
> sun.misc.ClassLoaderUtil.releaseLoader. Just peeking at the implementation of 
> this method clearly showed this:
> {code}
> System.out.println ("classLoader = " + classLoader);
> System.out.println 
> ("SharedSecrets.getJavaNetAccess()="+SharedSecrets.getJavaNetAccess());
> {code}
> See line 80-81 at 
> http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/sun/misc/ClassLoaderUtil.java.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6674) "show grant on all" throws NPE

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-6674:
---

   Resolution: Fixed
Fix Version/s: 0.13.0
   Status: Resolved  (was: Patch Available)

Committed to 0.13 & trunk.

> "show grant on all" throws NPE
> --
>
> Key: HIVE-6674
> URL: https://issues.apache.org/jira/browse/HIVE-6674
> Project: Hive
>  Issue Type: Bug
>Reporter: Thejas M Nair
>Assignee: Navis
> Fix For: 0.13.0
>
> Attachments: HIVE-6674.1.patch.txt, HIVE-6674.2.patch.txt
>
>
> "show grant on all" is supposed to show all the grants in the system on all 
> the objects. But it fails with NPE with both SQL standard auth, and legacy 
> auth.
> {code}
> Caused by: java.lang.NullPointerException
> at 
> org.apache.hadoop.hive.metastore.ObjectStore.listPrincipalAllDBGrant(ObjectStore.java:4206)
> at 
> org.apache.hadoop.hive.metastore.ObjectStore.listPrincipalDBGrantsAll(ObjectStore.java:4169)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at 
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:108)
> at com.sun.proxy.$Proxy6.listPrincipalDBGrantsAll(Unknown Source)
> at 
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.list_db_privileges(HiveMetaStore.java:4295)
> ... 36 more
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6750) Hive printing debug information in stdout after the end of CLI session

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-6750:
---

   Resolution: Fixed
Fix Version/s: (was: 0.14.0)
   0.13.0
   Status: Resolved  (was: Patch Available)

Committed to trunk & 0.13. Thanks, Vaibhav!

> Hive printing debug information in stdout after the end of CLI session
> --
>
> Key: HIVE-6750
> URL: https://issues.apache.org/jira/browse/HIVE-6750
> Project: Hive
>  Issue Type: Bug
>  Components: CLI
>Affects Versions: 0.14.0
> Environment: Open JDK6
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
> Fix For: 0.13.0
>
> Attachments: HIVE-6750.1.patch, HIVE-6750.2.patch
>
>
> {noformat}
> $ hive -e "show tables;"
> OK
> table1
> table2
> table3
> Time taken: 0.694 seconds, Fetched: 3 row(s)
> classLoader = java.net.URLClassLoader@29978933
> SharedSecrets.getJavaNetAccess()=java.net.URLClassLoader$7@4b7d03c5
> {noformat}
> Observe the last two lines that are printed upon the end of Hive CLI session. 
> JavaUtils#closeClassLoader uses reflection to invoke 
> sun.misc.ClassLoaderUtil.releaseLoader. Just peeking at the implementation of 
> this method clearly showed this:
> {code}
> System.out.println ("classLoader = " + classLoader);
> System.out.println 
> ("SharedSecrets.getJavaNetAccess()="+SharedSecrets.getJavaNetAccess());
> {code}
> See line 80-81 at 
> http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/sun/misc/ClassLoaderUtil.java.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6738) HiveServer2 secure Thrift/HTTP needs to accept doAs parameter from proxying intermediary

[Dilli Arumugam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dilli Arumugam updated HIVE-6738:
-

Attachment: HIVE-6738.patch

Patch implementing the enhancement.
This has to go in after the patch for HIVE-6697 that is pending in queue is 
merged.


> HiveServer2 secure Thrift/HTTP needs to accept doAs parameter from proxying 
> intermediary
> 
>
> Key: HIVE-6738
> URL: https://issues.apache.org/jira/browse/HIVE-6738
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Reporter: Dilli Arumugam
>Assignee: Dilli Arumugam
> Attachments: HIVE-6738.patch, hive-6738-req-impl-verify.md
>
>
> See already implemented JIra
>  https://issues.apache.org/jira/browse/HIVE-5155
> Support secure proxy user access to HiveServer2
> That fix expects the hive.server2.proxy.user parameter to come in Thrift body.
> When an intermediary gateway like Apache Knox is authenticating the end 
> client and then proxying the request to HiveServer2,  it is not practical for 
> the intermediary like Apache Knox to modify thrift content.
> Intermediary like Apache Knox should be able to assert doAs in a query 
> parameter. This paradigm is already established by other Hadoop ecosystem 
> components like WebHDFS, WebHCat, Oozie and HBase and Hive needs to be 
> aligned with them.
> The doAs asserted in query parameter should override if doAs specified in 
> Thrift body.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Work started] (HIVE-6738) HiveServer2 secure Thrift/HTTP needs to accept doAs parameter from proxying intermediary

[Dilli Arumugam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HIVE-6738 started by Dilli Arumugam.

> HiveServer2 secure Thrift/HTTP needs to accept doAs parameter from proxying 
> intermediary
> 
>
> Key: HIVE-6738
> URL: https://issues.apache.org/jira/browse/HIVE-6738
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Reporter: Dilli Arumugam
>Assignee: Dilli Arumugam
> Attachments: hive-6738-req-impl-verify.md
>
>
> See already implemented JIra
>  https://issues.apache.org/jira/browse/HIVE-5155
> Support secure proxy user access to HiveServer2
> That fix expects the hive.server2.proxy.user parameter to come in Thrift body.
> When an intermediary gateway like Apache Knox is authenticating the end 
> client and then proxying the request to HiveServer2,  it is not practical for 
> the intermediary like Apache Knox to modify thrift content.
> Intermediary like Apache Knox should be able to assert doAs in a query 
> parameter. This paradigm is already established by other Hadoop ecosystem 
> components like WebHDFS, WebHCat, Oozie and HBase and Hive needs to be 
> aligned with them.
> The doAs asserted in query parameter should override if doAs specified in 
> Thrift body.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6738) HiveServer2 secure Thrift/HTTP needs to accept doAs parameter from proxying intermediary

[Dilli Arumugam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dilli Arumugam updated HIVE-6738:
-

Attachment: hive-6738-req-impl-verify.md

short description of requirement, some implementation details and verification 
of the enhancement

> HiveServer2 secure Thrift/HTTP needs to accept doAs parameter from proxying 
> intermediary
> 
>
> Key: HIVE-6738
> URL: https://issues.apache.org/jira/browse/HIVE-6738
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Reporter: Dilli Arumugam
>Assignee: Dilli Arumugam
> Attachments: hive-6738-req-impl-verify.md
>
>
> See already implemented JIra
>  https://issues.apache.org/jira/browse/HIVE-5155
> Support secure proxy user access to HiveServer2
> That fix expects the hive.server2.proxy.user parameter to come in Thrift body.
> When an intermediary gateway like Apache Knox is authenticating the end 
> client and then proxying the request to HiveServer2,  it is not practical for 
> the intermediary like Apache Knox to modify thrift content.
> Intermediary like Apache Knox should be able to assert doAs in a query 
> parameter. This paradigm is already established by other Hadoop ecosystem 
> components like WebHDFS, WebHCat, Oozie and HBase and Hive needs to be 
> aligned with them.
> The doAs asserted in query parameter should override if doAs specified in 
> Thrift body.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6492) limit partition number involved in a table scan

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-6492:
---

Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed to 0.13 & trunk. Thanks, Selina!

> limit partition number involved in a table scan
> ---
>
> Key: HIVE-6492
> URL: https://issues.apache.org/jira/browse/HIVE-6492
> Project: Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Affects Versions: 0.12.0
>Reporter: Selina Zhang
>Assignee: Ashutosh Chauhan
> Fix For: 0.13.0
>
> Attachments: HIVE-6492.1.patch.txt, HIVE-6492.2.patch.txt, 
> HIVE-6492.3.patch.txt, HIVE-6492.4.patch.txt, HIVE-6492.4.patch_suggestion, 
> HIVE-6492.5.patch.txt, HIVE-6492.6.patch.txt, HIVE-6492.7.parch.txt
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> To protect the cluster, a new configure variable 
> "hive.limit.query.max.table.partition" is added to hive configuration to
> limit the table partitions involved in a table scan. 
> The default value will be set to -1 which means there is no limit by default. 
> This variable will not affect "metadata only" query.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HIVE-6762) VectorGroupBy operator should produce vectorized row batch and let a terminal operator convert to row mode.

[Jitendra Nath Pandey (JIRA)]

Jitendra Nath Pandey created HIVE-6762:
--

 Summary: VectorGroupBy operator should produce vectorized row 
batch and let a terminal operator convert to row mode.
 Key: HIVE-6762
 URL: https://issues.apache.org/jira/browse/HIVE-6762
 Project: Hive
  Issue Type: Bug
Reporter: Jitendra Nath Pandey


VectorGroupBy operator should produce vectorized row batch and let a terminal 
operator convert to row mode.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6734) DDL locking too course grained in new db txn manager

[Alan Gates (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Gates updated HIVE-6734:
-

Status: Patch Available  (was: Open)

> DDL locking too course grained in new db txn manager
> 
>
> Key: HIVE-6734
> URL: https://issues.apache.org/jira/browse/HIVE-6734
> Project: Hive
>  Issue Type: Bug
>  Components: Locking
>Affects Versions: 0.13.0
>Reporter: Alan Gates
>Assignee: Alan Gates
> Fix For: 0.13.0
>
> Attachments: HIVE-6734.patch
>
>
> All DDL operations currently acquire an exclusive lock.  This is too course 
> grained, as some operations like alter table add partition shouldn't get an 
> exclusive lock on the entire table.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6670) ClassNotFound with Serde

[Jason Dere (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948677#comment-13948677
 ] 

Jason Dere commented on HIVE-6670:
--

+1

> ClassNotFound with Serde
> 
>
> Key: HIVE-6670
> URL: https://issues.apache.org/jira/browse/HIVE-6670
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.12.0
>Reporter: Abin Shahab
>Assignee: Abin Shahab
> Attachments: HIVE-6670-branch-0.12.patch, HIVE-6670.1.patch, 
> HIVE-6670.patch
>
>
> We are finding a ClassNotFound exception when we use 
> CSVSerde(https://github.com/ogrodnek/csv-serde) to create a table.
> This is happening because MapredLocalTask does not pass the local added jars 
> to ExecDriver when that is launched.
> ExecDriver's classpath does not include the added jars. Therefore, when the 
> plan is deserialized, it throws a ClassNotFoundException in the 
> deserialization code, and results in a TableDesc object with a Null 
> DeserializerClass.
> This results in an NPE during Fetch.
> Steps to reproduce:
> wget 
> https://drone.io/github.com/ogrodnek/csv-serde/files/target/csv-serde-1.1.2-0.11.0-all.jar
>  into somewhere local eg. 
> /home/soam/HiveSerdeIssue/csv-serde-1.1.2-0.11.0-all.jar.
> Place some sample SCV files in HDFS as follows:
> hdfs dfs -mkdir /user/soam/HiveSerdeIssue/sampleCSV/
> hdfs dfs -put /home/soam/sampleCSV.csv /user/soam/HiveSerdeIssue/sampleCSV/
> hdfs dfs -mkdir /user/soam/HiveSerdeIssue/sampleJoinTarget/
> hdfs dfs -put /home/soam/sampleJoinTarget.csv 
> /user/soam/HiveSerdeIssue/sampleJoinTarget/
> 
> create the tables in hive:
> ADD JAR /home/soam/HiveSerdeIssue/csv-serde-1.1.2-0.11.0-all.jar;
> create external table sampleCSV (md5hash string, filepath string)
> row format serde 'com.bizo.hive.serde.csv.CSVSerde'
> stored as textfile
> location '/user/soam/HiveSerdeIssue/sampleCSV/'
> ;
> create external table sampleJoinTarget (md5hash string, filepath string, 
> datestamp string, nblines string, nberrors string)
> ROW FORMAT DELIMITED 
> FIELDS TERMINATED BY ',' 
> LINES TERMINATED BY '\n'
> STORED AS TEXTFILE
> LOCATION '/user/soam/HiveSerdeIssue/sampleJoinTarget/'
> ;
> ===
> Now, try the following JOIN:
> ADD JAR /home/soam/HiveSerdeIssue/csv-serde-1.1.2-0.11.0-all.jar;
> SELECT 
> sampleCSV.md5hash, 
> sampleCSV.filepath 
> FROM sampleCSV
> JOIN sampleJoinTarget
> ON (sampleCSV.md5hash = sampleJoinTarget.md5hash) 
> ;
> —
> This will fail with the error:
> Execution log at: /tmp/soam/.log
> java.lang.ClassNotFoundException: com/bizo/hive/serde/csv/CSVSerde
> Continuing ...
> 2014-03-11 10:35:03 Starting to launch local task to process map join; 
> maximum memory = 238551040
> Execution failed with exit status: 2
> Obtaining error information
> Task failed!
> Task ID:
> Stage-4
> Logs:
> /var/log/hive/soam/hive.log
> FAILED: Execution Error, return code 2 from 
> org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> Try the following LEFT JOIN. This will work:
> SELECT 
> sampleCSV.md5hash, 
> sampleCSV.filepath 
> FROM sampleCSV
> LEFT JOIN sampleJoinTarget
> ON (sampleCSV.md5hash = sampleJoinTarget.md5hash) 
> ;
> ==



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6734) DDL locking too course grained in new db txn manager

[Alan Gates (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Gates updated HIVE-6734:
-

Attachment: HIVE-6734.patch

> DDL locking too course grained in new db txn manager
> 
>
> Key: HIVE-6734
> URL: https://issues.apache.org/jira/browse/HIVE-6734
> Project: Hive
>  Issue Type: Bug
>  Components: Locking
>Affects Versions: 0.13.0
>Reporter: Alan Gates
>Assignee: Alan Gates
> Fix For: 0.13.0
>
> Attachments: HIVE-6734.patch
>
>
> All DDL operations currently acquire an exclusive lock.  This is too course 
> grained, as some operations like alter table add partition shouldn't get an 
> exclusive lock on the entire table.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6129) alter exchange is implemented in inverted manner

[Alan Gates (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948673#comment-13948673
 ] 

Alan Gates commented on HIVE-6129:
--

Ran tests on the latest patch, all looks good.

> alter exchange is implemented in inverted manner
> 
>
> Key: HIVE-6129
> URL: https://issues.apache.org/jira/browse/HIVE-6129
> Project: Hive
>  Issue Type: Bug
>Reporter: Navis
>Assignee: Navis
>Priority: Critical
> Attachments: HIVE-6129.1.patch.txt, HIVE-6129.2.patch
>
>
> see 
> https://issues.apache.org/jira/browse/HIVE-4095?focusedCommentId=13819885&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13819885
> alter exchange should be implemented accord to document in 
> https://cwiki.apache.org/confluence/display/Hive/Exchange+Partition. i.e 
> {code}
> alter table T1 exchange partition (ds='1') with table T2" 
> {code}
> should be (after creating T1@ds=1) 
> {quote}
> moves the data from T2 to T1@ds=1 
> {quote}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6314) The logging (progress reporting) is too verbose

[Alan Gates (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948674#comment-13948674
 ] 

Alan Gates commented on HIVE-6314:
--

Ran tests on the latest patch, all looks good.

> The logging (progress reporting) is too verbose
> ---
>
> Key: HIVE-6314
> URL: https://issues.apache.org/jira/browse/HIVE-6314
> Project: Hive
>  Issue Type: Bug
>Reporter: Sam
>Assignee: Navis
>  Labels: logger
> Attachments: HIVE-6314.1.patch.txt, HIVE-6314.2.patch
>
>
> The progress report is issued every second even when no progress have been 
> made:
> {code}
> 2014-01-27 10:35:55,209 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 6.68 
> sec
> 2014-01-27 10:35:56,678 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 6.68 
> sec
> 2014-01-27 10:35:59,344 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 6.68 
> sec
> 2014-01-27 10:36:01,268 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 
> 8.67 sec
> 2014-01-27 10:36:03,149 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 
> 8.67 sec
> {code}
> This pollutes the logs and the screen, and people do not appreciate it as 
> much as the designers might have thought 
> ([http://stackoverflow.com/questions/20849289/how-do-i-limit-log-verbosity-of-hive],
>  
> [http://stackoverflow.com/questions/14121543/controlling-the-level-of-verbosity-in-hive]).
> It would be nice to be able to control the level of verbosity (but *not* by 
> the {{-v}} switch!):
> # Make sure that the progress report is only issued where there is something 
> new to report; or
> # Remove all the progress messages; or
> # Make sure that progress is reported only every X sec (instead of every 1 
> second)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6674) "show grant on all" throws NPE

[Alan Gates (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948668#comment-13948668
 ] 

Alan Gates commented on HIVE-6674:
--

Ran the tests, all looks good.

> "show grant on all" throws NPE
> --
>
> Key: HIVE-6674
> URL: https://issues.apache.org/jira/browse/HIVE-6674
> Project: Hive
>  Issue Type: Bug
>Reporter: Thejas M Nair
>Assignee: Navis
> Attachments: HIVE-6674.1.patch.txt, HIVE-6674.2.patch.txt
>
>
> "show grant on all" is supposed to show all the grants in the system on all 
> the objects. But it fails with NPE with both SQL standard auth, and legacy 
> auth.
> {code}
> Caused by: java.lang.NullPointerException
> at 
> org.apache.hadoop.hive.metastore.ObjectStore.listPrincipalAllDBGrant(ObjectStore.java:4206)
> at 
> org.apache.hadoop.hive.metastore.ObjectStore.listPrincipalDBGrantsAll(ObjectStore.java:4169)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at 
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:108)
> at com.sun.proxy.$Proxy6.listPrincipalDBGrantsAll(Unknown Source)
> at 
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.list_db_privileges(HiveMetaStore.java:4295)
> ... 36 more
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6757) Remove deprecated parquet classes from outside of org.apache package

[Thejas M Nair (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948663#comment-13948663
 ] 

Thejas M Nair commented on HIVE-6757:
-

If we look at precedence for this in hive community, there has lot of work put 
into keeping the package structure clean.  All the hcatalog classes were moved 
into org.apache.hive. They old packaging is being deprecated in hive 0.14. And 
that already in org.apache.* . 

cc [~cwsteinbach]


> Remove deprecated parquet classes from outside of org.apache package
> 
>
> Key: HIVE-6757
> URL: https://issues.apache.org/jira/browse/HIVE-6757
> Project: Hive
>  Issue Type: Bug
>Reporter: Owen O'Malley
>Assignee: Owen O'Malley
> Fix For: 0.13.0
>
>
> Apache shouldn't release projects with files outside of the org.apache 
> namespace.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: Review Request 19705: Fix add jar problem with map join.

[Jason Dere]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19705/#review38675
---

Ship it!


Ship It!

- Jason Dere


On March 26, 2014, 11:08 p.m., Ashutosh Chauhan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19705/
> ---
> 
> (Updated March 26, 2014, 11:08 p.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-6670
> https://issues.apache.org/jira/browse/HIVE-6670
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Fix add jar problem with map join.
> 
> 
> Diffs
> -
> 
>   data/files/sample.json PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java 63f41ae 
>   ql/src/test/queries/clientpositive/mapjoin_addjar.q PRE-CREATION 
>   ql/src/test/results/clientpositive/mapjoin_addjar.q.out PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/19705/diff/
> 
> 
> Testing
> ---
> 
> Added new test.
> 
> 
> Thanks,
> 
> Ashutosh Chauhan
> 
>

[jira] [Commented] (HIVE-6757) Remove deprecated parquet classes from outside of org.apache package

[Brock Noland (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948655#comment-13948655
 ] 

Brock Noland commented on HIVE-6757:


bq. There is no pressing need for them to be in Hive's jar

There is no pressing need to remove them either.

> Remove deprecated parquet classes from outside of org.apache package
> 
>
> Key: HIVE-6757
> URL: https://issues.apache.org/jira/browse/HIVE-6757
> Project: Hive
>  Issue Type: Bug
>Reporter: Owen O'Malley
>Assignee: Owen O'Malley
> Fix For: 0.13.0
>
>
> Apache shouldn't release projects with files outside of the org.apache 
> namespace.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6750) Hive printing debug information in stdout after the end of CLI session

[Alan Gates (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948651#comment-13948651
 ] 

Alan Gates commented on HIVE-6750:
--

Ran the tests, all looks good.

> Hive printing debug information in stdout after the end of CLI session
> --
>
> Key: HIVE-6750
> URL: https://issues.apache.org/jira/browse/HIVE-6750
> Project: Hive
>  Issue Type: Bug
>  Components: CLI
>Affects Versions: 0.14.0
> Environment: Open JDK6
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
> Fix For: 0.14.0
>
> Attachments: HIVE-6750.1.patch, HIVE-6750.2.patch
>
>
> {noformat}
> $ hive -e "show tables;"
> OK
> table1
> table2
> table3
> Time taken: 0.694 seconds, Fetched: 3 row(s)
> classLoader = java.net.URLClassLoader@29978933
> SharedSecrets.getJavaNetAccess()=java.net.URLClassLoader$7@4b7d03c5
> {noformat}
> Observe the last two lines that are printed upon the end of Hive CLI session. 
> JavaUtils#closeClassLoader uses reflection to invoke 
> sun.misc.ClassLoaderUtil.releaseLoader. Just peeking at the implementation of 
> this method clearly showed this:
> {code}
> System.out.println ("classLoader = " + classLoader);
> System.out.println 
> ("SharedSecrets.getJavaNetAccess()="+SharedSecrets.getJavaNetAccess());
> {code}
> See line 80-81 at 
> http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/sun/misc/ClassLoaderUtil.java.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6735) Make scalable dynamic partitioning work in vectorized mode

[Prasanth J (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasanth J updated HIVE-6735:
-

Attachment: HIVE-6735.3.patch

Same as .1 patch with some redundant code removed.

> Make scalable dynamic partitioning work in vectorized mode
> --
>
> Key: HIVE-6735
> URL: https://issues.apache.org/jira/browse/HIVE-6735
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor
>Affects Versions: 0.13.0, 0.14.0
>Reporter: Prasanth J
>Assignee: Prasanth J
> Fix For: 0.13.0, 0.14.0
>
> Attachments: HIVE-6735.1.patch, HIVE-6735.2.patch, HIVE-6735.2.patch, 
> HIVE-6735.3.patch
>
>
> HIVE-6455 added support for scalable dynamic partitioning. This is subtask to 
> make HIVE-6455 work with vectorized operators.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6735) Make scalable dynamic partitioning work in vectorized mode

[Prasanth J (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948647#comment-13948647
 ] 

Prasanth J commented on HIVE-6735:
--

[~gopalv] the reason for the lesser hashcode generation is mentioned in 
HIVE-6761. I will fix this patch to include all partition columns.

> Make scalable dynamic partitioning work in vectorized mode
> --
>
> Key: HIVE-6735
> URL: https://issues.apache.org/jira/browse/HIVE-6735
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor
>Affects Versions: 0.13.0, 0.14.0
>Reporter: Prasanth J
>Assignee: Prasanth J
> Fix For: 0.13.0, 0.14.0
>
> Attachments: HIVE-6735.1.patch, HIVE-6735.2.patch, HIVE-6735.2.patch
>
>
> HIVE-6455 added support for scalable dynamic partitioning. This is subtask to 
> make HIVE-6455 work with vectorized operators.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6761) Hashcode computation does not use maximum parallelism for scalable dynamic partitioning

[Prasanth J (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasanth J updated HIVE-6761:
-

Status: Patch Available  (was: Open)

> Hashcode computation does not use maximum parallelism for scalable dynamic 
> partitioning
> ---
>
> Key: HIVE-6761
> URL: https://issues.apache.org/jira/browse/HIVE-6761
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor
>Affects Versions: 0.13.0, 0.14.0
>Reporter: Prasanth J
>Assignee: Prasanth J
> Fix For: 0.13.0, 0.14.0
>
> Attachments: HIVE-6761.1.patch
>
>
> Hashcode computation for HIVE-6455 should consider all the partitioning 
> columns and bucket number to distribute the rows. The following code 
> {code}
> for (int i = 0; i < partitionEval.length - 1; i++) {
> {code}
> ignores the last partition column thereby generating lesser hashcodes.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6761) Hashcode computation does not use maximum parallelism for scalable dynamic partitioning

[Prasanth J (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948644#comment-13948644
 ] 

Prasanth J commented on HIVE-6761:
--

This is the reason of lesser hashcodes generation as mentioned by [~gopalv] 
here 
https://issues.apache.org/jira/browse/HIVE-6735?focusedCommentId=13948046&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13948046

> Hashcode computation does not use maximum parallelism for scalable dynamic 
> partitioning
> ---
>
> Key: HIVE-6761
> URL: https://issues.apache.org/jira/browse/HIVE-6761
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor
>Affects Versions: 0.13.0, 0.14.0
>Reporter: Prasanth J
>Assignee: Prasanth J
> Fix For: 0.13.0, 0.14.0
>
> Attachments: HIVE-6761.1.patch
>
>
> Hashcode computation for HIVE-6455 should consider all the partitioning 
> columns and bucket number to distribute the rows. The following code 
> {code}
> for (int i = 0; i < partitionEval.length - 1; i++) {
> {code}
> ignores the last partition column thereby generating lesser hashcodes.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6761) Hashcode computation does not use maximum parallelism for scalable dynamic partitioning

[Prasanth J (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasanth J updated HIVE-6761:
-

Attachment: HIVE-6761.1.patch

> Hashcode computation does not use maximum parallelism for scalable dynamic 
> partitioning
> ---
>
> Key: HIVE-6761
> URL: https://issues.apache.org/jira/browse/HIVE-6761
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor
>Affects Versions: 0.13.0, 0.14.0
>Reporter: Prasanth J
>Assignee: Prasanth J
> Fix For: 0.13.0, 0.14.0
>
> Attachments: HIVE-6761.1.patch
>
>
> Hashcode computation for HIVE-6455 should consider all the partitioning 
> columns and bucket number to distribute the rows. The following code 
> {code}
> for (int i = 0; i < partitionEval.length - 1; i++) {
> {code}
> ignores the last partition column thereby generating lesser hashcodes.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6710) Deadlocks seen in transaction handler using mysql

[Alan Gates (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Gates updated HIVE-6710:
-

Status: Patch Available  (was: Open)

> Deadlocks seen in transaction handler using mysql
> -
>
> Key: HIVE-6710
> URL: https://issues.apache.org/jira/browse/HIVE-6710
> Project: Hive
>  Issue Type: Bug
>  Components: Locking
>Affects Versions: 0.13.0
>Reporter: Alan Gates
>Assignee: Alan Gates
> Fix For: 0.13.0
>
> Attachments: HIVE-6710.patch
>
>
> When multiple clients attempt to obtain locks a deadlock on the mysql 
> database occasionally occurs.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6642) Query fails to vectorize when a non string partition column is part of the query expression

[Hari Sankar Sivarama Subramaniyan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hari Sankar Sivarama Subramaniyan updated HIVE-6642:


Status: Patch Available  (was: Open)

> Query fails to vectorize when a non string partition column is part of the 
> query expression
> ---
>
> Key: HIVE-6642
> URL: https://issues.apache.org/jira/browse/HIVE-6642
> Project: Hive
>  Issue Type: Bug
>Reporter: Hari Sankar Sivarama Subramaniyan
>Assignee: Hari Sankar Sivarama Subramaniyan
> Attachments: HIVE-6642-2.patch, HIVE-6642-3.patch, HIVE-6642-4.patch, 
> HIVE-6642.1.patch
>
>
> drop table if exists alltypesorc_part;
> CREATE TABLE alltypesorc_part (
> ctinyint tinyint,
> csmallint smallint,
> cint int,
> cbigint bigint,
> cfloat float,
> cdouble double,
> cstring1 string,
> cstring2 string,
> ctimestamp1 timestamp,
> ctimestamp2 timestamp,
> cboolean1 boolean,
> cboolean2 boolean) partitioned by (ds int) STORED AS ORC;
> insert overwrite table alltypesorc_part partition (ds=2011) select * from 
> alltypesorc limit 100;
> insert overwrite table alltypesorc_part partition (ds=2012) select * from 
> alltypesorc limit 200;
> explain select *
> from (select ds from alltypesorc_part) t1,
>  alltypesorc t2
> where t1.ds = t2.cint
> order by t2.ctimestamp1
> limit 100;
> The above query fails to vectorize because (select ds from alltypesorc_part) 
> t1 returns a string column and the join equality on t2 is performed on an int 
> column. The correct output when vectorization is turned on should be:
> STAGE DEPENDENCIES:
>   Stage-5 is a root stage
>   Stage-2 depends on stages: Stage-5
>   Stage-0 is a root stage
> STAGE PLANS:
>   Stage: Stage-5
> Map Reduce Local Work
>   Alias -> Map Local Tables:
> t1:alltypesorc_part
>   Fetch Operator
> limit: -1
>   Alias -> Map Local Operator Tree:
> t1:alltypesorc_part
>   TableScan
> alias: alltypesorc_part
> Statistics: Num rows: 300 Data size: 62328 Basic stats: COMPLETE 
> Column stats: COMPLETE
> Select Operator
>   expressions: ds (type: int)
>   outputColumnNames: _col0
>   Statistics: Num rows: 300 Data size: 1200 Basic stats: COMPLETE 
> Column stats: COMPLETE
>   HashTable Sink Operator
> condition expressions:
>   0 {_col0}
>   1 {ctinyint} {csmallint} {cint} {cbigint} {cfloat} 
> {cdouble} {cstring1} {cstring2} {ctimestamp1} {ctimestamp2} {cboolean1} 
> {cboolean2}
> keys:
>   0 _col0 (type: int)
>   1 cint (type: int)
>   Stage: Stage-2
> Map Reduce
>   Map Operator Tree:
>   TableScan
> alias: t2
> Statistics: Num rows: 3536 Data size: 1131711 Basic stats: 
> COMPLETE Column stats: NONE
> Map Join Operator
>   condition map:
>Inner Join 0 to 1
>   condition expressions:
> 0 {_col0}
> 1 {ctinyint} {csmallint} {cint} {cbigint} {cfloat} {cdouble} 
> {cstring1} {cstring2} {ctimestamp1} {ctimestamp2} {cboolean1} {cboolean2}
>   keys:
> 0 _col0 (type: int)
> 1 cint (type: int)
>   outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, 
> _col6, _col7, _col8, _col9, _col10, _col11, _col12
>   Statistics: Num rows: 3889 Data size: 1244882 Basic stats: 
> COMPLETE Column stats: NONE
>   Filter Operator
> predicate: (_col0 = _col3) (type: boolean)
> Statistics: Num rows: 1944 Data size: 622280 Basic stats: 
> COMPLETE Column stats: NONE
> Select Operator
>   expressions: _col0 (type: int), _col1 (type: tinyint), 
> _col2 (type: smallint), _col3 (type: int), _col4 (type: bigint), _col5 (type: 
> float), _col6 (type: double), _col7 (type: string), _col8 (type: string), 
> _col\
> 9 (type: timestamp), _col10 (type: timestamp), _col11 (type: boolean), _col12 
> (type: boolean)
>   outputColumnNames: _col0, _col1, _col2, _col3, _col4, 
> _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12
>   Statistics: Num rows: 1944 Data size: 622280 Basic stats: 
> COMPLETE Column stats: NONE
>   Reduce Output Operator
> key expressions: _col9 (type: timestamp)
> sort order: +
> Statistics: Num rows: 1944 Data size: 622280 Basic stats: 
> COMPLETE Column stats: NONE
> value expressions: _col0 (type: int), _col1 (type: 
> tinyint), _col2 (type: smallint), _col3 (type: int), _col4 (type: bi

[jira] [Updated] (HIVE-6642) Query fails to vectorize when a non string partition column is part of the query expression

[Hari Sankar Sivarama Subramaniyan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hari Sankar Sivarama Subramaniyan updated HIVE-6642:


Attachment: HIVE-6642-4.patch

added test cases as well.

> Query fails to vectorize when a non string partition column is part of the 
> query expression
> ---
>
> Key: HIVE-6642
> URL: https://issues.apache.org/jira/browse/HIVE-6642
> Project: Hive
>  Issue Type: Bug
>Reporter: Hari Sankar Sivarama Subramaniyan
>Assignee: Hari Sankar Sivarama Subramaniyan
> Attachments: HIVE-6642-2.patch, HIVE-6642-3.patch, HIVE-6642-4.patch, 
> HIVE-6642.1.patch
>
>
> drop table if exists alltypesorc_part;
> CREATE TABLE alltypesorc_part (
> ctinyint tinyint,
> csmallint smallint,
> cint int,
> cbigint bigint,
> cfloat float,
> cdouble double,
> cstring1 string,
> cstring2 string,
> ctimestamp1 timestamp,
> ctimestamp2 timestamp,
> cboolean1 boolean,
> cboolean2 boolean) partitioned by (ds int) STORED AS ORC;
> insert overwrite table alltypesorc_part partition (ds=2011) select * from 
> alltypesorc limit 100;
> insert overwrite table alltypesorc_part partition (ds=2012) select * from 
> alltypesorc limit 200;
> explain select *
> from (select ds from alltypesorc_part) t1,
>  alltypesorc t2
> where t1.ds = t2.cint
> order by t2.ctimestamp1
> limit 100;
> The above query fails to vectorize because (select ds from alltypesorc_part) 
> t1 returns a string column and the join equality on t2 is performed on an int 
> column. The correct output when vectorization is turned on should be:
> STAGE DEPENDENCIES:
>   Stage-5 is a root stage
>   Stage-2 depends on stages: Stage-5
>   Stage-0 is a root stage
> STAGE PLANS:
>   Stage: Stage-5
> Map Reduce Local Work
>   Alias -> Map Local Tables:
> t1:alltypesorc_part
>   Fetch Operator
> limit: -1
>   Alias -> Map Local Operator Tree:
> t1:alltypesorc_part
>   TableScan
> alias: alltypesorc_part
> Statistics: Num rows: 300 Data size: 62328 Basic stats: COMPLETE 
> Column stats: COMPLETE
> Select Operator
>   expressions: ds (type: int)
>   outputColumnNames: _col0
>   Statistics: Num rows: 300 Data size: 1200 Basic stats: COMPLETE 
> Column stats: COMPLETE
>   HashTable Sink Operator
> condition expressions:
>   0 {_col0}
>   1 {ctinyint} {csmallint} {cint} {cbigint} {cfloat} 
> {cdouble} {cstring1} {cstring2} {ctimestamp1} {ctimestamp2} {cboolean1} 
> {cboolean2}
> keys:
>   0 _col0 (type: int)
>   1 cint (type: int)
>   Stage: Stage-2
> Map Reduce
>   Map Operator Tree:
>   TableScan
> alias: t2
> Statistics: Num rows: 3536 Data size: 1131711 Basic stats: 
> COMPLETE Column stats: NONE
> Map Join Operator
>   condition map:
>Inner Join 0 to 1
>   condition expressions:
> 0 {_col0}
> 1 {ctinyint} {csmallint} {cint} {cbigint} {cfloat} {cdouble} 
> {cstring1} {cstring2} {ctimestamp1} {ctimestamp2} {cboolean1} {cboolean2}
>   keys:
> 0 _col0 (type: int)
> 1 cint (type: int)
>   outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, 
> _col6, _col7, _col8, _col9, _col10, _col11, _col12
>   Statistics: Num rows: 3889 Data size: 1244882 Basic stats: 
> COMPLETE Column stats: NONE
>   Filter Operator
> predicate: (_col0 = _col3) (type: boolean)
> Statistics: Num rows: 1944 Data size: 622280 Basic stats: 
> COMPLETE Column stats: NONE
> Select Operator
>   expressions: _col0 (type: int), _col1 (type: tinyint), 
> _col2 (type: smallint), _col3 (type: int), _col4 (type: bigint), _col5 (type: 
> float), _col6 (type: double), _col7 (type: string), _col8 (type: string), 
> _col\
> 9 (type: timestamp), _col10 (type: timestamp), _col11 (type: boolean), _col12 
> (type: boolean)
>   outputColumnNames: _col0, _col1, _col2, _col3, _col4, 
> _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12
>   Statistics: Num rows: 1944 Data size: 622280 Basic stats: 
> COMPLETE Column stats: NONE
>   Reduce Output Operator
> key expressions: _col9 (type: timestamp)
> sort order: +
> Statistics: Num rows: 1944 Data size: 622280 Basic stats: 
> COMPLETE Column stats: NONE
> value expressions: _col0 (type: int), _col1 (type: 
> tinyint), _col2 (type: smallint), _col3 (type: i

[jira] [Updated] (HIVE-6642) Query fails to vectorize when a non string partition column is part of the query expression

[Hari Sankar Sivarama Subramaniyan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hari Sankar Sivarama Subramaniyan updated HIVE-6642:


Status: Open  (was: Patch Available)

> Query fails to vectorize when a non string partition column is part of the 
> query expression
> ---
>
> Key: HIVE-6642
> URL: https://issues.apache.org/jira/browse/HIVE-6642
> Project: Hive
>  Issue Type: Bug
>Reporter: Hari Sankar Sivarama Subramaniyan
>Assignee: Hari Sankar Sivarama Subramaniyan
> Attachments: HIVE-6642-2.patch, HIVE-6642-3.patch, HIVE-6642-4.patch, 
> HIVE-6642.1.patch
>
>
> drop table if exists alltypesorc_part;
> CREATE TABLE alltypesorc_part (
> ctinyint tinyint,
> csmallint smallint,
> cint int,
> cbigint bigint,
> cfloat float,
> cdouble double,
> cstring1 string,
> cstring2 string,
> ctimestamp1 timestamp,
> ctimestamp2 timestamp,
> cboolean1 boolean,
> cboolean2 boolean) partitioned by (ds int) STORED AS ORC;
> insert overwrite table alltypesorc_part partition (ds=2011) select * from 
> alltypesorc limit 100;
> insert overwrite table alltypesorc_part partition (ds=2012) select * from 
> alltypesorc limit 200;
> explain select *
> from (select ds from alltypesorc_part) t1,
>  alltypesorc t2
> where t1.ds = t2.cint
> order by t2.ctimestamp1
> limit 100;
> The above query fails to vectorize because (select ds from alltypesorc_part) 
> t1 returns a string column and the join equality on t2 is performed on an int 
> column. The correct output when vectorization is turned on should be:
> STAGE DEPENDENCIES:
>   Stage-5 is a root stage
>   Stage-2 depends on stages: Stage-5
>   Stage-0 is a root stage
> STAGE PLANS:
>   Stage: Stage-5
> Map Reduce Local Work
>   Alias -> Map Local Tables:
> t1:alltypesorc_part
>   Fetch Operator
> limit: -1
>   Alias -> Map Local Operator Tree:
> t1:alltypesorc_part
>   TableScan
> alias: alltypesorc_part
> Statistics: Num rows: 300 Data size: 62328 Basic stats: COMPLETE 
> Column stats: COMPLETE
> Select Operator
>   expressions: ds (type: int)
>   outputColumnNames: _col0
>   Statistics: Num rows: 300 Data size: 1200 Basic stats: COMPLETE 
> Column stats: COMPLETE
>   HashTable Sink Operator
> condition expressions:
>   0 {_col0}
>   1 {ctinyint} {csmallint} {cint} {cbigint} {cfloat} 
> {cdouble} {cstring1} {cstring2} {ctimestamp1} {ctimestamp2} {cboolean1} 
> {cboolean2}
> keys:
>   0 _col0 (type: int)
>   1 cint (type: int)
>   Stage: Stage-2
> Map Reduce
>   Map Operator Tree:
>   TableScan
> alias: t2
> Statistics: Num rows: 3536 Data size: 1131711 Basic stats: 
> COMPLETE Column stats: NONE
> Map Join Operator
>   condition map:
>Inner Join 0 to 1
>   condition expressions:
> 0 {_col0}
> 1 {ctinyint} {csmallint} {cint} {cbigint} {cfloat} {cdouble} 
> {cstring1} {cstring2} {ctimestamp1} {ctimestamp2} {cboolean1} {cboolean2}
>   keys:
> 0 _col0 (type: int)
> 1 cint (type: int)
>   outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, 
> _col6, _col7, _col8, _col9, _col10, _col11, _col12
>   Statistics: Num rows: 3889 Data size: 1244882 Basic stats: 
> COMPLETE Column stats: NONE
>   Filter Operator
> predicate: (_col0 = _col3) (type: boolean)
> Statistics: Num rows: 1944 Data size: 622280 Basic stats: 
> COMPLETE Column stats: NONE
> Select Operator
>   expressions: _col0 (type: int), _col1 (type: tinyint), 
> _col2 (type: smallint), _col3 (type: int), _col4 (type: bigint), _col5 (type: 
> float), _col6 (type: double), _col7 (type: string), _col8 (type: string), 
> _col\
> 9 (type: timestamp), _col10 (type: timestamp), _col11 (type: boolean), _col12 
> (type: boolean)
>   outputColumnNames: _col0, _col1, _col2, _col3, _col4, 
> _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12
>   Statistics: Num rows: 1944 Data size: 622280 Basic stats: 
> COMPLETE Column stats: NONE
>   Reduce Output Operator
> key expressions: _col9 (type: timestamp)
> sort order: +
> Statistics: Num rows: 1944 Data size: 622280 Basic stats: 
> COMPLETE Column stats: NONE
> value expressions: _col0 (type: int), _col1 (type: 
> tinyint), _col2 (type: smallint), _col3 (type: int), _col4 (type: bi

[jira] [Updated] (HIVE-6710) Deadlocks seen in transaction handler using mysql

[Alan Gates (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Gates updated HIVE-6710:
-

Attachment: HIVE-6710.patch

> Deadlocks seen in transaction handler using mysql
> -
>
> Key: HIVE-6710
> URL: https://issues.apache.org/jira/browse/HIVE-6710
> Project: Hive
>  Issue Type: Bug
>  Components: Locking
>Affects Versions: 0.13.0
>Reporter: Alan Gates
>Assignee: Alan Gates
> Fix For: 0.13.0
>
> Attachments: HIVE-6710.patch
>
>
> When multiple clients attempt to obtain locks a deadlock on the mysql 
> database occasionally occurs.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6760) Scalable dynamic partitioning should bail out properly for list bucketing

[Prasanth J (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasanth J updated HIVE-6760:
-

Attachment: HIVE-6760.2.patch

Looks like list bucketing context is always available. This patch checks 
further deeper to see if there are any skewed columns and values before bailing 
out.

> Scalable dynamic partitioning should bail out properly for list bucketing
> -
>
> Key: HIVE-6760
> URL: https://issues.apache.org/jira/browse/HIVE-6760
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor
>Affects Versions: 0.13.0, 0.14.0
>Reporter: Prasanth J
>Assignee: Prasanth J
> Fix For: 0.13.0, 0.14.0
>
> Attachments: HIVE-6760.1.patch, HIVE-6760.2.patch
>
>
> In case of list bucketing HIVE-6455 looks only at this config 
> "hive.optimize.listbucketing" to bail out. There are cases when this config 
> ("hive.optimize.listbucketing") is not set but list bucketing is enabled 
> using SKEWED BY in CREATE TABLE statement.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6670) ClassNotFound with Serde

[Abin Shahab (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948620#comment-13948620
 ] 

Abin Shahab commented on HIVE-6670:
---

Thanks for rolling it forward!



> ClassNotFound with Serde
> 
>
> Key: HIVE-6670
> URL: https://issues.apache.org/jira/browse/HIVE-6670
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.12.0
>Reporter: Abin Shahab
>Assignee: Abin Shahab
> Attachments: HIVE-6670-branch-0.12.patch, HIVE-6670.1.patch, 
> HIVE-6670.patch
>
>
> We are finding a ClassNotFound exception when we use 
> CSVSerde(https://github.com/ogrodnek/csv-serde) to create a table.
> This is happening because MapredLocalTask does not pass the local added jars 
> to ExecDriver when that is launched.
> ExecDriver's classpath does not include the added jars. Therefore, when the 
> plan is deserialized, it throws a ClassNotFoundException in the 
> deserialization code, and results in a TableDesc object with a Null 
> DeserializerClass.
> This results in an NPE during Fetch.
> Steps to reproduce:
> wget 
> https://drone.io/github.com/ogrodnek/csv-serde/files/target/csv-serde-1.1.2-0.11.0-all.jar
>  into somewhere local eg. 
> /home/soam/HiveSerdeIssue/csv-serde-1.1.2-0.11.0-all.jar.
> Place some sample SCV files in HDFS as follows:
> hdfs dfs -mkdir /user/soam/HiveSerdeIssue/sampleCSV/
> hdfs dfs -put /home/soam/sampleCSV.csv /user/soam/HiveSerdeIssue/sampleCSV/
> hdfs dfs -mkdir /user/soam/HiveSerdeIssue/sampleJoinTarget/
> hdfs dfs -put /home/soam/sampleJoinTarget.csv 
> /user/soam/HiveSerdeIssue/sampleJoinTarget/
> 
> create the tables in hive:
> ADD JAR /home/soam/HiveSerdeIssue/csv-serde-1.1.2-0.11.0-all.jar;
> create external table sampleCSV (md5hash string, filepath string)
> row format serde 'com.bizo.hive.serde.csv.CSVSerde'
> stored as textfile
> location '/user/soam/HiveSerdeIssue/sampleCSV/'
> ;
> create external table sampleJoinTarget (md5hash string, filepath string, 
> datestamp string, nblines string, nberrors string)
> ROW FORMAT DELIMITED 
> FIELDS TERMINATED BY ',' 
> LINES TERMINATED BY '\n'
> STORED AS TEXTFILE
> LOCATION '/user/soam/HiveSerdeIssue/sampleJoinTarget/'
> ;
> ===
> Now, try the following JOIN:
> ADD JAR /home/soam/HiveSerdeIssue/csv-serde-1.1.2-0.11.0-all.jar;
> SELECT 
> sampleCSV.md5hash, 
> sampleCSV.filepath 
> FROM sampleCSV
> JOIN sampleJoinTarget
> ON (sampleCSV.md5hash = sampleJoinTarget.md5hash) 
> ;
> —
> This will fail with the error:
> Execution log at: /tmp/soam/.log
> java.lang.ClassNotFoundException: com/bizo/hive/serde/csv/CSVSerde
> Continuing ...
> 2014-03-11 10:35:03 Starting to launch local task to process map join; 
> maximum memory = 238551040
> Execution failed with exit status: 2
> Obtaining error information
> Task failed!
> Task ID:
> Stage-4
> Logs:
> /var/log/hive/soam/hive.log
> FAILED: Execution Error, return code 2 from 
> org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> Try the following LEFT JOIN. This will work:
> SELECT 
> sampleCSV.md5hash, 
> sampleCSV.filepath 
> FROM sampleCSV
> LEFT JOIN sampleJoinTarget
> ON (sampleCSV.md5hash = sampleJoinTarget.md5hash) 
> ;
> ==



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Review Request 19705: Fix add jar problem with map join.

[Ashutosh Chauhan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19705/
---

Review request for hive.


Bugs: HIVE-6670
https://issues.apache.org/jira/browse/HIVE-6670


Repository: hive-git


Description
---

Fix add jar problem with map join.


Diffs
-

  data/files/sample.json PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java 63f41ae 
  ql/src/test/queries/clientpositive/mapjoin_addjar.q PRE-CREATION 
  ql/src/test/results/clientpositive/mapjoin_addjar.q.out PRE-CREATION 

Diff: https://reviews.apache.org/r/19705/diff/


Testing
---

Added new test.


Thanks,

Ashutosh Chauhan

[jira] [Commented] (HIVE-6758) Beeline only works in interactive mode

[Xuefu Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948616#comment-13948616
 ] 

Xuefu Zhang commented on HIVE-6758:
---

I put the query into a file and use -f option to execute, it also worked.

> Beeline only works in interactive mode
> --
>
> Key: HIVE-6758
> URL: https://issues.apache.org/jira/browse/HIVE-6758
> Project: Hive
>  Issue Type: Improvement
>  Components: CLI
>Affects Versions: 0.11.0, 0.12.0
> Environment: CDH4.5
>Reporter: Johndee Burks
>
> In hive CLI you could easily integrate its use into a script and back ground 
> the process like this: 
> hive -e "some query" &
> Beeline does not run when you do the same even with the -f switch. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6670) ClassNotFound with Serde

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-6670:
---

Attachment: HIVE-6670.1.patch

Patch with my suggested fix and test case.

> ClassNotFound with Serde
> 
>
> Key: HIVE-6670
> URL: https://issues.apache.org/jira/browse/HIVE-6670
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.12.0
>Reporter: Abin Shahab
>Assignee: Abin Shahab
> Attachments: HIVE-6670-branch-0.12.patch, HIVE-6670.1.patch, 
> HIVE-6670.patch
>
>
> We are finding a ClassNotFound exception when we use 
> CSVSerde(https://github.com/ogrodnek/csv-serde) to create a table.
> This is happening because MapredLocalTask does not pass the local added jars 
> to ExecDriver when that is launched.
> ExecDriver's classpath does not include the added jars. Therefore, when the 
> plan is deserialized, it throws a ClassNotFoundException in the 
> deserialization code, and results in a TableDesc object with a Null 
> DeserializerClass.
> This results in an NPE during Fetch.
> Steps to reproduce:
> wget 
> https://drone.io/github.com/ogrodnek/csv-serde/files/target/csv-serde-1.1.2-0.11.0-all.jar
>  into somewhere local eg. 
> /home/soam/HiveSerdeIssue/csv-serde-1.1.2-0.11.0-all.jar.
> Place some sample SCV files in HDFS as follows:
> hdfs dfs -mkdir /user/soam/HiveSerdeIssue/sampleCSV/
> hdfs dfs -put /home/soam/sampleCSV.csv /user/soam/HiveSerdeIssue/sampleCSV/
> hdfs dfs -mkdir /user/soam/HiveSerdeIssue/sampleJoinTarget/
> hdfs dfs -put /home/soam/sampleJoinTarget.csv 
> /user/soam/HiveSerdeIssue/sampleJoinTarget/
> 
> create the tables in hive:
> ADD JAR /home/soam/HiveSerdeIssue/csv-serde-1.1.2-0.11.0-all.jar;
> create external table sampleCSV (md5hash string, filepath string)
> row format serde 'com.bizo.hive.serde.csv.CSVSerde'
> stored as textfile
> location '/user/soam/HiveSerdeIssue/sampleCSV/'
> ;
> create external table sampleJoinTarget (md5hash string, filepath string, 
> datestamp string, nblines string, nberrors string)
> ROW FORMAT DELIMITED 
> FIELDS TERMINATED BY ',' 
> LINES TERMINATED BY '\n'
> STORED AS TEXTFILE
> LOCATION '/user/soam/HiveSerdeIssue/sampleJoinTarget/'
> ;
> ===
> Now, try the following JOIN:
> ADD JAR /home/soam/HiveSerdeIssue/csv-serde-1.1.2-0.11.0-all.jar;
> SELECT 
> sampleCSV.md5hash, 
> sampleCSV.filepath 
> FROM sampleCSV
> JOIN sampleJoinTarget
> ON (sampleCSV.md5hash = sampleJoinTarget.md5hash) 
> ;
> —
> This will fail with the error:
> Execution log at: /tmp/soam/.log
> java.lang.ClassNotFoundException: com/bizo/hive/serde/csv/CSVSerde
> Continuing ...
> 2014-03-11 10:35:03 Starting to launch local task to process map join; 
> maximum memory = 238551040
> Execution failed with exit status: 2
> Obtaining error information
> Task failed!
> Task ID:
> Stage-4
> Logs:
> /var/log/hive/soam/hive.log
> FAILED: Execution Error, return code 2 from 
> org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> Try the following LEFT JOIN. This will work:
> SELECT 
> sampleCSV.md5hash, 
> sampleCSV.filepath 
> FROM sampleCSV
> LEFT JOIN sampleJoinTarget
> ON (sampleCSV.md5hash = sampleJoinTarget.md5hash) 
> ;
> ==



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6492) limit partition number involved in a table scan

[Alan Gates (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948609#comment-13948609
 ] 

Alan Gates commented on HIVE-6492:
--

Ran the tests locally, all looks good.

> limit partition number involved in a table scan
> ---
>
> Key: HIVE-6492
> URL: https://issues.apache.org/jira/browse/HIVE-6492
> Project: Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Affects Versions: 0.12.0
>Reporter: Selina Zhang
>Assignee: Ashutosh Chauhan
> Fix For: 0.13.0
>
> Attachments: HIVE-6492.1.patch.txt, HIVE-6492.2.patch.txt, 
> HIVE-6492.3.patch.txt, HIVE-6492.4.patch.txt, HIVE-6492.4.patch_suggestion, 
> HIVE-6492.5.patch.txt, HIVE-6492.6.patch.txt, HIVE-6492.7.parch.txt
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> To protect the cluster, a new configure variable 
> "hive.limit.query.max.table.partition" is added to hive configuration to
> limit the table partitions involved in a table scan. 
> The default value will be set to -1 which means there is no limit by default. 
> This variable will not affect "metadata only" query.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6758) Beeline only works in interactive mode

[Xuefu Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948608#comment-13948608
 ] 

Xuefu Zhang commented on HIVE-6758:
---

I'm not sure what query you're executing, but I tried the following, which 
worked:

{code}
xzhang@xzlt:~/apa/hive6$ 
./packaging/target/apache-hive-0.14.0-SNAPSHOT-bin/apache-hive-0.14.0-SNAPSHOT-bin/bin/beeline
 -u jdbc:hive2:// -e "show databases"
...
14/03/26 16:02:38 INFO mapred.FileInputFormat: Total input paths to process : 1
++
| database_name  |
++
| default|
++
1 row selected (0.437 seconds)
14/03/26 16:02:38 INFO exec.ListSinkOperator: 0 finished. closing... 
...
{code}

> Beeline only works in interactive mode
> --
>
> Key: HIVE-6758
> URL: https://issues.apache.org/jira/browse/HIVE-6758
> Project: Hive
>  Issue Type: Improvement
>  Components: CLI
>Affects Versions: 0.11.0, 0.12.0
> Environment: CDH4.5
>Reporter: Johndee Burks
>
> In hive CLI you could easily integrate its use into a script and back ground 
> the process like this: 
> hive -e "some query" &
> Beeline does not run when you do the same even with the -f switch. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6670) ClassNotFound with Serde

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-6670:
---

Status: Patch Available  (was: Open)

> ClassNotFound with Serde
> 
>
> Key: HIVE-6670
> URL: https://issues.apache.org/jira/browse/HIVE-6670
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.12.0
>Reporter: Abin Shahab
>Assignee: Abin Shahab
> Attachments: HIVE-6670-branch-0.12.patch, HIVE-6670.1.patch, 
> HIVE-6670.patch
>
>
> We are finding a ClassNotFound exception when we use 
> CSVSerde(https://github.com/ogrodnek/csv-serde) to create a table.
> This is happening because MapredLocalTask does not pass the local added jars 
> to ExecDriver when that is launched.
> ExecDriver's classpath does not include the added jars. Therefore, when the 
> plan is deserialized, it throws a ClassNotFoundException in the 
> deserialization code, and results in a TableDesc object with a Null 
> DeserializerClass.
> This results in an NPE during Fetch.
> Steps to reproduce:
> wget 
> https://drone.io/github.com/ogrodnek/csv-serde/files/target/csv-serde-1.1.2-0.11.0-all.jar
>  into somewhere local eg. 
> /home/soam/HiveSerdeIssue/csv-serde-1.1.2-0.11.0-all.jar.
> Place some sample SCV files in HDFS as follows:
> hdfs dfs -mkdir /user/soam/HiveSerdeIssue/sampleCSV/
> hdfs dfs -put /home/soam/sampleCSV.csv /user/soam/HiveSerdeIssue/sampleCSV/
> hdfs dfs -mkdir /user/soam/HiveSerdeIssue/sampleJoinTarget/
> hdfs dfs -put /home/soam/sampleJoinTarget.csv 
> /user/soam/HiveSerdeIssue/sampleJoinTarget/
> 
> create the tables in hive:
> ADD JAR /home/soam/HiveSerdeIssue/csv-serde-1.1.2-0.11.0-all.jar;
> create external table sampleCSV (md5hash string, filepath string)
> row format serde 'com.bizo.hive.serde.csv.CSVSerde'
> stored as textfile
> location '/user/soam/HiveSerdeIssue/sampleCSV/'
> ;
> create external table sampleJoinTarget (md5hash string, filepath string, 
> datestamp string, nblines string, nberrors string)
> ROW FORMAT DELIMITED 
> FIELDS TERMINATED BY ',' 
> LINES TERMINATED BY '\n'
> STORED AS TEXTFILE
> LOCATION '/user/soam/HiveSerdeIssue/sampleJoinTarget/'
> ;
> ===
> Now, try the following JOIN:
> ADD JAR /home/soam/HiveSerdeIssue/csv-serde-1.1.2-0.11.0-all.jar;
> SELECT 
> sampleCSV.md5hash, 
> sampleCSV.filepath 
> FROM sampleCSV
> JOIN sampleJoinTarget
> ON (sampleCSV.md5hash = sampleJoinTarget.md5hash) 
> ;
> —
> This will fail with the error:
> Execution log at: /tmp/soam/.log
> java.lang.ClassNotFoundException: com/bizo/hive/serde/csv/CSVSerde
> Continuing ...
> 2014-03-11 10:35:03 Starting to launch local task to process map join; 
> maximum memory = 238551040
> Execution failed with exit status: 2
> Obtaining error information
> Task failed!
> Task ID:
> Stage-4
> Logs:
> /var/log/hive/soam/hive.log
> FAILED: Execution Error, return code 2 from 
> org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> Try the following LEFT JOIN. This will work:
> SELECT 
> sampleCSV.md5hash, 
> sampleCSV.filepath 
> FROM sampleCSV
> LEFT JOIN sampleJoinTarget
> ON (sampleCSV.md5hash = sampleJoinTarget.md5hash) 
> ;
> ==



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6670) ClassNotFound with Serde

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-6670:
---

Status: Open  (was: Patch Available)

> ClassNotFound with Serde
> 
>
> Key: HIVE-6670
> URL: https://issues.apache.org/jira/browse/HIVE-6670
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.12.0
>Reporter: Abin Shahab
>Assignee: Abin Shahab
> Attachments: HIVE-6670-branch-0.12.patch, HIVE-6670.1.patch, 
> HIVE-6670.patch
>
>
> We are finding a ClassNotFound exception when we use 
> CSVSerde(https://github.com/ogrodnek/csv-serde) to create a table.
> This is happening because MapredLocalTask does not pass the local added jars 
> to ExecDriver when that is launched.
> ExecDriver's classpath does not include the added jars. Therefore, when the 
> plan is deserialized, it throws a ClassNotFoundException in the 
> deserialization code, and results in a TableDesc object with a Null 
> DeserializerClass.
> This results in an NPE during Fetch.
> Steps to reproduce:
> wget 
> https://drone.io/github.com/ogrodnek/csv-serde/files/target/csv-serde-1.1.2-0.11.0-all.jar
>  into somewhere local eg. 
> /home/soam/HiveSerdeIssue/csv-serde-1.1.2-0.11.0-all.jar.
> Place some sample SCV files in HDFS as follows:
> hdfs dfs -mkdir /user/soam/HiveSerdeIssue/sampleCSV/
> hdfs dfs -put /home/soam/sampleCSV.csv /user/soam/HiveSerdeIssue/sampleCSV/
> hdfs dfs -mkdir /user/soam/HiveSerdeIssue/sampleJoinTarget/
> hdfs dfs -put /home/soam/sampleJoinTarget.csv 
> /user/soam/HiveSerdeIssue/sampleJoinTarget/
> 
> create the tables in hive:
> ADD JAR /home/soam/HiveSerdeIssue/csv-serde-1.1.2-0.11.0-all.jar;
> create external table sampleCSV (md5hash string, filepath string)
> row format serde 'com.bizo.hive.serde.csv.CSVSerde'
> stored as textfile
> location '/user/soam/HiveSerdeIssue/sampleCSV/'
> ;
> create external table sampleJoinTarget (md5hash string, filepath string, 
> datestamp string, nblines string, nberrors string)
> ROW FORMAT DELIMITED 
> FIELDS TERMINATED BY ',' 
> LINES TERMINATED BY '\n'
> STORED AS TEXTFILE
> LOCATION '/user/soam/HiveSerdeIssue/sampleJoinTarget/'
> ;
> ===
> Now, try the following JOIN:
> ADD JAR /home/soam/HiveSerdeIssue/csv-serde-1.1.2-0.11.0-all.jar;
> SELECT 
> sampleCSV.md5hash, 
> sampleCSV.filepath 
> FROM sampleCSV
> JOIN sampleJoinTarget
> ON (sampleCSV.md5hash = sampleJoinTarget.md5hash) 
> ;
> —
> This will fail with the error:
> Execution log at: /tmp/soam/.log
> java.lang.ClassNotFoundException: com/bizo/hive/serde/csv/CSVSerde
> Continuing ...
> 2014-03-11 10:35:03 Starting to launch local task to process map join; 
> maximum memory = 238551040
> Execution failed with exit status: 2
> Obtaining error information
> Task failed!
> Task ID:
> Stage-4
> Logs:
> /var/log/hive/soam/hive.log
> FAILED: Execution Error, return code 2 from 
> org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> Try the following LEFT JOIN. This will work:
> SELECT 
> sampleCSV.md5hash, 
> sampleCSV.filepath 
> FROM sampleCSV
> LEFT JOIN sampleJoinTarget
> ON (sampleCSV.md5hash = sampleJoinTarget.md5hash) 
> ;
> ==



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6708) ConstantVectorExpression should create copies of data objects rather than referencing them

[Jitendra Nath Pandey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jitendra Nath Pandey updated HIVE-6708:
---

Resolution: Fixed
Status: Resolved  (was: Patch Available)

I have committed this to trunk and branch-0.13. Thanks to [~hari.s]!

> ConstantVectorExpression should create copies of data objects rather than 
> referencing them
> --
>
> Key: HIVE-6708
> URL: https://issues.apache.org/jira/browse/HIVE-6708
> Project: Hive
>  Issue Type: Bug
>Reporter: Hari Sankar Sivarama Subramaniyan
>Assignee: Hari Sankar Sivarama Subramaniyan
> Fix For: 0.13.0
>
> Attachments: HIVE-6708-1.patch, HIVE-6708-3.patch, HIVE-6708-4.patch, 
> HIVE-6708.2.patch
>
>
> 1. ConstantVectorExpression vector should be updated for bytecolumnvectors 
> and decimalColumnVectors. The current code changes the reference to the 
> vector which might be shared across multiple columns
> 2. VectorizationContext.foldConstantsForUnaryExpression(ExprNodeDesc 
> exprDesc) has a minor bug as to when to constant fold the expression.
> The following code should replace the corresponding piece of code in the 
> trunk.
> ..
> GenericUDF gudf = ((ExprNodeGenericFuncDesc) exprDesc).getGenericUDF();
> if (gudf instanceof GenericUDFOPNegative || gudf instanceof 
> GenericUDFOPPositive
> || castExpressionUdfs.contains(gudf.getClass())
> ... 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6760) Scalable dynamic partitioning should bail out properly for list bucketing

[Prasanth J (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasanth J updated HIVE-6760:
-

Attachment: HIVE-6760.1.patch

This patch looks for list bucketing context and then bails out.

> Scalable dynamic partitioning should bail out properly for list bucketing
> -
>
> Key: HIVE-6760
> URL: https://issues.apache.org/jira/browse/HIVE-6760
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor
>Affects Versions: 0.13.0, 0.14.0
>Reporter: Prasanth J
>Assignee: Prasanth J
> Fix For: 0.13.0, 0.14.0
>
> Attachments: HIVE-6760.1.patch
>
>
> In case of list bucketing HIVE-6455 looks only at this config 
> "hive.optimize.listbucketing" to bail out. There are cases when this config 
> ("hive.optimize.listbucketing") is not set but list bucketing is enabled 
> using SKEWED BY in CREATE TABLE statement.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6760) Scalable dynamic partitioning should bail out properly for list bucketing

[Prasanth J (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasanth J updated HIVE-6760:
-

Status: Patch Available  (was: Open)

> Scalable dynamic partitioning should bail out properly for list bucketing
> -
>
> Key: HIVE-6760
> URL: https://issues.apache.org/jira/browse/HIVE-6760
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor
>Affects Versions: 0.13.0, 0.14.0
>Reporter: Prasanth J
>Assignee: Prasanth J
> Fix For: 0.13.0, 0.14.0
>
> Attachments: HIVE-6760.1.patch
>
>
> In case of list bucketing HIVE-6455 looks only at this config 
> "hive.optimize.listbucketing" to bail out. There are cases when this config 
> ("hive.optimize.listbucketing") is not set but list bucketing is enabled 
> using SKEWED BY in CREATE TABLE statement.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HIVE-6761) Hashcode computation does not use maximum parallelism for scalable dynamic partitioning

[Prasanth J (JIRA)]

Prasanth J created HIVE-6761:


 Summary: Hashcode computation does not use maximum parallelism for 
scalable dynamic partitioning
 Key: HIVE-6761
 URL: https://issues.apache.org/jira/browse/HIVE-6761
 Project: Hive
  Issue Type: Sub-task
Affects Versions: 0.13.0, 0.14.0
Reporter: Prasanth J
Assignee: Prasanth J


Hashcode computation for HIVE-6455 should consider all the partitioning columns 
and bucket number to distribute the rows. The following code 
{code}
for (int i = 0; i < partitionEval.length - 1; i++) {
{code}

ignores the last partition column thereby generating lesser hashcodes.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HIVE-6760) Scalable dynamic partitioning should bail out properly for list bucketing

[Prasanth J (JIRA)]

Prasanth J created HIVE-6760:


 Summary: Scalable dynamic partitioning should bail out properly 
for list bucketing
 Key: HIVE-6760
 URL: https://issues.apache.org/jira/browse/HIVE-6760
 Project: Hive
  Issue Type: Sub-task
Affects Versions: 0.13.0, 0.14.0
Reporter: Prasanth J
Assignee: Prasanth J


In case of list bucketing HIVE-6455 looks only at this config 
"hive.optimize.listbucketing" to bail out. There are cases when this config 
("hive.optimize.listbucketing") is not set but list bucketing is enabled using 
SKEWED BY in CREATE TABLE statement.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HIVE-6759) Fix reading partial ORC files while they are being written

[Owen O'Malley (JIRA)]

Owen O'Malley created HIVE-6759:
---

 Summary: Fix reading partial ORC files while they are being written
 Key: HIVE-6759
 URL: https://issues.apache.org/jira/browse/HIVE-6759
 Project: Hive
  Issue Type: Sub-task
Reporter: Owen O'Malley


HDFS with the hflush ensures the bytes are visible, but doesn't update the file 
length on the NameNode. Currently the Orc reader will only read up to the 
length on the NameNode. If the user specified a length from a flush_length 
file, the Orc reader should trust it to be right.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6749) Turn hive.auto.convert.join.use.nonstaged off by default

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948569#comment-13948569
 ] 

Hive QA commented on HIVE-6749:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12636822/HIVE-6749.patch

{color:red}ERROR:{color} -1 due to 109 failed/errored test(s), 5488 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join15
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join17
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join19
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join21
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join22
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join24
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join25
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join26
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join28
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join29
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join30
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join31
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join32
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join9
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_11
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_14
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_15
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_7
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_9
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket_map_join_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket_map_join_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_7
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin10
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin11
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin9
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin_negative
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin_negative2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin_negative3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer4
org.apache.hadoop.hive.cli.TestCl

[jira] [Commented] (HIVE-6757) Remove deprecated parquet classes from outside of org.apache package

[Owen O'Malley (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948564#comment-13948564
 ] 

Owen O'Malley commented on HIVE-6757:
-

There is no pressing need for them to be in Hive's jar. They are brand new 
classes and can easily be put in to the parquet jars that are distributed by 
parquet.

> Remove deprecated parquet classes from outside of org.apache package
> 
>
> Key: HIVE-6757
> URL: https://issues.apache.org/jira/browse/HIVE-6757
> Project: Hive
>  Issue Type: Bug
>Reporter: Owen O'Malley
>Assignee: Owen O'Malley
> Fix For: 0.13.0
>
>
> Apache shouldn't release projects with files outside of the org.apache 
> namespace.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6733) Driver context logs every query in the "warn" level

[Gunther Hagleitner (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gunther Hagleitner updated HIVE-6733:
-

   Resolution: Fixed
Fix Version/s: 0.13.0
   Status: Resolved  (was: Patch Available)

Committed to trunk and branch. Thanks Thejas!

> Driver context logs every query in the "warn" level
> ---
>
> Key: HIVE-6733
> URL: https://issues.apache.org/jira/browse/HIVE-6733
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.13.0
>Reporter: Gunther Hagleitner
>Assignee: Gunther Hagleitner
> Fix For: 0.13.0
>
> Attachments: HIVE-6733.1.patch
>
>
> Trivial, just lower the log level on one statement.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6733) Driver context logs every query in the "warn" level

[Gunther Hagleitner (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948549#comment-13948549
 ] 

Gunther Hagleitner commented on HIVE-6733:
--

ran tests locally. passed.

> Driver context logs every query in the "warn" level
> ---
>
> Key: HIVE-6733
> URL: https://issues.apache.org/jira/browse/HIVE-6733
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.13.0
>Reporter: Gunther Hagleitner
>Assignee: Gunther Hagleitner
> Attachments: HIVE-6733.1.patch
>
>
> Trivial, just lower the log level on one statement.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6733) Driver context logs every query in the "warn" level

[Gunther Hagleitner (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gunther Hagleitner updated HIVE-6733:
-

Status: Patch Available  (was: Open)

> Driver context logs every query in the "warn" level
> ---
>
> Key: HIVE-6733
> URL: https://issues.apache.org/jira/browse/HIVE-6733
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.13.0
>Reporter: Gunther Hagleitner
>Assignee: Gunther Hagleitner
> Attachments: HIVE-6733.1.patch
>
>
> Trivial, just lower the log level on one statement.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6546) WebHCat job submission for pig with -useHCatalog argument fails on Windows

[Eric Hanson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Hanson updated HIVE-6546:
--

Attachment: HIVE-6546.03.patch

Uploading patch yet again to try to kick off pre-commit tests.

> WebHCat job submission for pig with -useHCatalog argument fails on Windows
> --
>
> Key: HIVE-6546
> URL: https://issues.apache.org/jira/browse/HIVE-6546
> Project: Hive
>  Issue Type: Bug
>  Components: WebHCat
>Affects Versions: 0.11.0, 0.12.0, 0.13.0
> Environment: HDInsight deploying HDP 1.3:  
> c:\apps\dist\pig-0.11.0.1.3.2.0-05
> Also on Windows HDP 1.3 one-box configuration.
>Reporter: Eric Hanson
>Assignee: Eric Hanson
> Fix For: 0.13.0
>
> Attachments: HIVE-6546.01.patch, HIVE-6546.02.patch, 
> HIVE-6546.03.patch, HIVE-6546.03.patch, HIVE-6546.03.patch
>
>
> On a one-box windows setup, do the following from a powershell prompt:
> cmd /c curl.exe -s `
>   -d user.name=hadoop `
>   -d arg=-useHCatalog `
>   -d execute="emp = load '/data/emp/emp_0.dat'; dump emp;" `
>   -d statusdir="/tmp/webhcat.output01" `
>   'http://localhost:50111/templeton/v1/pig' -v
> The job fails with error code 7, but it should run. 
> I traced this down to the following. In the job configuration for the 
> TempletonJobController, we have templeton.args set to
> cmd,/c,call,C:\\hadooppig-0.11.0.1.3.0.0-0846/bin/pig.cmd,-D__WEBHCAT_TOKEN_FILE_LOCATION__="-useHCatalog",-execute,"emp
>  = load '/data/emp/emp_0.dat'; dump emp;"
> Notice the = sign before "-useHCatalog". I think this should be a comma.
> The bad string D__WEBHCAT_TOKEN_FILE_LOCATION__="-useHCatalog" gets created 
> in  org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows().
> It happens at line 434:
> {code}
>   } else {
>   if (i < args.length - 1) {
> prop += "=" + args[++i];   // RIGHT HERE! at iterations i = 37, 38
>   }
> }
> {code}
> Bug is here:
> {code}
>   if (prop != null) {
> if (prop.contains("=")) {  // -D__WEBHCAT_TOKEN_FILE_LOCATION__ does 
> not contain equal, so else branch is run and appends ="-useHCatalog",
>   // everything good
> } else {
>   if (i < args.length - 1) {
> prop += "=" + args[++i];
>   }
> }
> newArgs.add(prop);
>   }
> {code}
> One possible fix is to change the string constant 
> org.apache.hcatalog.templeton.tool.TempletonControllerJob.TOKEN_FILE_ARG_PLACEHOLDER
>  to have an "=" sign in it. Or, preProcessForWindows() itself could be 
> changed.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6546) WebHCat job submission for pig with -useHCatalog argument fails on Windows

[Thejas M Nair (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948542#comment-13948542
 ] 

Thejas M Nair commented on HIVE-6546:
-

Thanks for the clarification Eric. I thought that the problem was with parsing 
the command launched from the map task, but looks like this is happening before 
that. I was worried that having "=" sign could cause problems in other systems, 
but since the shell never sees this arguement, it is fine.

+1 .


> WebHCat job submission for pig with -useHCatalog argument fails on Windows
> --
>
> Key: HIVE-6546
> URL: https://issues.apache.org/jira/browse/HIVE-6546
> Project: Hive
>  Issue Type: Bug
>  Components: WebHCat
>Affects Versions: 0.11.0, 0.12.0, 0.13.0
> Environment: HDInsight deploying HDP 1.3:  
> c:\apps\dist\pig-0.11.0.1.3.2.0-05
> Also on Windows HDP 1.3 one-box configuration.
>Reporter: Eric Hanson
>Assignee: Eric Hanson
> Fix For: 0.13.0
>
> Attachments: HIVE-6546.01.patch, HIVE-6546.02.patch, 
> HIVE-6546.03.patch, HIVE-6546.03.patch
>
>
> On a one-box windows setup, do the following from a powershell prompt:
> cmd /c curl.exe -s `
>   -d user.name=hadoop `
>   -d arg=-useHCatalog `
>   -d execute="emp = load '/data/emp/emp_0.dat'; dump emp;" `
>   -d statusdir="/tmp/webhcat.output01" `
>   'http://localhost:50111/templeton/v1/pig' -v
> The job fails with error code 7, but it should run. 
> I traced this down to the following. In the job configuration for the 
> TempletonJobController, we have templeton.args set to
> cmd,/c,call,C:\\hadooppig-0.11.0.1.3.0.0-0846/bin/pig.cmd,-D__WEBHCAT_TOKEN_FILE_LOCATION__="-useHCatalog",-execute,"emp
>  = load '/data/emp/emp_0.dat'; dump emp;"
> Notice the = sign before "-useHCatalog". I think this should be a comma.
> The bad string D__WEBHCAT_TOKEN_FILE_LOCATION__="-useHCatalog" gets created 
> in  org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows().
> It happens at line 434:
> {code}
>   } else {
>   if (i < args.length - 1) {
> prop += "=" + args[++i];   // RIGHT HERE! at iterations i = 37, 38
>   }
> }
> {code}
> Bug is here:
> {code}
>   if (prop != null) {
> if (prop.contains("=")) {  // -D__WEBHCAT_TOKEN_FILE_LOCATION__ does 
> not contain equal, so else branch is run and appends ="-useHCatalog",
>   // everything good
> } else {
>   if (i < args.length - 1) {
> prop += "=" + args[++i];
>   }
> }
> newArgs.add(prop);
>   }
> {code}
> One possible fix is to change the string constant 
> org.apache.hcatalog.templeton.tool.TempletonControllerJob.TOKEN_FILE_ARG_PLACEHOLDER
>  to have an "=" sign in it. Or, preProcessForWindows() itself could be 
> changed.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6447) Bucket map joins in hive-tez

[Vikram Dixit K (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vikram Dixit K updated HIVE-6447:
-

Attachment: HIVE-6447.12.patch

> Bucket map joins in hive-tez
> 
>
> Key: HIVE-6447
> URL: https://issues.apache.org/jira/browse/HIVE-6447
> Project: Hive
>  Issue Type: Bug
>  Components: Tez
>Affects Versions: tez-branch
>Reporter: Vikram Dixit K
>Assignee: Vikram Dixit K
> Attachments: HIVE-6447.1.patch, HIVE-6447.10.patch, 
> HIVE-6447.11.patch, HIVE-6447.12.patch, HIVE-6447.2.patch, HIVE-6447.3.patch, 
> HIVE-6447.4.patch, HIVE-6447.5.patch, HIVE-6447.6.patch, HIVE-6447.7.patch, 
> HIVE-6447.8.patch, HIVE-6447.9.patch, HIVE-6447.WIP.patch
>
>
> Support bucket map joins in tez.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HIVE-6758) Beeline only works in interactive mode

[Johndee Burks (JIRA)]

Johndee Burks created HIVE-6758:
---

 Summary: Beeline only works in interactive mode
 Key: HIVE-6758
 URL: https://issues.apache.org/jira/browse/HIVE-6758
 Project: Hive
  Issue Type: Improvement
  Components: CLI
Affects Versions: 0.12.0, 0.11.0
 Environment: CDH4.5
Reporter: Johndee Burks


In hive CLI you could easily integrate its use into a script and back ground 
the process like this: 

hive -e "some query" &

Beeline does not run when you do the same even with the -f switch. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)