[jira] [Commented] (HIVE-8054) Disable hive.optimize.union.remove when hive.execution.engine=spark [Spark Branch]

[Na Yang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14133111#comment-14133111
 ] 

Na Yang commented on HIVE-8054:
---

review board link: https://reviews.apache.org/r/25619

> Disable hive.optimize.union.remove when hive.execution.engine=spark [Spark 
> Branch]
> --
>
> Key: HIVE-8054
> URL: https://issues.apache.org/jira/browse/HIVE-8054
> Project: Hive
>  Issue Type: Improvement
>  Components: Spark
>Reporter: Xuefu Zhang
>Assignee: Na Yang
>  Labels: Spark-M1
> Attachments: HIVE-8054-spark.patch
>
>
> Option hive.optimize.union.remove introduced in HIVE-3276 removes union 
> operators from the operator graph in certain cases as an optimization reduce 
> the number of MR jobs. While making sense in MR, this optimization is 
> actually harmful to an execution engine such as Spark, which natives supports 
> union without requiring additional jobs. This is because removing union 
> operator creates disjointed operator graphs, each graph generating a job, and 
> thus this optimization requires more jobs to run the query. Not to mention 
> the additional complexity handling linked FS descriptors.
> I propose that we disable such optimization when the execution engine is 
> Spark.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 25619: HIVE-8054:Disable hive.optimize.union.remove when hive.execution.engine=spark [Spark Branch]

[Na Yang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25619/
---

Review request for hive, Brock Noland, Szehon Ho, and Xuefu Zhang.


Repository: hive-git


Description
---

1. Disabled hive.optimize.union.remove for spark
2. removed the duplicated filesinks to avoid generating muliple merge/move 
works when the hive.merge.sparkfiles=true


Diffs
-

  ql/src/java/org/apache/hadoop/hive/ql/optimizer/unionproc/UnionProcessor.java 
c973d98 
  ql/src/java/org/apache/hadoop/hive/ql/parse/spark/GenSparkUtils.java dbf4fb2 
  ql/src/test/queries/clientpositive/union_remove_1.q c87b3fe 
  ql/src/test/queries/clientpositive/union_remove_10.q 6701952 
  ql/src/test/queries/clientpositive/union_remove_11.q 4b2fa42 
  ql/src/test/queries/clientpositive/union_remove_12.q 69d0d0a 
  ql/src/test/queries/clientpositive/union_remove_13.q 7605f0e 
  ql/src/test/queries/clientpositive/union_remove_14.q a4fdfc8 
  ql/src/test/queries/clientpositive/union_remove_15.q e3c937b 
  ql/src/test/queries/clientpositive/union_remove_16.q 537078b 
  ql/src/test/queries/clientpositive/union_remove_17.q d70f3d3 
  ql/src/test/queries/clientpositive/union_remove_18.q 6352bc3 
  ql/src/test/queries/clientpositive/union_remove_19.q 8d75afa 
  ql/src/test/queries/clientpositive/union_remove_2.q 83cd288 
  ql/src/test/queries/clientpositive/union_remove_20.q f80f7c1 
  ql/src/test/queries/clientpositive/union_remove_21.q 8963c25 
  ql/src/test/queries/clientpositive/union_remove_22.q b0c1ccd 
  ql/src/test/queries/clientpositive/union_remove_23.q a1b989a 
  ql/src/test/queries/clientpositive/union_remove_24.q ec561e0 
  ql/src/test/queries/clientpositive/union_remove_25.q 76c1ff5 
  ql/src/test/queries/clientpositive/union_remove_3.q 9617f73 
  ql/src/test/queries/clientpositive/union_remove_4.q cae323b 
  ql/src/test/queries/clientpositive/union_remove_5.q 5df84e1 
  ql/src/test/queries/clientpositive/union_remove_6.q bfce26d 
  ql/src/test/queries/clientpositive/union_remove_7.q 3a95674 
  ql/src/test/queries/clientpositive/union_remove_8.q a83a43e 
  ql/src/test/queries/clientpositive/union_remove_9.q e71f6dd 
  ql/src/test/results/clientpositive/spark/union10.q.out 20c681e 
  ql/src/test/results/clientpositive/spark/union18.q.out 3f37a0a 
  ql/src/test/results/clientpositive/spark/union19.q.out 6922fcd 
  ql/src/test/results/clientpositive/spark/union28.q.out 8bd5218 
  ql/src/test/results/clientpositive/spark/union29.q.out b9546ef 
  ql/src/test/results/clientpositive/spark/union3.q.out 3ae6536 
  ql/src/test/results/clientpositive/spark/union30.q.out 12717a1 
  ql/src/test/results/clientpositive/spark/union33.q.out b89757f 
  ql/src/test/results/clientpositive/spark/union4.q.out 6341cd9 
  ql/src/test/results/clientpositive/spark/union6.q.out 263d9f4 
  ql/src/test/results/clientpositive/spark/union_remove_1.q.out c07fbb1 
  ql/src/test/results/clientpositive/spark/union_remove_10.q.out 927a15d 
  ql/src/test/results/clientpositive/spark/union_remove_11.q.out 96651e1 
  ql/src/test/results/clientpositive/spark/union_remove_15.q.out 463ae63 
  ql/src/test/results/clientpositive/spark/union_remove_16.q.out 0954ae4 
  ql/src/test/results/clientpositive/spark/union_remove_18.q.out fba6634 
  ql/src/test/results/clientpositive/spark/union_remove_19.q.out a486323 
  ql/src/test/results/clientpositive/spark/union_remove_2.q.out 268a8d9 
  ql/src/test/results/clientpositive/spark/union_remove_21.q.out 384f191 
  ql/src/test/results/clientpositive/spark/union_remove_24.q.out 0494612 
  ql/src/test/results/clientpositive/spark/union_remove_25.q.out e93868e 
  ql/src/test/results/clientpositive/spark/union_remove_4.q.out cc46dda 
  ql/src/test/results/clientpositive/spark/union_remove_5.q.out f6cdeb3 
  ql/src/test/results/clientpositive/spark/union_remove_7.q.out b3ec85b 
  ql/src/test/results/clientpositive/spark/union_remove_8.q.out d55b60d 
  ql/src/test/results/clientpositive/spark/union_remove_9.q.out 1f0260c 

Diff: https://reviews.apache.org/r/25619/diff/


Testing
---


Thanks,

Na Yang

Re: Review Request 25619: HIVE-8054:Disable hive.optimize.union.remove when hive.execution.engine=spark [Spark Branch]

[Na Yang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25619/
---

(Updated Sept. 14, 2014, 6:38 a.m.)


Review request for hive, Brock Noland, Szehon Ho, and Xuefu Zhang.


Repository: hive-git


Description
---

1. Disabled hive.optimize.union.remove for spark
2. removed the duplicated filesinks to avoid generating muliple merge/move 
works when the hive.merge.sparkfiles=true


Diffs
-

  ql/src/java/org/apache/hadoop/hive/ql/optimizer/unionproc/UnionProcessor.java 
c973d98 
  ql/src/java/org/apache/hadoop/hive/ql/parse/spark/GenSparkUtils.java dbf4fb2 
  ql/src/test/queries/clientpositive/union_remove_1.q c87b3fe 
  ql/src/test/queries/clientpositive/union_remove_10.q 6701952 
  ql/src/test/queries/clientpositive/union_remove_11.q 4b2fa42 
  ql/src/test/queries/clientpositive/union_remove_12.q 69d0d0a 
  ql/src/test/queries/clientpositive/union_remove_13.q 7605f0e 
  ql/src/test/queries/clientpositive/union_remove_14.q a4fdfc8 
  ql/src/test/queries/clientpositive/union_remove_15.q e3c937b 
  ql/src/test/queries/clientpositive/union_remove_16.q 537078b 
  ql/src/test/queries/clientpositive/union_remove_17.q d70f3d3 
  ql/src/test/queries/clientpositive/union_remove_18.q 6352bc3 
  ql/src/test/queries/clientpositive/union_remove_19.q 8d75afa 
  ql/src/test/queries/clientpositive/union_remove_2.q 83cd288 
  ql/src/test/queries/clientpositive/union_remove_20.q f80f7c1 
  ql/src/test/queries/clientpositive/union_remove_21.q 8963c25 
  ql/src/test/queries/clientpositive/union_remove_22.q b0c1ccd 
  ql/src/test/queries/clientpositive/union_remove_23.q a1b989a 
  ql/src/test/queries/clientpositive/union_remove_24.q ec561e0 
  ql/src/test/queries/clientpositive/union_remove_25.q 76c1ff5 
  ql/src/test/queries/clientpositive/union_remove_3.q 9617f73 
  ql/src/test/queries/clientpositive/union_remove_4.q cae323b 
  ql/src/test/queries/clientpositive/union_remove_5.q 5df84e1 
  ql/src/test/queries/clientpositive/union_remove_6.q bfce26d 
  ql/src/test/queries/clientpositive/union_remove_7.q 3a95674 
  ql/src/test/queries/clientpositive/union_remove_8.q a83a43e 
  ql/src/test/queries/clientpositive/union_remove_9.q e71f6dd 
  ql/src/test/results/clientpositive/spark/union10.q.out 20c681e 
  ql/src/test/results/clientpositive/spark/union18.q.out 3f37a0a 
  ql/src/test/results/clientpositive/spark/union19.q.out 6922fcd 
  ql/src/test/results/clientpositive/spark/union28.q.out 8bd5218 
  ql/src/test/results/clientpositive/spark/union29.q.out b9546ef 
  ql/src/test/results/clientpositive/spark/union3.q.out 3ae6536 
  ql/src/test/results/clientpositive/spark/union30.q.out 12717a1 
  ql/src/test/results/clientpositive/spark/union33.q.out b89757f 
  ql/src/test/results/clientpositive/spark/union4.q.out 6341cd9 
  ql/src/test/results/clientpositive/spark/union6.q.out 263d9f4 
  ql/src/test/results/clientpositive/spark/union_remove_1.q.out c07fbb1 
  ql/src/test/results/clientpositive/spark/union_remove_10.q.out 927a15d 
  ql/src/test/results/clientpositive/spark/union_remove_11.q.out 96651e1 
  ql/src/test/results/clientpositive/spark/union_remove_15.q.out 463ae63 
  ql/src/test/results/clientpositive/spark/union_remove_16.q.out 0954ae4 
  ql/src/test/results/clientpositive/spark/union_remove_18.q.out fba6634 
  ql/src/test/results/clientpositive/spark/union_remove_19.q.out a486323 
  ql/src/test/results/clientpositive/spark/union_remove_2.q.out 268a8d9 
  ql/src/test/results/clientpositive/spark/union_remove_21.q.out 384f191 
  ql/src/test/results/clientpositive/spark/union_remove_24.q.out 0494612 
  ql/src/test/results/clientpositive/spark/union_remove_25.q.out e93868e 
  ql/src/test/results/clientpositive/spark/union_remove_4.q.out cc46dda 
  ql/src/test/results/clientpositive/spark/union_remove_5.q.out f6cdeb3 
  ql/src/test/results/clientpositive/spark/union_remove_7.q.out b3ec85b 
  ql/src/test/results/clientpositive/spark/union_remove_8.q.out d55b60d 
  ql/src/test/results/clientpositive/spark/union_remove_9.q.out 1f0260c 

Diff: https://reviews.apache.org/r/25619/diff/


Testing
---


Thanks,

Na Yang

[jira] [Updated] (HIVE-8054) Disable hive.optimize.union.remove when hive.execution.engine=spark [Spark Branch]

[Na Yang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Na Yang updated HIVE-8054:
--
Attachment: HIVE-8054-spark.patch

> Disable hive.optimize.union.remove when hive.execution.engine=spark [Spark 
> Branch]
> --
>
> Key: HIVE-8054
> URL: https://issues.apache.org/jira/browse/HIVE-8054
> Project: Hive
>  Issue Type: Improvement
>  Components: Spark
>Reporter: Xuefu Zhang
>Assignee: Na Yang
>  Labels: Spark-M1
> Attachments: HIVE-8054-spark.patch
>
>
> Option hive.optimize.union.remove introduced in HIVE-3276 removes union 
> operators from the operator graph in certain cases as an optimization reduce 
> the number of MR jobs. While making sense in MR, this optimization is 
> actually harmful to an execution engine such as Spark, which natives supports 
> union without requiring additional jobs. This is because removing union 
> operator creates disjointed operator graphs, each graph generating a job, and 
> thus this optimization requires more jobs to run the query. Not to mention 
> the additional complexity handling linked FS descriptors.
> I propose that we disable such optimization when the execution engine is 
> Spark.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8094) add LIKE keyword support for SHOW FUNCTIONS

[peter liu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14133100#comment-14133100
 ] 

peter liu commented on HIVE-8094:
-

I'm working on it, can we assign it to me?

> add LIKE keyword support for SHOW FUNCTIONS
> ---
>
> Key: HIVE-8094
> URL: https://issues.apache.org/jira/browse/HIVE-8094
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 0.13.1
>Reporter: peter liu
> Fix For: 0.14.0
>
>
> It would be nice to  add LIKE keyword support for SHOW FUNCTIONS as below, 
> and keep the patterns consistent to the way as SHOW DATABASES, SHOW TABLES.
> bq. SHOW FUNCTIONS LIKE 'foo*';



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-7790) Update privileges to check for update and delete

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14133097#comment-14133097
 ] 

Hive QA commented on HIVE-7790:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12668634/HIVE-7790.patch

{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 6282 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_metadataonly1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge5
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge6
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge7
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_select_dummy_source
{noformat}

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/787/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/787/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-787/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12668634

> Update privileges to check for update and delete
> 
>
> Key: HIVE-7790
> URL: https://issues.apache.org/jira/browse/HIVE-7790
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Alan Gates
>Assignee: Alan Gates
> Attachments: HIVE-7790.patch
>
>
> In the new SQLStdAuth scheme, we need to add UPDATE and DELETE as operations 
> and add ability check for them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-7822) how to merge two hive metastores' metadata stored in different databases (such as mysql)

[wangmeng (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14133095#comment-14133095
 ] 

wangmeng commented on HIVE-7822:


thanks. for. your. advice! 发自网易邮箱手机版 在2014年09月14日 11:21，Xuefu Zhang (JIRA)写道： [ 
https://issues.apache.org/jira/browse/HIVE-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14133064#comment-14133064
 ] Xuefu Zhang commented on HIVE-7822: --- 
[~wangmeng] JIRA is to report issues or request features. Questions like what 
you presented is better sent to user list. I'm closing this JIRA. > how to 
merge two  hive metastores' metadata  stored in different databases (such as 
mysql) > 
--
 > >                 Key: HIVE-7822 >                 URL: 
https://issues.apache.org/jira/browse/HIVE-7822 >             Project: Hive >   
       Issue Type: Improvement >            Reporter: wangmeng > > Hi, What is 
a good way to merge  two hive metadata stored in different databases(such as 
mysql)? > Is there any way to get all history Hqls  from metastore?  I think  I 
need  to  run these  Hqls   in  another hive  metadata database  again. > 
Thanks -- This message was sent by Atlassian JIRA (v6.3.4#6332)


> how to merge two  hive metastores' metadata  stored in different databases 
> (such as mysql)
> --
>
> Key: HIVE-7822
> URL: https://issues.apache.org/jira/browse/HIVE-7822
> Project: Hive
>  Issue Type: Improvement
>Reporter: wangmeng
>
> Hi, What is a good way to merge  two hive metadata stored in different 
> databases(such as mysql)?
> Is there any way to get all history Hqls  from metastore?  I think  I need  
> to  run these  Hqls   in  another hive  metadata database  again.
> Thanks



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8094) add LIKE keyword support for SHOW FUNCTIONS

[peter liu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

peter liu updated HIVE-8094:

Description: 
It would be nice to  add LIKE keyword support for SHOW FUNCTIONS as below, and 
keep the patterns consistent to the way as SHOW DATABASES, SHOW TABLES.

bq. SHOW FUNCTIONS LIKE 'foo*';

  was:
It would be nice to  add LIKE keyword support for SHOW FUNCTIONS, and keep the 
patterns consistent to the way as SHOW DATABASES, SHOW TABLES.

bq. SHOW FUNCTIONS LIKE 'foo*';


> add LIKE keyword support for SHOW FUNCTIONS
> ---
>
> Key: HIVE-8094
> URL: https://issues.apache.org/jira/browse/HIVE-8094
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 0.13.1
>Reporter: peter liu
> Fix For: 0.14.0
>
>
> It would be nice to  add LIKE keyword support for SHOW FUNCTIONS as below, 
> and keep the patterns consistent to the way as SHOW DATABASES, SHOW TABLES.
> bq. SHOW FUNCTIONS LIKE 'foo*';



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8094) add LIKE keyword support for SHOW FUNCTIONS

[peter liu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

peter liu updated HIVE-8094:

Description: 
It would be nice to  add LIKE keyword support for SHOW FUNCTIONS, and keep the 
patterns consistent to the way as SHOW DATABASES, SHOW TABLES.

bq. SHOW FUNCTIONS LIKE 'foo*';

  was:
It would be nice to  add LIKE keyword support for SHOW FUNCTIONS, and keep the 
patterns consistent to the way as SHOW DATABASES, SHOW TABLES.

`SHOW FUNCTIONS LIKE 'foo*';


> add LIKE keyword support for SHOW FUNCTIONS
> ---
>
> Key: HIVE-8094
> URL: https://issues.apache.org/jira/browse/HIVE-8094
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 0.13.1
>Reporter: peter liu
> Fix For: 0.14.0
>
>
> It would be nice to  add LIKE keyword support for SHOW FUNCTIONS, and keep 
> the patterns consistent to the way as SHOW DATABASES, SHOW TABLES.
> bq. SHOW FUNCTIONS LIKE 'foo*';



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8094) add LIKE keyword support for SHOW FUNCTIONS

[peter liu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

peter liu updated HIVE-8094:

Description: 
It would be nice to  add LIKE keyword support for SHOW FUNCTIONS, and keep the 
patterns consistent to the way as SHOW DATABASES, SHOW TABLES.

`SHOW FUNCTIONS LIKE 'foo*';

  was:It would be nice to  add LIKE keyword support for SHOW FUNCTIONS, and 
keep the patterns consistent to the way as SHOW DATABASES, SHOW TABLES.


> add LIKE keyword support for SHOW FUNCTIONS
> ---
>
> Key: HIVE-8094
> URL: https://issues.apache.org/jira/browse/HIVE-8094
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 0.13.1
>Reporter: peter liu
> Fix For: 0.14.0
>
>
> It would be nice to  add LIKE keyword support for SHOW FUNCTIONS, and keep 
> the patterns consistent to the way as SHOW DATABASES, SHOW TABLES.
> `SHOW FUNCTIONS LIKE 'foo*';



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-8094) add LIKE keyword support for SHOW FUNCTIONS

[peter liu (JIRA)]

peter liu created HIVE-8094:
---

 Summary: add LIKE keyword support for SHOW FUNCTIONS
 Key: HIVE-8094
 URL: https://issues.apache.org/jira/browse/HIVE-8094
 Project: Hive
  Issue Type: Improvement
Affects Versions: 0.13.1
Reporter: peter liu
 Fix For: 0.14.0


It would be nice to  add LIKE keyword support for SHOW FUNCTIONS, and keep the 
patterns consistent to the way as SHOW DATABASES, SHOW TABLES.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-7790) Update privileges to check for update and delete

[Alan Gates (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14133077#comment-14133077
 ] 

Alan Gates commented on HIVE-7790:
--

Added review https://reviews.apache.org/r/25616/

> Update privileges to check for update and delete
> 
>
> Key: HIVE-7790
> URL: https://issues.apache.org/jira/browse/HIVE-7790
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Alan Gates
>Assignee: Alan Gates
> Attachments: HIVE-7790.patch
>
>
> In the new SQLStdAuth scheme, we need to add UPDATE and DELETE as operations 
> and add ability check for them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 25616: HIVE-7790 Update privileges to check for update and delete

[Alan Gates]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25616/
---

Review request for hive and Thejas Nair.


Bugs: HIVE-7790
https://issues.apache.org/jira/browse/HIVE-7790


Repository: hive-git


Description
---

Adds update and delete as action and adds checks for authorization during 
update and delete. Also adds passing of updated columns in case authorizer 
wishes to check them.


Diffs
-

  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/authorization/plugin/TestHiveAuthorizerCheckInvocation.java
 53d88b0 
  ql/src/java/org/apache/hadoop/hive/ql/Driver.java 298f429 
  ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java b2f66e0 
  ql/src/java/org/apache/hadoop/hive/ql/parse/UpdateDeleteSemanticAnalyzer.java 
3aaa09c 
  
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/AuthorizationUtils.java
 93df9f4 
  
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HivePrivilegeObject.java
 093b4fd 
  
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/sqlstd/Operation2Privilege.java
 3236341 
  ql/src/test/queries/clientnegative/authorization_delete_nodeletepriv.q 
PRE-CREATION 
  ql/src/test/queries/clientnegative/authorization_update_noupdatepriv.q 
PRE-CREATION 
  ql/src/test/queries/clientpositive/authorization_delete.q PRE-CREATION 
  ql/src/test/queries/clientpositive/authorization_delete_own_table.q 
PRE-CREATION 
  ql/src/test/queries/clientpositive/authorization_update.q PRE-CREATION 
  ql/src/test/queries/clientpositive/authorization_update_own_table.q 
PRE-CREATION 
  ql/src/test/results/clientnegative/authorization_delete_nodeletepriv.q.out 
PRE-CREATION 
  ql/src/test/results/clientnegative/authorization_update_noupdatepriv.q.out 
PRE-CREATION 
  ql/src/test/results/clientpositive/authorization_delete.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/authorization_delete_own_table.q.out 
PRE-CREATION 
  ql/src/test/results/clientpositive/authorization_update.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/authorization_update_own_table.q.out 
PRE-CREATION 

Diff: https://reviews.apache.org/r/25616/diff/


Testing
---

Added tests, both positive and negative, for update and delete, including 
ability to update and delete tables created by user.  Also added tests for 
passing correct update columns.


Thanks,

Alan Gates

[jira] [Updated] (HIVE-7790) Update privileges to check for update and delete

[Alan Gates (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Gates updated HIVE-7790:
-
Attachment: HIVE-7790.patch

Attaching a patch that adds update and delete as action and adds checks for 
authorization during update and delete.  Also adds passing of updated columns 
in case authorizer wishes to check them.

> Update privileges to check for update and delete
> 
>
> Key: HIVE-7790
> URL: https://issues.apache.org/jira/browse/HIVE-7790
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Alan Gates
>Assignee: Alan Gates
> Attachments: HIVE-7790.patch
>
>
> In the new SQLStdAuth scheme, we need to add UPDATE and DELETE as operations 
> and add ability check for them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-7790) Update privileges to check for update and delete

[Alan Gates (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Gates updated HIVE-7790:
-
Status: Patch Available  (was: Open)

> Update privileges to check for update and delete
> 
>
> Key: HIVE-7790
> URL: https://issues.apache.org/jira/browse/HIVE-7790
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Alan Gates
>Assignee: Alan Gates
> Attachments: HIVE-7790.patch
>
>
> In the new SQLStdAuth scheme, we need to add UPDATE and DELETE as operations 
> and add ability check for them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-6455) Scalable dynamic partitioning and bucketing optimization

[Lefty Leverenz (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14133075#comment-14133075
 ] 

Lefty Leverenz commented on HIVE-6455:
--

Doc note:  *hive.optimize.sort.dynamic.partition* has its description in 
HiveConf.java (post HIVE-6037), but it isn't documented in the wiki yet.

> Scalable dynamic partitioning and bucketing optimization
> 
>
> Key: HIVE-6455
> URL: https://issues.apache.org/jira/browse/HIVE-6455
> Project: Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Affects Versions: 0.13.0
>Reporter: Prasanth J
>Assignee: Prasanth J
>  Labels: TODOC13, TODOC14, optimization
> Fix For: 0.13.0, 0.14.0
>
> Attachments: HIVE-6455.1.patch, HIVE-6455.1.patch, 
> HIVE-6455.10.patch, HIVE-6455.10.patch, HIVE-6455.11.patch, 
> HIVE-6455.12.patch, HIVE-6455.13.patch, HIVE-6455.13.patch, 
> HIVE-6455.14.patch, HIVE-6455.15.patch, HIVE-6455.16.patch, 
> HIVE-6455.17.patch, HIVE-6455.17.patch.txt, HIVE-6455.18.patch, 
> HIVE-6455.19.patch, HIVE-6455.2.patch, HIVE-6455.20.patch, 
> HIVE-6455.21.patch, HIVE-6455.3.patch, HIVE-6455.4.patch, HIVE-6455.4.patch, 
> HIVE-6455.5.patch, HIVE-6455.6.patch, HIVE-6455.7.patch, HIVE-6455.8.patch, 
> HIVE-6455.9.patch, HIVE-6455.9.patch
>
>
> The current implementation of dynamic partition works by keeping at least one 
> record writer open per dynamic partition directory. In case of bucketing 
> there can be multispray file writers which further adds up to the number of 
> open record writers. The record writers of column oriented file format (like 
> ORC, RCFile etc.) keeps some sort of in-memory buffers (value buffer or 
> compression buffers) open all the time to buffer up the rows and compress 
> them before flushing it to disk. Since these buffers are maintained per 
> column basis the amount of constant memory that will required at runtime 
> increases as the number of partitions and number of columns per partition 
> increases. This often leads to OutOfMemory (OOM) exception in mappers or 
> reducers depending on the number of open record writers. Users often tune the 
> JVM heapsize (runtime memory) to get over such OOM issues. 
> With this optimization, the dynamic partition columns and bucketing columns 
> (in case of bucketed tables) are sorted before being fed to the reducers. 
> Since the partitioning and bucketing columns are sorted, each reducers can 
> keep only one record writer open at any time thereby reducing the memory 
> pressure on the reducers. This optimization is highly scalable as the number 
> of partition and number of columns per partition increases at the cost of 
> sorting the columns.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-860) Persistent distributed cache

[Lefty Leverenz (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14133073#comment-14133073
 ] 

Lefty Leverenz commented on HIVE-860:
-

Did this fall by the wayside?

> Persistent distributed cache
> 
>
> Key: HIVE-860
> URL: https://issues.apache.org/jira/browse/HIVE-860
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 0.12.0
>Reporter: Zheng Shao
>Assignee: Brock Noland
> Fix For: 0.14.0
>
> Attachments: HIVE-860.patch, HIVE-860.patch, HIVE-860.patch, 
> HIVE-860.patch, HIVE-860.patch, HIVE-860.patch, HIVE-860.patch, 
> HIVE-860.patch, HIVE-860.patch, HIVE-860.patch, HIVE-860.patch
>
>
> DistributedCache is shared across multiple jobs, if the hdfs file name is the 
> same.
> We need to make sure Hive put the same file into the same location every time 
> and do not overwrite if the file content is the same.
> We can achieve 2 different results:
> A1. Files added with the same name, timestamp, and md5 in the same session 
> will have a single copy in distributed cache.
> A2. Filed added with the same name, timestamp, and md5 will have a single 
> copy in distributed cache.
> A2 has a bigger benefit in sharing but may raise a question on when Hive 
> should clean it up in hdfs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8092) count(*) returns NULL instead of 0 when result is empty

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14133072#comment-14133072
 ] 

Hive QA commented on HIVE-8092:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12668630/HIVE-8092.03.patch

{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 6276 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_metadataonly1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge5
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge6
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge7
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_select_dummy_source
org.apache.hadoop.hive.ql.parse.TestParse.testParse_union
{noformat}

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/786/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/786/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-786/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12668630

> count(*) returns NULL instead of 0 when result is empty
> ---
>
> Key: HIVE-8092
> URL: https://issues.apache.org/jira/browse/HIVE-8092
> Project: Hive
>  Issue Type: Bug
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-8092.01.patch, HIVE-8092.02.patch, 
> HIVE-8092.03.patch
>
>
> In tez mode when vectorization is enabled, count returns NULL when result is 
> empty.
> Expected behavior: It should return 0
> count works as expected when vectorization is off



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HIVE-7822) how to merge two hive metastores' metadata stored in different databases (such as mysql)

[Xuefu Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang resolved HIVE-7822.
---
Resolution: Invalid

> how to merge two  hive metastores' metadata  stored in different databases 
> (such as mysql)
> --
>
> Key: HIVE-7822
> URL: https://issues.apache.org/jira/browse/HIVE-7822
> Project: Hive
>  Issue Type: Improvement
>Reporter: wangmeng
>
> Hi, What is a good way to merge  two hive metadata stored in different 
> databases(such as mysql)?
> Is there any way to get all history Hqls  from metastore?  I think  I need  
> to  run these  Hqls   in  another hive  metadata database  again.
> Thanks



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-7822) how to merge two hive metastores' metadata stored in different databases (such as mysql)

[Xuefu Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14133064#comment-14133064
 ] 

Xuefu Zhang commented on HIVE-7822:
---

[~wangmeng] JIRA is to report issues or request features. Questions like what 
you presented is better sent to user list.

I'm closing this JIRA.

> how to merge two  hive metastores' metadata  stored in different databases 
> (such as mysql)
> --
>
> Key: HIVE-7822
> URL: https://issues.apache.org/jira/browse/HIVE-7822
> Project: Hive
>  Issue Type: Improvement
>Reporter: wangmeng
>
> Hi, What is a good way to merge  two hive metadata stored in different 
> databases(such as mysql)?
> Is there any way to get all history Hqls  from metastore?  I think  I need  
> to  run these  Hqls   in  another hive  metadata database  again.
> Thanks



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8092) count(*) returns NULL instead of 0 when result is empty

[Matt McCline (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt McCline updated HIVE-8092:
---
Status: Patch Available  (was: In Progress)

> count(*) returns NULL instead of 0 when result is empty
> ---
>
> Key: HIVE-8092
> URL: https://issues.apache.org/jira/browse/HIVE-8092
> Project: Hive
>  Issue Type: Bug
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-8092.01.patch, HIVE-8092.02.patch, 
> HIVE-8092.03.patch
>
>
> In tez mode when vectorization is enabled, count returns NULL when result is 
> empty.
> Expected behavior: It should return 0
> count works as expected when vectorization is off



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8092) count(*) returns NULL instead of 0 when result is empty

[Matt McCline (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt McCline updated HIVE-8092:
---
Status: In Progress  (was: Patch Available)

> count(*) returns NULL instead of 0 when result is empty
> ---
>
> Key: HIVE-8092
> URL: https://issues.apache.org/jira/browse/HIVE-8092
> Project: Hive
>  Issue Type: Bug
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-8092.01.patch, HIVE-8092.02.patch
>
>
> In tez mode when vectorization is enabled, count returns NULL when result is 
> empty.
> Expected behavior: It should return 0
> count works as expected when vectorization is off



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8092) count(*) returns NULL instead of 0 when result is empty

[Matt McCline (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt McCline updated HIVE-8092:
---
Attachment: HIVE-8092.03.patch

> count(*) returns NULL instead of 0 when result is empty
> ---
>
> Key: HIVE-8092
> URL: https://issues.apache.org/jira/browse/HIVE-8092
> Project: Hive
>  Issue Type: Bug
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-8092.01.patch, HIVE-8092.02.patch, 
> HIVE-8092.03.patch
>
>
> In tez mode when vectorization is enabled, count returns NULL when result is 
> empty.
> Expected behavior: It should return 0
> count works as expected when vectorization is off



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-5690) Support subquery for single sourced multi query

[Xuefu Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-5690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14133033#comment-14133033
 ] 

Xuefu Zhang commented on HIVE-5690:
---

Sorry for chiming in late, but I'm not sure if I understand the use case 
correctly. Wouldn't the above query can be rewritten as
{code}
explain from (select distinct key, value ) X
insert overwrite table x1 select *  order by key
insert overwrite table x2 select *  order by value;
{code}

Since the multi-insert is to insert from a single source specified by the 
from-clause, allowing subquery in later select clause seems defeating that 
purpose.


> Support subquery for single sourced multi query
> ---
>
> Key: HIVE-5690
> URL: https://issues.apache.org/jira/browse/HIVE-5690
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: D13791.1.patch, HIVE-5690.10.patch.txt, 
> HIVE-5690.11.patch.txt, HIVE-5690.12.patch.txt, HIVE-5690.2.patch.txt, 
> HIVE-5690.3.patch.txt, HIVE-5690.4.patch.txt, HIVE-5690.5.patch.txt, 
> HIVE-5690.6.patch.txt, HIVE-5690.7.patch.txt, HIVE-5690.8.patch.txt, 
> HIVE-5690.9.patch.txt
>
>
> Single sourced multi (insert) query is very useful for various ETL processes 
> but it does not allow subqueries included. For example, 
> {noformat}
> explain from src 
> insert overwrite table x1 select * from (select distinct key,value) b order 
> by key
> insert overwrite table x2 select * from (select distinct key,value) c order 
> by value;
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8092) count(*) returns NULL instead of 0 when result is empty

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14133001#comment-14133001
 ] 

Hive QA commented on HIVE-8092:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12668604/HIVE-8092.02.patch

{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 6208 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_metadataonly1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge5
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge6
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge7
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_select_dummy_source
org.apache.hive.hcatalog.pig.TestOrcHCatLoader.testReadDataPrimitiveTypes
{noformat}

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/785/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/785/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-785/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12668604

> count(*) returns NULL instead of 0 when result is empty
> ---
>
> Key: HIVE-8092
> URL: https://issues.apache.org/jira/browse/HIVE-8092
> Project: Hive
>  Issue Type: Bug
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-8092.01.patch, HIVE-8092.02.patch
>
>
> In tez mode when vectorization is enabled, count returns NULL when result is 
> empty.
> Expected behavior: It should return 0
> count works as expected when vectorization is off



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-7788) Generate plans for insert, update, and delete

[Alan Gates (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Gates updated HIVE-7788:
-
   Resolution: Fixed
Fix Version/s: 0.14.0
   Status: Resolved  (was: Patch Available)

Patch 4 checked in.  Thank you Thejas, Eugene, and Brock for the reviews.

> Generate plans for insert, update, and delete
> -
>
> Key: HIVE-7788
> URL: https://issues.apache.org/jira/browse/HIVE-7788
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor
>Reporter: Alan Gates
>Assignee: Alan Gates
> Fix For: 0.14.0
>
> Attachments: HIVE-7788.2.patch, HIVE-7788.3.patch, HIVE-7788.4.patch, 
> HIVE-7788.WIP.patch, HIVE-7788.patch
>
>
> Insert plans needs to be generated differently for ACID tables, plus we need 
> to be able to generate plans in the semantic analyzer for update and delete.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8088) HIVE-7812 makes TestMiniTezCliDriver tests to fail

[Gunther Hagleitner (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14132992#comment-14132992
 ] 

Gunther Hagleitner commented on HIVE-8088:
--

I've verified that the orc_merge failures are also related to 7812.

> HIVE-7812 makes TestMiniTezCliDriver tests to fail
> --
>
> Key: HIVE-8088
> URL: https://issues.apache.org/jira/browse/HIVE-8088
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.14.0
>Reporter: Prasanth J
>Assignee: Owen O'Malley
>
> HIVE-7812 makes the following test fail in trunk
> org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_metadataonly1
> org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan
> org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_select_dummy_source



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-7812) Disable CombineHiveInputFormat when ACID format is used

[Gunther Hagleitner (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14132962#comment-14132962
 ] 

Gunther Hagleitner commented on HIVE-7812:
--

These test failures are definitely related. I also don't see a +1 on this 
ticket. [~owen.omalley] can you revert this?

> Disable CombineHiveInputFormat when ACID format is used
> ---
>
> Key: HIVE-7812
> URL: https://issues.apache.org/jira/browse/HIVE-7812
> Project: Hive
>  Issue Type: Bug
>  Components: File Formats
>Reporter: Owen O'Malley
>Assignee: Owen O'Malley
> Fix For: 0.14.0
>
> Attachments: HIVE-7812.patch, HIVE-7812.patch, HIVE-7812.patch
>
>
> Currently the HiveCombineInputFormat complains when called on an ACID 
> directory. Modify HiveCombineInputFormat so that HiveInputFormat is used 
> instead if the directory is ACID format.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8081) "drop index if exists" fails if table specified does not exist

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14132946#comment-14132946
 ] 

Hive QA commented on HIVE-8081:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12668588/HIVE-8081.2.patch

{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 6208 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_drop_index
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_metadataonly1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge5
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge6
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge7
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_select_dummy_source
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority
org.apache.hive.hcatalog.pig.TestOrcHCatLoader.testReadDataPrimitiveTypes
org.apache.hive.service.TestHS2ImpersonationWithRemoteMS.testImpersonation
{noformat}

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/784/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/784/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-784/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12668588

> "drop index if exists" fails if table specified does not exist
> --
>
> Key: HIVE-8081
> URL: https://issues.apache.org/jira/browse/HIVE-8081
> Project: Hive
>  Issue Type: Bug
>Reporter: Jason Dere
>Assignee: Jason Dere
> Attachments: HIVE-8081.1.patch, HIVE-8081.2.patch
>
>
> Seems to be a regression in behavior from HIVE-7648.
> {noformat}
> FAILED: SemanticException [Error 10001]: Table not found missing_ddl_3
> 14/09/09 16:12:46 [main]: ERROR ql.Driver: FAILED: SemanticException [Error 
> 10001]: Table not found missing_ddl_3
> org.apache.hadoop.hive.ql.parse.SemanticException: Table not found 
> missing_ddl_3
>   at 
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.getTable(BaseSemanticAnalyzer.java:1243)
>   at 
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.getTable(BaseSemanticAnalyzer.java:1226)
>   at 
> org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeDropIndex(DDLSemanticAnalyzer.java:1148)
>   at 
> org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:326)
>   at 
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:208)
>   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:402)
>   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:298)
>   at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:992)
>   at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1062)
>   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:929)
>   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:919)
>   at 
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:246)
>   at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:198)
>   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:408)
>   at 
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:781)
>   at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675)
>   at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8092) count(*) returns NULL instead of 0 when result is empty

[Matt McCline (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt McCline updated HIVE-8092:
---
Status: Patch Available  (was: In Progress)

> count(*) returns NULL instead of 0 when result is empty
> ---
>
> Key: HIVE-8092
> URL: https://issues.apache.org/jira/browse/HIVE-8092
> Project: Hive
>  Issue Type: Bug
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-8092.01.patch, HIVE-8092.02.patch
>
>
> In tez mode when vectorization is enabled, count returns NULL when result is 
> empty.
> Expected behavior: It should return 0
> count works as expected when vectorization is off



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8092) count(*) returns NULL instead of 0 when result is empty

[Matt McCline (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt McCline updated HIVE-8092:
---
Attachment: HIVE-8092.02.patch

> count(*) returns NULL instead of 0 when result is empty
> ---
>
> Key: HIVE-8092
> URL: https://issues.apache.org/jira/browse/HIVE-8092
> Project: Hive
>  Issue Type: Bug
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-8092.01.patch, HIVE-8092.02.patch
>
>
> In tez mode when vectorization is enabled, count returns NULL when result is 
> empty.
> Expected behavior: It should return 0
> count works as expected when vectorization is off



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8092) count(*) returns NULL instead of 0 when result is empty

[Matt McCline (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt McCline updated HIVE-8092:
---
Status: In Progress  (was: Patch Available)

> count(*) returns NULL instead of 0 when result is empty
> ---
>
> Key: HIVE-8092
> URL: https://issues.apache.org/jira/browse/HIVE-8092
> Project: Hive
>  Issue Type: Bug
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-8092.01.patch
>
>
> In tez mode when vectorization is enabled, count returns NULL when result is 
> empty.
> Expected behavior: It should return 0
> count works as expected when vectorization is off



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8088) HIVE-7812 makes TestMiniTezCliDriver tests to fail

[Alan Gates (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14132907#comment-14132907
 ] 

Alan Gates commented on HIVE-8088:
--

org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge5
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge6
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge7

Also now failing.

> HIVE-7812 makes TestMiniTezCliDriver tests to fail
> --
>
> Key: HIVE-8088
> URL: https://issues.apache.org/jira/browse/HIVE-8088
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.14.0
>Reporter: Prasanth J
>Assignee: Owen O'Malley
>
> HIVE-7812 makes the following test fail in trunk
> org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_metadataonly1
> org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan
> org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_select_dummy_source



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HIVE-8093) Seeing failures in tez qfile tests on trunk

[Alan Gates (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Gates resolved HIVE-8093.
--
Resolution: Duplicate

> Seeing failures in tez qfile tests on trunk
> ---
>
> Key: HIVE-8093
> URL: https://issues.apache.org/jira/browse/HIVE-8093
> Project: Hive
>  Issue Type: Bug
>  Components: Tez
>Affects Versions: 0.14.0
>Reporter: Alan Gates
>Priority: Critical
>
> We are seeing the following tests fail on trunk and in every Hive QA run:
> org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_metadataonly1
> org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan
> org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge1
> org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge5
> org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge6
> org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge7
> org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_select_dummy_source



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-6090) Audit logs for HiveServer2

[Thiruvel Thirumoolan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14132902#comment-14132902
 ] 

Thiruvel Thirumoolan commented on HIVE-6090:


Thanks [~farisa], will rebase and upload.

> Audit logs for HiveServer2
> --
>
> Key: HIVE-6090
> URL: https://issues.apache.org/jira/browse/HIVE-6090
> Project: Hive
>  Issue Type: Improvement
>  Components: Diagnosability, HiveServer2
>Reporter: Thiruvel Thirumoolan
>Assignee: Thiruvel Thirumoolan
> Attachments: HIVE-6090.patch
>
>
> HiveMetastore has audit logs and would like to audit all queries or requests 
> to HiveServer2 also. This will help in understanding how the APIs were used, 
> queries submitted, users etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8072) TesParse_union is failing on trunk

[Prasanth J (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14132898#comment-14132898
 ] 

Prasanth J commented on HIVE-8072:
--

My experience with TestParse failires was random as well. Most of the time, 
explicitly running the single test like union used to fail but when all tests 
are ran together like "mvn test -Dtest=TestParse -Phadoop-2" it used to pass. 
My suspicion is related to interners and it depends on GC as they use weak 
references. 

> TesParse_union is failing on trunk
> --
>
> Key: HIVE-8072
> URL: https://issues.apache.org/jira/browse/HIVE-8072
> Project: Hive
>  Issue Type: Task
>  Components: Tests
>Affects Versions: 0.14.0
>Reporter: Ashutosh Chauhan
> Attachments: HIVE-8072.patch
>
>
> Needs golden file update



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-7946) CBO: Merge CBO changes to Trunk

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14132897#comment-14132897
 ] 

Hive QA commented on HIVE-7946:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12668561/HIVE-7946.8.patch

{color:red}ERROR:{color} -1 due to 350 failed/errored test(s), 5577 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver_accumulo_predicate_pushdown
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_allcolref_in_udf
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_partition_coltype
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ambiguous_col
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_analyze_table_null_partition
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_filter
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_groupby
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_join
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_limit
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_part
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_select
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_table
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_union
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ansi_sql_arithmetic
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_explain
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_view_sqlstd
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join10
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join11
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join16
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join18
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join18_multi_distinct
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join22
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join27
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join_nulls
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join_reordering_values
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join_without_localtask
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_avro_partitioned_native
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_binarysortable_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket_groupby
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_char_cast
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_combine2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_constprog2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer10
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer15
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer7
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_count
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_create_func1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_create_genericudaf
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_create_view
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_create_view_partitioned
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cross_join
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cross_product_check_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cross_product_check_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_database
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_precision
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_udf
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_distinct_stats
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_dynpart_sort_opt_vectorization
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_exim_04_evolved_parts
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_explain_dependency
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_explain_logical
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver

Re: Broken build

[Prasanth Jayachandran]

Hi Alan

The failures looks related to HIVE-7812. I created 
https://issues.apache.org/jira/browse/HIVE-8088 for the same. HIVE-7704 had a 
clean run just before the last one. The last run was just to make sure the 
rebase of patch was good. Since all tests in HIVE-7704 uses 
CombineHiveInputFormat the recent changes to HIVE-7812 seems to be the reason. 

Thanks
Prasanth

Sent from my iPhone

> On Sep 13, 2014, at 6:52 AM, Alan Gates  wrote:
> 
> There are 7 tez qfile tests that are failing on every HiveQA run.  They fail 
> for me when I run them on trunk.  I've filed: 
> https://issues.apache.org/jira/browse/HIVE-8093 for them.
> 
> I'm guessing this is related to the recent checkin for HIVE-7704, since many 
> of those tests were added in that commit.
> 
> Alan.
> -- 
> Sent with Postbox 
> 
> -- 
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to 
> which it is addressed and may contain information that is confidential, 
> privileged and exempt from disclosure under applicable law. If the reader of 
> this message is not the intended recipient, you are hereby notified that any 
> printing, copying, dissemination, distribution, disclosure or forwarding of 
> this communication is strictly prohibited. If you have received this 
> communication in error, please contact the sender immediately and delete it 
> from your system. Thank You.

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

[jira] [Updated] (HIVE-8039) [CBO] Handle repeated alias

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-8039:
---
Attachment: HIVE-8039.cbo.patch

> [CBO] Handle repeated alias
> ---
>
> Key: HIVE-8039
> URL: https://issues.apache.org/jira/browse/HIVE-8039
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-8039.cbo.patch
>
>
> Relax condition in CBO of not allowing repeated alias.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8039) [CBO] Handle repeated alias

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-8039:
---
Status: Patch Available  (was: Open)

> [CBO] Handle repeated alias
> ---
>
> Key: HIVE-8039
> URL: https://issues.apache.org/jira/browse/HIVE-8039
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-8039.cbo.patch
>
>
> Relax condition in CBO of not allowing repeated alias.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8072) TesParse_union is failing on trunk

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-8072:
---
Assignee: (was: Ashutosh Chauhan)

> TesParse_union is failing on trunk
> --
>
> Key: HIVE-8072
> URL: https://issues.apache.org/jira/browse/HIVE-8072
> Project: Hive
>  Issue Type: Task
>  Components: Tests
>Affects Versions: 0.14.0
>Reporter: Ashutosh Chauhan
> Attachments: HIVE-8072.patch
>
>
> Needs golden file update



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8072) TesParse_union is failing on trunk

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-8072:
---
Status: Open  (was: Patch Available)

This needs more investigation, since it seems this failure is random.

> TesParse_union is failing on trunk
> --
>
> Key: HIVE-8072
> URL: https://issues.apache.org/jira/browse/HIVE-8072
> Project: Hive
>  Issue Type: Task
>  Components: Tests
>Affects Versions: 0.14.0
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-8072.patch
>
>
> Needs golden file update



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8081) "drop index if exists" fails if table specified does not exist

[Jason Dere (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Dere updated HIVE-8081:
-
Attachment: HIVE-8081.2.patch

> "drop index if exists" fails if table specified does not exist
> --
>
> Key: HIVE-8081
> URL: https://issues.apache.org/jira/browse/HIVE-8081
> Project: Hive
>  Issue Type: Bug
>Reporter: Jason Dere
>Assignee: Jason Dere
> Attachments: HIVE-8081.1.patch, HIVE-8081.2.patch
>
>
> Seems to be a regression in behavior from HIVE-7648.
> {noformat}
> FAILED: SemanticException [Error 10001]: Table not found missing_ddl_3
> 14/09/09 16:12:46 [main]: ERROR ql.Driver: FAILED: SemanticException [Error 
> 10001]: Table not found missing_ddl_3
> org.apache.hadoop.hive.ql.parse.SemanticException: Table not found 
> missing_ddl_3
>   at 
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.getTable(BaseSemanticAnalyzer.java:1243)
>   at 
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.getTable(BaseSemanticAnalyzer.java:1226)
>   at 
> org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeDropIndex(DDLSemanticAnalyzer.java:1148)
>   at 
> org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:326)
>   at 
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:208)
>   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:402)
>   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:298)
>   at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:992)
>   at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1062)
>   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:929)
>   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:919)
>   at 
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:246)
>   at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:198)
>   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:408)
>   at 
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:781)
>   at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675)
>   at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-7932) It may cause NP exception when add accessed columns to ReadEntity

[Prasad Mujumdar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14132828#comment-14132828
 ] 

Prasad Mujumdar commented on HIVE-7932:
---

[~Huang Xiaomeng] Thanks for finding the issue and providing a patch. 
The change look fine to me. Would it be possible to add a simple testcase that 
verifies this codepath ? 

> It may cause NP exception when add accessed columns to ReadEntity
> -
>
> Key: HIVE-7932
> URL: https://issues.apache.org/jira/browse/HIVE-7932
> Project: Hive
>  Issue Type: Bug
>Reporter: Xiaomeng Huang
>Assignee: Xiaomeng Huang
> Attachments: HIVE-7932.001.patch
>
>
> {code}
> case TABLE:
>entity.getAccessedColumns().addAll(
>   tableToColumnAccessMap.get(entity.getTable().getCompleteName()));
> {code}
> if  tableToColumnAccessMap.get(entity.getTable().getCompleteName()) is null, 
> addAll(null) will throw null pointer exception.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8086) Disable Trivial Project Removal Rule, Fix Result Schema

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14132823#comment-14132823
 ] 

Hive QA commented on HIVE-8086:
---



{color:red}Overall{color}: -1 no tests executed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12668560/HIVE-8086.1.patch

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/782/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/782/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-782/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit 
status 1 and output '+ [[ -n /usr/java/jdk1.7.0_45-cloudera ]]
+ export JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ export 
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/java/jdk1.6.0_34/bin:/usr/local/apache-maven-3.0.5/bin:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.6.0_34/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+ 
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/java/jdk1.6.0_34/bin:/usr/local/apache-maven-3.0.5/bin:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.6.0_34/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost 
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost 
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-782/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ svn = \s\v\n ]]
+ [[ -n '' ]]
+ [[ -d apache-svn-trunk-source ]]
+ [[ ! -d apache-svn-trunk-source/.svn ]]
+ [[ ! -d apache-svn-trunk-source ]]
+ cd apache-svn-trunk-source
+ svn revert -R .
Reverted 
'ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/VectorUDAFCountStar.java'
Reverted 
'ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/VectorUDAFCount.java'
Reverted 
'ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java'
++ egrep -v '^X|^Performing status on external'
++ awk '{print $2}'
++ svn status --no-ignore
+ rm -rf target datanucleus.log ant/target shims/target shims/0.20/target 
shims/0.20S/target shims/0.23/target shims/aggregator/target 
shims/common/target shims/common-secure/target packaging/target 
hbase-handler/target testutils/target jdbc/target metastore/target 
itests/target itests/hcatalog-unit/target itests/test-serde/target 
itests/qtest/target itests/hive-unit-hadoop2/target itests/hive-minikdc/target 
itests/hive-unit/target itests/custom-serde/target itests/util/target 
hcatalog/target hcatalog/core/target hcatalog/streaming/target 
hcatalog/server-extensions/target hcatalog/webhcat/svr/target 
hcatalog/webhcat/java-client/target hcatalog/hcatalog-pig-adapter/target 
accumulo-handler/target hwi/target common/target common/src/gen contrib/target 
service/target serde/target beeline/target odbc/target cli/target 
ql/dependency-reduced-pom.xml ql/target 
ql/src/test/results/clientpositive/vector_count_empty.q.out 
ql/src/test/results/clientpositive/tez/vector_count_empty.q.out 
ql/src/test/queries/clientpositive/vector_count_empty.q 
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/VectorUDAFCountMerge.java
+ svn update

Fetching external item into 'hcatalog/src/test/e2e/harness'
External at revision 1624764.

At revision 1624764.
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh 
/data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12668560

> Disable Trivial Project Removal Rule, Fix Result Schema
> ---
>
> Key: HIVE-8086
> URL: https://issues.apache.org/jira/browse/HIVE-8086
> Project: Hive
>  Issue Type: Sub-task
>  Components: CBO
>Reporter: Laljo John Pullokkaran
>Assignee: Laljo John Pullokkaran
> Attachments: HIVE-8086.1.patch, HIVE-8086.patch
>
>
> Disable trivial project rule till Optiq-407 gets

[jira] [Commented] (HIVE-8092) count(*) returns NULL instead of 0 when result is empty

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14132822#comment-14132822
 ] 

Hive QA commented on HIVE-8092:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12668548/HIVE-8092.01.patch

{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 6209 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_metadataonly1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge5
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge6
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge7
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_select_dummy_source
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testCountLongEmpty
org.apache.hive.hcatalog.pig.TestOrcHCatLoader.testReadDataPrimitiveTypes
org.apache.hive.minikdc.TestJdbcWithMiniKdcSQLAuthHttp.org.apache.hive.minikdc.TestJdbcWithMiniKdcSQLAuthHttp
{noformat}

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/781/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/781/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-781/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12668548

> count(*) returns NULL instead of 0 when result is empty
> ---
>
> Key: HIVE-8092
> URL: https://issues.apache.org/jira/browse/HIVE-8092
> Project: Hive
>  Issue Type: Bug
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-8092.01.patch
>
>
> In tez mode when vectorization is enabled, count returns NULL when result is 
> empty.
> Expected behavior: It should return 0
> count works as expected when vectorization is off



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8083) Authorization DDLs should not enforce hive identifier syntax for user or group

[Prasad Mujumdar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14132811#comment-14132811
 ] 

Prasad Mujumdar commented on HIVE-8083:
---

The test failures don't look related to the patch.

> Authorization DDLs should not enforce hive identifier syntax for user or group
> --
>
> Key: HIVE-8083
> URL: https://issues.apache.org/jira/browse/HIVE-8083
> Project: Hive
>  Issue Type: Bug
>  Components: SQL, SQLStandardAuthorization
>Affects Versions: 0.13.0, 0.13.1
>Reporter: Prasad Mujumdar
>Assignee: Prasad Mujumdar
> Attachments: HIVE-8083.1.patch
>
>
> The compiler expects principals (user, group and role) as hive identifiers 
> for authorization DDLs. The user and group are entities that belong to 
> external namespace and we can't expect those to follow hive identifier syntax 
> rules. For example, a userid or group can contain '-' which is not allowed by 
> compiler.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-6090) Audit logs for HiveServer2

[Adam Faris (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14132804#comment-14132804
 ] 

Adam Faris commented on HIVE-6090:
--

Hi Thiruvel, I came across this jira while looking for a means of logging 
client connections to HS2.  When I use 'git apply' against your patch, git 
states the patch is corrupt on line 79.  As the patch is ~9 months old, would 
you mind refactoring it against a fresh pull on master? If you read the [how to 
contribute|https://cwiki.apache.org/confluence/display/Hive/HowToContribute#HowToContribute-AttachingandSubmittingaPatch]
 wiki it explains how to generate a diff using 'git diff --no-prefix  > 
HIVE-6090.2.patch'.  Once that's done you'll want to open a reviewboard request 
(also on the HowToContribute wiki).  Thanks

> Audit logs for HiveServer2
> --
>
> Key: HIVE-6090
> URL: https://issues.apache.org/jira/browse/HIVE-6090
> Project: Hive
>  Issue Type: Improvement
>  Components: Diagnosability, HiveServer2
>Reporter: Thiruvel Thirumoolan
>Assignee: Thiruvel Thirumoolan
> Attachments: HIVE-6090.patch
>
>
> HiveMetastore has audit logs and would like to audit all queries or requests 
> to HiveServer2 also. This will help in understanding how the APIs were used, 
> queries submitted, users etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [ANNOUNCE] New Hive Committer - Eugene Koifman

[Ted Yu]

Congratulations, Eugene.

[jira] [Commented] (HIVE-8078) ORC Delta encoding corrupts data when delta overflows long

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14132785#comment-14132785
 ] 

Hive QA commented on HIVE-8078:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12668506/HIVE-8078.1.patch

{color:red}ERROR:{color} -1 due to 25 failed/errored test(s), 6216 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_part
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_dynpart_sort_opt_vectorization
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_extrapolate_part_stats_full
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_extrapolate_part_stats_partial
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_analyze
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_merge1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_merge5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_merge6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_merge7
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_merge_incompat1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_merge_incompat2
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_opt_vectorization
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_metadataonly1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_analyze
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge5
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge6
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge7
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge_incompat1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge_incompat2
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_select_dummy_source
org.apache.hadoop.hive.ql.io.orc.TestFileDump.testDictionaryThreshold
org.apache.hadoop.hive.ql.io.orc.TestFileDump.testDump
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testCombinationInputFormatWithAcid
{noformat}

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/780/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/780/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-780/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 25 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12668506

> ORC Delta encoding corrupts data when delta overflows long
> --
>
> Key: HIVE-8078
> URL: https://issues.apache.org/jira/browse/HIVE-8078
> Project: Hive
>  Issue Type: Bug
>  Components: File Formats
>Affects Versions: 0.14.0, 0.13.1
>Reporter: Tim Patterson
>Priority: Critical
> Attachments: HIVE-8078-testcase.patch, HIVE-8078.1.patch
>
>
> There is an issue with the integer encoding that can cause corruption in 
> certain cases.
> The following 3 longs cause this failure.
> 4513343538618202711
> 2911390882471569739
> -9181829309989854913
> I believe that even though the numbers are in decreasing order, the delta 
> between the last two numbers overflows causing a positive delta, in this case 
> the last digit ends up being corrupted (the delta is applied for the wrong 
> sign resulting in -3442132998776557225 instead of -9181829309989854913.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Broken build

[Alan Gates]

There are 7 tez qfile tests that are failing on every HiveQA run.  They 
fail for me when I run them on trunk.  I've filed: 
https://issues.apache.org/jira/browse/HIVE-8093 for them.


I'm guessing this is related to the recent checkin for HIVE-7704, since 
many of those tests were added in that commit.


Alan.
--
Sent with Postbox 

--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

[jira] [Commented] (HIVE-8085) stats optimizer should not use Description annotation to figure out function mapping (because FunctionRegistry doesn't)

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14132734#comment-14132734
 ] 

Hive QA commented on HIVE-8085:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12668490/HIVE-8085.patch

{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 6208 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_metadataonly1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge5
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge6
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge7
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_select_dummy_source
{noformat}

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/779/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/779/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-779/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12668490

> stats optimizer should not use Description annotation to figure out function 
> mapping (because FunctionRegistry doesn't)
> ---
>
> Key: HIVE-8085
> URL: https://issues.apache.org/jira/browse/HIVE-8085
> Project: Hive
>  Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Minor
> Attachments: HIVE-8085.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-8093) Seeing failures in tez qfile tests on trunk

[Alan Gates (JIRA)]

Alan Gates created HIVE-8093:


 Summary: Seeing failures in tez qfile tests on trunk
 Key: HIVE-8093
 URL: https://issues.apache.org/jira/browse/HIVE-8093
 Project: Hive
  Issue Type: Bug
  Components: Tez
Affects Versions: 0.14.0
Reporter: Alan Gates
Priority: Critical


We are seeing the following tests fail on trunk and in every Hive QA run:

org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_metadataonly1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge5
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge6
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge7
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_select_dummy_source



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-7788) Generate plans for insert, update, and delete

[Alan Gates (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14132700#comment-14132700
 ] 

Alan Gates commented on HIVE-7788:
--

I see these 7 tests failing on trunk as well, so I'm guessing they aren't 
related to this patch.

> Generate plans for insert, update, and delete
> -
>
> Key: HIVE-7788
> URL: https://issues.apache.org/jira/browse/HIVE-7788
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor
>Reporter: Alan Gates
>Assignee: Alan Gates
> Attachments: HIVE-7788.2.patch, HIVE-7788.3.patch, HIVE-7788.4.patch, 
> HIVE-7788.WIP.patch, HIVE-7788.patch
>
>
> Insert plans needs to be generated differently for ACID tables, plus we need 
> to be able to generate plans in the semantic analyzer for update and delete.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8083) Authorization DDLs should not enforce hive identifier syntax for user or group

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14132679#comment-14132679
 ] 

Hive QA commented on HIVE-8083:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12668475/HIVE-8083.1.patch

{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 6209 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_metadataonly1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge5
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge6
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge7
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_select_dummy_source
org.apache.hive.hcatalog.pig.TestOrcHCatLoader.testReadDataPrimitiveTypes
org.apache.hive.service.TestHS2ImpersonationWithRemoteMS.testImpersonation
{noformat}

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/778/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/778/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-778/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12668475

> Authorization DDLs should not enforce hive identifier syntax for user or group
> --
>
> Key: HIVE-8083
> URL: https://issues.apache.org/jira/browse/HIVE-8083
> Project: Hive
>  Issue Type: Bug
>  Components: SQL, SQLStandardAuthorization
>Affects Versions: 0.13.0, 0.13.1
>Reporter: Prasad Mujumdar
>Assignee: Prasad Mujumdar
> Attachments: HIVE-8083.1.patch
>
>
> The compiler expects principals (user, group and role) as hive identifiers 
> for authorization DDLs. The user and group are entities that belong to 
> external namespace and we can't expect those to follow hive identifier syntax 
> rules. For example, a userid or group can contain '-' which is not allowed by 
> compiler.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-7100) Users of hive should be able to specify skipTrash when dropping tables.

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14132632#comment-14132632
 ] 

Hive QA commented on HIVE-7100:
---



{color:red}Overall{color}: -1 no tests executed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12668474/HIVE-7100.9.patch

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/777/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/777/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-777/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit 
status 1 and output '+ [[ -n /usr/java/jdk1.7.0_45-cloudera ]]
+ export JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ export 
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/java/jdk1.6.0_34/bin:/usr/local/apache-maven-3.0.5/bin:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.6.0_34/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+ 
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/java/jdk1.6.0_34/bin:/usr/local/apache-maven-3.0.5/bin:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.6.0_34/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost 
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost 
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-777/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ svn = \s\v\n ]]
+ [[ -n '' ]]
+ [[ -d apache-svn-trunk-source ]]
+ [[ ! -d apache-svn-trunk-source/.svn ]]
+ [[ ! -d apache-svn-trunk-source ]]
+ cd apache-svn-trunk-source
+ svn revert -R .
Reverted 'ql/src/test/results/clientpositive/drop_index.q.out'
Reverted 'ql/src/test/queries/clientpositive/drop_index.q'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java'
++ awk '{print $2}'
++ egrep -v '^X|^Performing status on external'
++ svn status --no-ignore
+ rm -rf target datanucleus.log ant/target shims/target shims/0.20/target 
shims/0.20S/target shims/0.23/target shims/aggregator/target 
shims/common/target shims/common-secure/target packaging/target 
hbase-handler/target testutils/target jdbc/target metastore/target 
itests/target itests/hcatalog-unit/target itests/test-serde/target 
itests/qtest/target itests/hive-unit-hadoop2/target itests/hive-minikdc/target 
itests/hive-unit/target itests/custom-serde/target itests/util/target 
hcatalog/target hcatalog/core/target hcatalog/streaming/target 
hcatalog/server-extensions/target hcatalog/webhcat/svr/target 
hcatalog/webhcat/java-client/target hcatalog/hcatalog-pig-adapter/target 
accumulo-handler/target hwi/target common/target common/src/gen service/target 
contrib/target serde/target beeline/target odbc/target cli/target 
ql/dependency-reduced-pom.xml ql/target
+ svn update

Fetching external item into 'hcatalog/src/test/e2e/harness'
External at revision 1624721.

At revision 1624721.
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh 
/data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12668474

> Users of hive should be able to specify skipTrash when dropping tables.
> ---
>
> Key: HIVE-7100
> URL: https://issues.apache.org/jira/browse/HIVE-7100
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 0.13.0
>Reporter: Ravi Prakash
>Assignee: david serafini
> Attachments: HIVE-7100.1.patch, HIVE-7100.2.patch, HIVE-7100.3.patch, 
> HIVE-7100.4.patch, HIVE-7100.5.patch, HIVE-7100.8.patch, HIVE-7100.9.patch, 
> HIVE-7100.patch
>
>
> Users of our clusters are often running up against their quota limits because 
> of Hive tables. When they drop tables, they have to then manually delete the 
> files from HDFS using skipTrash. This is cumbersome and unnecessary. We 
> should enable users to skipTrash directly when dropping t

[jira] [Commented] (HIVE-8081) "drop index if exists" fails if table specified does not exist

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14132631#comment-14132631
 ] 

Hive QA commented on HIVE-8081:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12668462/HIVE-8081.1.patch

{color:red}ERROR:{color} -1 due to 17 failed/errored test(s), 6208 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_index
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_mult_tables
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_mult_tables_compact
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_bitmap
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_compact
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_creation
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_metadataonly1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge5
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge6
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge7
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_select_dummy_source
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_index_compact_entry_limit
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_index_compact_size_limit
org.apache.hadoop.hive.ql.parse.TestParse.testParse_union
org.apache.hadoop.hive.ql.txn.compactor.TestCompactor.testStatsAfterCompactionPartTbl
{noformat}

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/776/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/776/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-776/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 17 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12668462

> "drop index if exists" fails if table specified does not exist
> --
>
> Key: HIVE-8081
> URL: https://issues.apache.org/jira/browse/HIVE-8081
> Project: Hive
>  Issue Type: Bug
>Reporter: Jason Dere
>Assignee: Jason Dere
> Attachments: HIVE-8081.1.patch
>
>
> Seems to be a regression in behavior from HIVE-7648.
> {noformat}
> FAILED: SemanticException [Error 10001]: Table not found missing_ddl_3
> 14/09/09 16:12:46 [main]: ERROR ql.Driver: FAILED: SemanticException [Error 
> 10001]: Table not found missing_ddl_3
> org.apache.hadoop.hive.ql.parse.SemanticException: Table not found 
> missing_ddl_3
>   at 
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.getTable(BaseSemanticAnalyzer.java:1243)
>   at 
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.getTable(BaseSemanticAnalyzer.java:1226)
>   at 
> org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeDropIndex(DDLSemanticAnalyzer.java:1148)
>   at 
> org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:326)
>   at 
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:208)
>   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:402)
>   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:298)
>   at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:992)
>   at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1062)
>   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:929)
>   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:919)
>   at 
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:246)
>   at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:198)
>   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:408)
>   at 
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:781)
>   at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675)
>   at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8047) Lazy char/varchar are not using escape char defined in serde params

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14132614#comment-14132614
 ] 

Hive QA commented on HIVE-8047:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12668250/HIVE-8047.3.patch

{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 6209 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_metadataonly1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge5
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge6
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge7
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_select_dummy_source
org.apache.hadoop.hive.ql.parse.TestParse.testParse_union
{noformat}

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/775/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/775/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-775/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12668250

> Lazy char/varchar are not using escape char defined in serde params
> ---
>
> Key: HIVE-8047
> URL: https://issues.apache.org/jira/browse/HIVE-8047
> Project: Hive
>  Issue Type: Bug
>  Components: Serializers/Deserializers
>Reporter: Jason Dere
>Assignee: Jason Dere
> Attachments: HIVE-8047.1.patch, HIVE-8047.2.patch, HIVE-8047.3.patch
>
>
> A table with char/varchar columns that specifies an escape character for 
> string data still has the escape characters in the char/varchar columns.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8086) Disable Trivial Project Removal Rule, Fix Result Schema

[Gunther Hagleitner (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14132611#comment-14132611
 ] 

Gunther Hagleitner commented on HIVE-8086:
--

Committed to branch. Thanks [~jpullokkaran]!

> Disable Trivial Project Removal Rule, Fix Result Schema
> ---
>
> Key: HIVE-8086
> URL: https://issues.apache.org/jira/browse/HIVE-8086
> Project: Hive
>  Issue Type: Sub-task
>  Components: CBO
>Reporter: Laljo John Pullokkaran
>Assignee: Laljo John Pullokkaran
> Attachments: HIVE-8086.1.patch, HIVE-8086.patch
>
>
> Disable trivial project rule till Optiq-407 gets fixed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-7946) CBO: Merge CBO changes to Trunk

[Laljo John Pullokkaran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Laljo John Pullokkaran updated HIVE-7946:
-
Status: Patch Available  (was: Open)

> CBO: Merge CBO changes to Trunk
> ---
>
> Key: HIVE-7946
> URL: https://issues.apache.org/jira/browse/HIVE-7946
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Reporter: Laljo John Pullokkaran
>Assignee: Laljo John Pullokkaran
> Attachments: HIVE-7946.1.patch, HIVE-7946.2.patch, HIVE-7946.3.patch, 
> HIVE-7946.4.patch, HIVE-7946.5.patch, HIVE-7946.6.patch, HIVE-7946.7.patch, 
> HIVE-7946.8.patch, HIVE-7946.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-7946) CBO: Merge CBO changes to Trunk

[Laljo John Pullokkaran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Laljo John Pullokkaran updated HIVE-7946:
-
Status: Open  (was: Patch Available)

> CBO: Merge CBO changes to Trunk
> ---
>
> Key: HIVE-7946
> URL: https://issues.apache.org/jira/browse/HIVE-7946
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Reporter: Laljo John Pullokkaran
>Assignee: Laljo John Pullokkaran
> Attachments: HIVE-7946.1.patch, HIVE-7946.2.patch, HIVE-7946.3.patch, 
> HIVE-7946.4.patch, HIVE-7946.5.patch, HIVE-7946.6.patch, HIVE-7946.7.patch, 
> HIVE-7946.8.patch, HIVE-7946.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-7946) CBO: Merge CBO changes to Trunk

[Laljo John Pullokkaran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Laljo John Pullokkaran updated HIVE-7946:
-
Attachment: HIVE-7946.8.patch

> CBO: Merge CBO changes to Trunk
> ---
>
> Key: HIVE-7946
> URL: https://issues.apache.org/jira/browse/HIVE-7946
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Reporter: Laljo John Pullokkaran
>Assignee: Laljo John Pullokkaran
> Attachments: HIVE-7946.1.patch, HIVE-7946.2.patch, HIVE-7946.3.patch, 
> HIVE-7946.4.patch, HIVE-7946.5.patch, HIVE-7946.6.patch, HIVE-7946.7.patch, 
> HIVE-7946.8.patch, HIVE-7946.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8086) Disable Trivial Project Removal Rule, Fix Result Schema

[Laljo John Pullokkaran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Laljo John Pullokkaran updated HIVE-8086:
-
Attachment: HIVE-8086.1.patch

> Disable Trivial Project Removal Rule, Fix Result Schema
> ---
>
> Key: HIVE-8086
> URL: https://issues.apache.org/jira/browse/HIVE-8086
> Project: Hive
>  Issue Type: Sub-task
>  Components: CBO
>Reporter: Laljo John Pullokkaran
>Assignee: Laljo John Pullokkaran
> Attachments: HIVE-8086.1.patch, HIVE-8086.patch
>
>
> Disable trivial project rule till Optiq-407 gets fixed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8086) Disable Trivial Project Removal Rule, Fix Result Schema

[Laljo John Pullokkaran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Laljo John Pullokkaran updated HIVE-8086:
-
Attachment: HIVE-8086.patch

> Disable Trivial Project Removal Rule, Fix Result Schema
> ---
>
> Key: HIVE-8086
> URL: https://issues.apache.org/jira/browse/HIVE-8086
> Project: Hive
>  Issue Type: Sub-task
>  Components: CBO
>Reporter: Laljo John Pullokkaran
>Assignee: Laljo John Pullokkaran
> Attachments: HIVE-8086.patch
>
>
> Disable trivial project rule till Optiq-407 gets fixed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8086) Disable Trivial Project Removal Rule, Fix Result Schema

[Laljo John Pullokkaran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Laljo John Pullokkaran updated HIVE-8086:
-
Status: Patch Available  (was: Open)

> Disable Trivial Project Removal Rule, Fix Result Schema
> ---
>
> Key: HIVE-8086
> URL: https://issues.apache.org/jira/browse/HIVE-8086
> Project: Hive
>  Issue Type: Sub-task
>  Components: CBO
>Reporter: Laljo John Pullokkaran
>Assignee: Laljo John Pullokkaran
> Attachments: HIVE-8086.patch
>
>
> Disable trivial project rule till Optiq-407 gets fixed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8086) Disable Trivial Project Removal Rule, Fix Result Schema

[Laljo John Pullokkaran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Laljo John Pullokkaran updated HIVE-8086:
-
Summary: Disable Trivial Project Removal Rule, Fix Result Schema  (was: 
Disable Trivial Project Removal Rule)

> Disable Trivial Project Removal Rule, Fix Result Schema
> ---
>
> Key: HIVE-8086
> URL: https://issues.apache.org/jira/browse/HIVE-8086
> Project: Hive
>  Issue Type: Sub-task
>  Components: CBO
>Reporter: Laljo John Pullokkaran
>Assignee: Laljo John Pullokkaran
>
> Disable trivial project rule till Optiq-407 gets fixed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8061) improve the partition col stats update speed

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14132597#comment-14132597
 ] 

Hive QA commented on HIVE-8061:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12668424/HIVE-8061.3.patch

{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 6208 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_metadataonly1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge5
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge6
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge7
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_select_dummy_source
{noformat}

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/774/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/774/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-774/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12668424

> improve the partition col stats update speed
> 
>
> Key: HIVE-8061
> URL: https://issues.apache.org/jira/browse/HIVE-8061
> Project: Hive
>  Issue Type: Improvement
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
>Priority: Minor
> Attachments: HIVE-8061.1.patch, HIVE-8061.2.patch, HIVE-8061.3.patch
>
>
> We worked hard towards faster update stats for columns of a partition of a 
> table previously HIVE-7736
> and HIVE-7876
> Although there is some improvement, it is only correct in the first run. 
> There will be duplicate column stats later. Thanks to Eugene Koifman 's 
> comments.
> We fixed this in HIVE-7944 by reversing the patch.
> This JIRA ticket is my another try to improve the speed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)