[jira] [Commented] (HIVE-17679) http-generic-click-jacking for WebHcat server

[Lefty Leverenz (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205033#comment-16205033
 ] 

Lefty Leverenz commented on HIVE-17679:
---

This adds a WebHCat configuration variable, *templeton.frame.options.filter*, 
which is documented in the wiki here:

* [WebHCat Configuration -- Configuration Variables | 
https://cwiki.apache.org/confluence/display/Hive/WebHCat+Configure#WebHCatConfigure-ConfigurationVariables]

Thanks for doing the doc, [~aihuaxu].

> http-generic-click-jacking for WebHcat server
> -
>
> Key: HIVE-17679
> URL: https://issues.apache.org/jira/browse/HIVE-17679
> Project: Hive
>  Issue Type: Bug
>  Components: Security, WebHCat
>Affects Versions: 2.1.1
>Reporter: Aihua Xu
>Assignee: Aihua Xu
> Fix For: 3.0.0
>
> Attachments: HIVE-17679.1.patch, HIVE-17679.2.patch
>
>
> The web UIs do not include the "X-Frame-Options" header to prevent the pages 
> from being framed from another site.
> Reference:
> https://www.owasp.org/index.php/Clickjacking
> https://www.owasp.org/index.php/Clickjacking_Defense_Cheat_Sheet
> https://developer.mozilla.org/en-US/docs/Web/HTTP/X-Frame-Options



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17508) Implement global execution triggers based on counters

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205022#comment-16205022
 ] 

Hive QA commented on HIVE-17508:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12892179/HIVE-17508.12.patch

{color:green}SUCCESS:{color} +1 due to 7 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 17 failed/errored test(s), 11266 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[index_auto_mult_tables] 
(batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[index_skewtable] 
(batchId=79)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[optimize_nullscan]
 (batchId=163)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi] 
(batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_notin] 
(batchId=133)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_scalar] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_select] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_views] 
(batchId=108)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query16] 
(batchId=243)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query94] 
(batchId=243)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query14] 
(batchId=241)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query16] 
(batchId=241)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query23] 
(batchId=241)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query94] 
(batchId=241)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut 
(batchId=204)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighShuffleBytes
 (batchId=229)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighShuffleBytes 
(batchId=229)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7310/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7310/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7310/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 17 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12892179 - PreCommit-HIVE-Build

> Implement global execution triggers based on counters
> -
>
> Key: HIVE-17508
> URL: https://issues.apache.org/jira/browse/HIVE-17508
> Project: Hive
>  Issue Type: Sub-task
>Affects Versions: 3.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-17508.1.patch, HIVE-17508.10.patch, 
> HIVE-17508.11.patch, HIVE-17508.12.patch, HIVE-17508.2.patch, 
> HIVE-17508.3.patch, HIVE-17508.3.patch, HIVE-17508.4.patch, 
> HIVE-17508.5.patch, HIVE-17508.6.patch, HIVE-17508.7.patch, 
> HIVE-17508.8.patch, HIVE-17508.9.patch, HIVE-17508.WIP.2.patch, 
> HIVE-17508.WIP.patch
>
>
> Workload management can defined Triggers that are bound to a resource plan. 
> Each trigger can have a trigger expression and an action associated with it. 
> Trigger expressions are evaluated at runtime after configurable check 
> interval, based on which actions like killing a query, moving a query to 
> different pool etc. will get invoked. Simple execution trigger could be 
> something like
> {code}
> CREATE TRIGGER slow_query IN global
> WHEN execution_time_ms > 1
> MOVE TO slow_queue
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17669) Cache to optimize SearchArgument deserialization

[Lefty Leverenz (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205020#comment-16205020
 ] 

Lefty Leverenz commented on HIVE-17669:
---

The new configuration parameter is documented in the wiki here (thanks, Mithun):

* [hive.io.sarg.cache.max.weight.mb | 
https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.io.sarg.cache.max.weight.mb]

But the fix versions should include 2.3.1.  I've changed that in the wiki and 
made some trivial edits.

> Cache to optimize SearchArgument deserialization
> 
>
> Key: HIVE-17669
> URL: https://issues.apache.org/jira/browse/HIVE-17669
> Project: Hive
>  Issue Type: Improvement
>  Components: ORC, Query Processor
>Affects Versions: 2.2.0, 3.0.0
>Reporter: Mithun Radhakrishnan
>Assignee: Mithun Radhakrishnan
> Fix For: 3.0.0, 2.4.0, 2.2.1
>
> Attachments: HIVE-17669.3.patch, HIVE-17669.4.patch, 
> HIVE-17699.1.patch, HIVE-17699.2.patch
>
>
> And another, from [~selinazh] and [~cdrome]. (YHIVE-927)
> When a mapper needs to process multiple ORC files, it might land up having 
> use essentially the same {{SearchArgument}} over several files. It would be 
> good not to have to deserialize from string, over and over again. Caching the 
> object against the string-form should speed things up.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17214) check/fix conversion of unbucketed non-acid to acid

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205009#comment-16205009
 ] 

Hive QA commented on HIVE-17214:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12892178/HIVE-17214.01.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 11234 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_schema_evol_3a]
 (batchId=145)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[optimize_nullscan]
 (batchId=163)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2] 
(batchId=101)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi] 
(batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_notin] 
(batchId=133)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_scalar] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_select] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_views] 
(batchId=108)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query16] 
(batchId=243)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query94] 
(batchId=243)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query14] 
(batchId=241)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query16] 
(batchId=241)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query94] 
(batchId=241)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut 
(batchId=204)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7309/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7309/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7309/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12892178 - PreCommit-HIVE-Build

> check/fix conversion of unbucketed non-acid to acid
> ---
>
> Key: HIVE-17214
> URL: https://issues.apache.org/jira/browse/HIVE-17214
> Project: Hive
>  Issue Type: Sub-task
>  Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Minor
> Attachments: HIVE-17214.01.patch
>
>
> bucketed tables have stricter rules for file layout on disk - bucket files 
> are direct children of a partition directory.
> for un-bucketed tables I'm not sure there are any rules
> for example, CTAS with Tez + Union operator creates 1 directory for each leg 
> of the union
> Supposedly Hive can read table by picking all files recursively.  
> Can it also write (other than CTAS example above) arbitrarily?
> Does it mean Acid write can also write anywhere?
> Figure out what can be supported and how can existing layout can be checked?  
> Examining a full "ls -l -R" for a large table could be expensive. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HIVE-17792) Enable Bucket Map Join when there are extra keys other than bucketed columns

[Deepak Jaiswal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-17792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deepak Jaiswal updated HIVE-17792:
--
Attachment: HIVE-17792.3.patch

Fixed some more failing tests which were mostly related to SMB.
Fall back to old logic which is used by SMB Join.

> Enable Bucket Map Join when there are extra keys other than bucketed columns
> 
>
> Key: HIVE-17792
> URL: https://issues.apache.org/jira/browse/HIVE-17792
> Project: Hive
>  Issue Type: Bug
>Reporter: Deepak Jaiswal
>Assignee: Deepak Jaiswal
> Attachments: HIVE-17792.1.patch, HIVE-17792.2.patch, 
> HIVE-17792.3.patch
>
>
> Currently this wont go through Bucket Map Join(BMJ)
> CREATE TABLE tab_part (key int, value string) PARTITIONED BY(ds STRING) 
> CLUSTERED BY (key) INTO 4 BUCKETS STORED AS TEXTFILE;
> CREATE TABLE tab(key int, value string) PARTITIONED BY(ds STRING) STORED AS 
> TEXTFILE;
> select a.key, a.value, b.value
> from tab a join tab_part b on a.key = b.key and a.value = b.value;



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HIVE-16601) Display Session Id, Query Name / Id, and Dag Id in Spark UI

[Sahil Takiar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sahil Takiar updated HIVE-16601:

Attachment: HIVE-16601.4.patch

> Display Session Id, Query Name / Id, and Dag Id in Spark UI
> ---
>
> Key: HIVE-16601
> URL: https://issues.apache.org/jira/browse/HIVE-16601
> Project: Hive
>  Issue Type: Bug
>  Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
> Attachments: HIVE-16601.1.patch, HIVE-16601.2.patch, 
> HIVE-16601.3.patch, HIVE-16601.4.patch, Spark UI Applications List.png, Spark 
> UI Jobs List.png
>
>
> We should display the session id for each HoS Application Launched, and the 
> Query Name / Id and Dag Id for each Spark job launched. Hive-on-MR does 
> something similar via the {{mapred.job.name}} parameter. The query name is 
> displayed in the Job Name of the MR app.
> The changes here should also allow us to leverage the config 
> {{hive.query.name}} for HoS.
> This should help with debuggability of HoS applications. The Hive-on-Tez UI 
> does something similar.
> Related issues for Hive-on-Tez: HIVE-12357, HIVE-12523



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-14731) Use Tez cartesian product edge in Hive (unpartitioned case only)

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-14731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204994#comment-16204994
 ] 

Hive QA commented on HIVE-14731:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12892176/HIVE-14731.22.patch

{color:green}SUCCESS:{color} +1 due to 9 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 11236 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1] 
(batchId=145)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi] 
(batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_scalar] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_views] 
(batchId=108)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query16] 
(batchId=243)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query94] 
(batchId=243)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query14] 
(batchId=241)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query16] 
(batchId=241)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query94] 
(batchId=241)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut 
(batchId=204)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7308/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7308/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7308/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12892176 - PreCommit-HIVE-Build

> Use Tez cartesian product edge in Hive (unpartitioned case only)
> 
>
> Key: HIVE-14731
> URL: https://issues.apache.org/jira/browse/HIVE-14731
> Project: Hive
>  Issue Type: Bug
>Reporter: Zhiyuan Yang
>Assignee: Zhiyuan Yang
> Attachments: HIVE-14731.1.patch, HIVE-14731.10.patch, 
> HIVE-14731.11.patch, HIVE-14731.12.patch, HIVE-14731.13.patch, 
> HIVE-14731.14.patch, HIVE-14731.15.patch, HIVE-14731.16.patch, 
> HIVE-14731.17.patch, HIVE-14731.18.patch, HIVE-14731.19.patch, 
> HIVE-14731.2.patch, HIVE-14731.20.patch, HIVE-14731.21.patch, 
> HIVE-14731.22.patch, HIVE-14731.3.patch, HIVE-14731.4.patch, 
> HIVE-14731.5.patch, HIVE-14731.6.patch, HIVE-14731.7.patch, 
> HIVE-14731.8.patch, HIVE-14731.9.patch
>
>
> Given cartesian product edge is available in Tez now (see TEZ-3230), let's 
> integrate it into Hive on Tez. This allows us to have more than one reducer 
> in cross product queries.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17806) Create directory for metrics file if it doesn't exist

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204981#comment-16204981
 ] 

Hive QA commented on HIVE-17806:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12892172/HIVE-17806.01.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 13 failed/errored test(s), 11233 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[optimize_nullscan]
 (batchId=163)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi] 
(batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_notin] 
(batchId=133)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_scalar] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_select] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_views] 
(batchId=108)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query16] 
(batchId=243)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query39] 
(batchId=243)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query94] 
(batchId=243)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query14] 
(batchId=241)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query16] 
(batchId=241)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query94] 
(batchId=241)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut 
(batchId=204)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7307/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7307/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7307/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 13 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12892172 - PreCommit-HIVE-Build

> Create directory for metrics file if it doesn't exist
> -
>
> Key: HIVE-17806
> URL: https://issues.apache.org/jira/browse/HIVE-17806
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Affects Versions: 3.0.0
>Reporter: Alexander Kolbasov
>Assignee: Alexander Kolbasov
> Attachments: HIVE-17806.01.patch
>
>
> HIVE-17563 changed metrics code to use local file system operations instead 
> of Hadoop local file system operations. There is an unintended side effect - 
> hadoop file systems create the directory if it doesn't exist and java nio 
> interfaces don't. The purpose of this fix is to revert the behavior to the 
> original one to avoid surprises.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-16688) Make sure Alter Table to set transaction=true acquires X lock

[Eugene Koifman (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204881#comment-16204881
 ] 

Eugene Koifman commented on HIVE-16688:
---

todo: run TestCliDriver.testCliDriver[update_all_non_partitioned]
other failures have age > 1

> Make sure Alter Table to set transaction=true acquires X lock
> -
>
> Key: HIVE-16688
> URL: https://issues.apache.org/jira/browse/HIVE-16688
> Project: Hive
>  Issue Type: Sub-task
>  Components: Transactions
>Affects Versions: 1.3.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Critical
> Attachments: HIVE-16688.01.patch, HIVE-16688.02.patch
>
>
> suppose we have non-acid table with some data
> An insert op starts (long running)  (with hive.txn.strict.locking.mode=false 
> this takes shared lock)
> An alter table runs to add (transactional=true)
> An update is run which will read the list of "original" files and assign IDs 
> on the fly which are written to a delta file.
> The long running insert completes.
> Another update is run which now sees a different set of "original" files and 
> will (most likely) assign different IDs.
> Need to make sure to mutex this



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17692) Block HCat on Acid tables

[Eugene Koifman (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204879#comment-16204879
 ] 

Eugene Koifman commented on HIVE-17692:
---

all failures have age > 9
[~sershe] could you review please


> Block HCat on Acid tables
> -
>
> Key: HIVE-17692
> URL: https://issues.apache.org/jira/browse/HIVE-17692
> Project: Hive
>  Issue Type: Sub-task
>  Components: Transactions
>Affects Versions: 1.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Critical
> Attachments: HIVE-17692.01.patch, HIVE-17692.02.patch
>
>
> See _DDLSemanticAnalzyer.analyzeAlterTablePartMergeFiles(ASTNode ast, String 
> tableName, HashMap partSpec)_
> This was fine before due to 
> {noformat}
>   // throw a HiveException if the table/partition is bucketized
>   if (bucketCols != null && bucketCols.size() > 0) {
> throw new 
> SemanticException(ErrorMsg.CONCATENATE_UNSUPPORTED_TABLE_BUCKETED.getMsg());
>   }
> {noformat}
> but now that we support unbucketed acid tables



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-16722) Converting bucketed non-acid table to acid should perform validation

[Eugene Koifman (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204877#comment-16204877
 ] 

Eugene Koifman commented on HIVE-16722:
---

TestEmbeddedHiveMetaStore etc have 
{noformat}
java.lang.IllegalArgumentException: Wrong FS: 
pfile:/home/hiveptest/104.197.70.162-hiveptest-0/apache-github-source-source/itests/hive-unit/target/warehouse/aciddb.db/acidtable,
 expected: file:///
{noformat}


> Converting bucketed non-acid table to acid should perform validation
> 
>
> Key: HIVE-16722
> URL: https://issues.apache.org/jira/browse/HIVE-16722
> Project: Hive
>  Issue Type: Sub-task
>  Components: Transactions
>Affects Versions: 1.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
> Attachments: HIVE-16722.01.patch, HIVE-16722.WIP.patch
>
>
> Converting a non acid table to acid only performs metadata validation (in 
> _TransactionalValidationListener_).
> The data read code path only understands certain directory layouts and file 
> names and ignores (generally) files that don't match the expected format.
> In Hive, directory layout and bucket file naming (especially older releases) 
> is poorly enforced.
> Need to add a validation step on 
> {noformat}
> alter table T SET TBLPROPERTIES ('transactional'='true')
> {noformat}
> to 
> scan the file system and report any possible data loss scenarios.
> Currently Acid understands bucket files name like "0_0" and (with 
> HIVE-16177) 0_0_copy1" etc at the root of the partition.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17730) Queries can be closed automatically

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204876#comment-16204876
 ] 

Hive QA commented on HIVE-17730:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12892170/HIVE-17730.05.patch

{color:red}ERROR:{color} -1 due to build exiting with an error

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7306/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7306/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7306/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit 
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2017-10-15 00:35:48.657
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export 
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ 
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-7306/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2017-10-15 00:35:48.660
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 133d3c4 HIVE-17733 Move RawStore to standalone metastore. This 
closes #258 github PR. (Alan Gates, reviewed by Sergey Shelukhin, Vihang 
Karajgaonkar, and Zoltan Haindrich)
+ git clean -f -d
Removing ql/src/test/queries/clientpositive/vector_like_2.q
Removing ql/src/test/results/clientpositive/vector_like_2.q.out
Removing standalone-metastore/src/gen/org/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 133d3c4 HIVE-17733 Move RawStore to standalone metastore. This 
closes #258 github PR. (Alan Gates, reviewed by Sergey Shelukhin, Vihang 
Karajgaonkar, and Zoltan Haindrich)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2017-10-15 00:35:52.017
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh 
/data/hiveptest/working/scratch/build.patch
error: a/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java: 
No such file or directory
error: 
a/metastore/src/java/org/apache/hadoop/hive/metastore/tools/HiveMetaTool.java: 
No such file or directory
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12892170 - PreCommit-HIVE-Build

> Queries can be closed automatically
> ---
>
> Key: HIVE-17730
> URL: https://issues.apache.org/jira/browse/HIVE-17730
> Project: Hive
>  Issue Type: Bug
>Reporter: Alexander Kolbasov
>Assignee: Alexander Kolbasov
> Attachments: HIVE-17730.05.patch
>
>
> HIVE-16213 made QueryWrapper AutoCloseable, but queries are still closed 
> manually and not by using try-with-resource. And now Query itself is auto 
> closeable, so we don't need the wrapper at all.
> So we should get rid of QueryWrapper and use try-with-resource to create 
> queries.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17804) Vectorization: Bug erroneously causes match for 1st row in batch (SelectStringColLikeStringScalar)

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204875#comment-16204875
 ] 

Hive QA commented on HIVE-17804:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12892171/HIVE-17804.01.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 11219 tests 
executed
*Failed tests:*
{noformat}
TestMiniLlapCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=145)

[intersect_all.q,unionDistinct_1.q,orc_ppd_schema_evol_3a.q,table_nonprintable.q,tez_union_dynamic_partition.q,tez_union_dynamic_partition_2.q,temp_table_external.q,global_limit.q,llap_udf.q,schemeAuthority.q,cte_2.q,rcfile_createas1.q,dynamic_partition_pruning_2.q,intersect_merge.q,parallel_colstats.q]
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_into_dynamic_partitions]
 (batchId=244)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[index_compact_3] 
(batchId=73)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[optimize_nullscan]
 (batchId=163)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi] 
(batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_notin] 
(batchId=133)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_scalar] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_select] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_views] 
(batchId=108)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query16] 
(batchId=243)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query94] 
(batchId=243)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query14] 
(batchId=241)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query16] 
(batchId=241)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query94] 
(batchId=241)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut 
(batchId=204)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7305/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7305/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7305/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 15 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12892171 - PreCommit-HIVE-Build

> Vectorization: Bug erroneously causes match for 1st row in batch 
> (SelectStringColLikeStringScalar)
> --
>
> Key: HIVE-17804
> URL: https://issues.apache.org/jira/browse/HIVE-17804
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HIVE-17804.01.patch
>
>
> Code setting output value to LongColumnVector.NULL_VALUE for null candidate 
> sets the 0th entry instead of the i'th.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17807) Execute maven commands in batch mode for ptests

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204850#comment-16204850
 ] 

Hive QA commented on HIVE-17807:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12892169/HIVE-17807.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 13 failed/errored test(s), 11233 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[optimize_nullscan]
 (batchId=163)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi] 
(batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_notin] 
(batchId=133)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_scalar] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_select] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_views] 
(batchId=108)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query16] 
(batchId=243)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query94] 
(batchId=243)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query14] 
(batchId=241)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query16] 
(batchId=241)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query23] 
(batchId=241)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query94] 
(batchId=241)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut 
(batchId=204)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7304/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7304/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7304/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 13 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12892169 - PreCommit-HIVE-Build

> Execute maven commands in batch mode for ptests
> ---
>
> Key: HIVE-17807
> URL: https://issues.apache.org/jira/browse/HIVE-17807
> Project: Hive
>  Issue Type: Bug
>Reporter: Vijay Kumar
>Assignee: Vijay Kumar
> Attachments: HIVE-17807.patch
>
>
> No need to run in interactive mode in CI environment.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17760) Create a unit test which validates HIVE-9423 does not regress

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204839#comment-16204839
 ] 

Hive QA commented on HIVE-17760:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12892165/HIVE-17760.3.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 11234 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[optimize_nullscan]
 (batchId=163)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning]
 (batchId=171)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi] 
(batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_notin] 
(batchId=133)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_scalar] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_select] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_views] 
(batchId=108)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query16] 
(batchId=243)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query94] 
(batchId=243)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query14] 
(batchId=241)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query16] 
(batchId=241)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query94] 
(batchId=241)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut 
(batchId=204)
org.apache.hive.beeline.TestBeelinePasswordOption.testMultiConnect (batchId=224)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7303/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7303/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7303/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12892165 - PreCommit-HIVE-Build

> Create a unit test which validates HIVE-9423 does not regress 
> --
>
> Key: HIVE-17760
> URL: https://issues.apache.org/jira/browse/HIVE-17760
> Project: Hive
>  Issue Type: Bug
>Reporter: Andrew Sherman
>Assignee: Andrew Sherman
> Attachments: HIVE-17760.1.patch, HIVE-17760.2.patch, 
> HIVE-17760.3.patch
>
>
> During [HIVE-9423] we verified that when the Thrift server pool is exhausted, 
> then Beeline connection times out, and provide a meaningful error message.
> Create a unit test which verifies this, and helps to keep this feature working



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17805) SchemaTool validate locations should not return exit 1

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204828#comment-16204828
 ] 

Hive QA commented on HIVE-17805:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12892164/HIVE-17805.01.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 13 failed/errored test(s), 11233 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[optimize_nullscan]
 (batchId=163)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2] 
(batchId=101)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi] 
(batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_notin] 
(batchId=133)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_scalar] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_select] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_views] 
(batchId=108)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query16] 
(batchId=243)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query94] 
(batchId=243)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query14] 
(batchId=241)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query16] 
(batchId=241)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query94] 
(batchId=241)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut 
(batchId=204)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7302/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7302/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7302/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 13 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12892164 - PreCommit-HIVE-Build

> SchemaTool validate locations should not return exit 1
> --
>
> Key: HIVE-17805
> URL: https://issues.apache.org/jira/browse/HIVE-17805
> Project: Hive
>  Issue Type: Improvement
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
> Attachments: HIVE-17805.01.patch
>
>
> {{HiveSchemaTool}} can be used by other applications to validate the 
> metastore schema. One of the validation check looks at the location URLs of 
> tables/DBs and returns {{false}} which causes HiveSchemaTool to exit 1 to the 
> calling script. Invalid locations although are a problem in some instances, 
> cannot be termed as catastrophic errors in the schema which should cause Hive 
> service failure/unusable. Ideally we should introduce warning levels and 
> error levels in schemaTool validations so the caller can take appropriate 
> action.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17803) With Pig multi-query, 2 HCatStorers writing to the same table will trample each other's outputs

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204800#comment-16204800
 ] 

Hive QA commented on HIVE-17803:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12892162/HIVE-17803.1.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 13 failed/errored test(s), 11233 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[optimize_nullscan]
 (batchId=163)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi] 
(batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_notin] 
(batchId=133)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_scalar] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_select] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_views] 
(batchId=108)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query16] 
(batchId=243)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query94] 
(batchId=243)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query14] 
(batchId=241)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query16] 
(batchId=241)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query23] 
(batchId=241)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query94] 
(batchId=241)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut 
(batchId=204)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7301/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7301/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7301/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 13 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12892162 - PreCommit-HIVE-Build

> With Pig multi-query, 2 HCatStorers writing to the same table will trample 
> each other's outputs
> ---
>
> Key: HIVE-17803
> URL: https://issues.apache.org/jira/browse/HIVE-17803
> Project: Hive
>  Issue Type: Bug
>  Components: HCatalog
>Affects Versions: 2.2.0, 3.0.0
>Reporter: Mithun Radhakrishnan
>Assignee: Chris Drome
> Attachments: HIVE-17803.1.patch
>
>
> When Pig scripts use multi-query and {{HCatStorer}} with 
> dynamic-partitioning, and use more than one {{HCatStorer}} instance to write 
> to the same table, they might trample on each other's outputs. The failure 
> looks as follows:
> {noformat}
> Caused by: org.apache.hive.hcatalog.common.HCatException : 2006 : Error 
> adding partition to metastore. Cause : 
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException):
>  No lease on /projects/foo/bar/activity_date=2016022306/_placeholder (inode 
> 2878224200): File does not exist. [Lease.  Holder: 
> DFSClient_NONMAPREDUCE_-1281544466_4952, pendingcreates: 1]
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:3429)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFileInternal(FSNamesystem.java:3517)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFile(FSNamesystem.java:3484)
>   at 
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.complete(NameNodeRpcServer.java:791)
>   at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.complete(ClientNamenodeProtocolServerSideTranslatorPB.java:537)
>   at 
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
>   at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:608)
>   at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
>   at org.apache.hadoop.ipc.Server.call(Server.java:2267)
>   at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:648)
>   at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:615)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:422)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1679)
>   at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2217)
>   

[jira] [Updated] (HIVE-17812) Move remaining classes that HiveMetaStore depends on

[Alan Gates (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-17812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Gates updated HIVE-17812:
--
Hadoop Flags: Incompatible change

> Move remaining classes that HiveMetaStore depends on 
> -
>
> Key: HIVE-17812
> URL: https://issues.apache.org/jira/browse/HIVE-17812
> Project: Hive
>  Issue Type: Sub-task
>  Components: Metastore
>Reporter: Alan Gates
>Assignee: Alan Gates
>  Labels: pull-request-available
> Attachments: HIVE-17812.patch
>
>
> There are several remaining pieces that need moved before we can move 
> HiveMetaStore itself.  These include NotificationListener and 
> implementations, Events, AlterHandler, and a few other miscellaneous pieces.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HIVE-17812) Move remaining classes that HiveMetaStore depends on

[Alan Gates (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-17812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Gates updated HIVE-17812:
--
Attachment: HIVE-17812.patch

This change does break compatibility for NotificationListeners and Events.  It 
does this by switching the reference in EventContext (and similar classes) from 
an HMSHandler to IHMSHandler (the interface).  This makes it easier to move the 
pieces separately, and is also a cleaner design since it makes more sense for 
potentially external pieces like listeners to depend on interfaces rather than 
implementations.

Is this avoidable?  Yes, I could move events and listeners in a bigger patch 
along with HiveMetaStore.  However, other changes are going to break listeners 
and events anyway.  Namely, the change from HiveConf -> Conf (which is not 
avoidable).  Also, if we do split the metastore into a separate TLP in the 
future it will change class names, which will also obviously impact 
implementations of listener.

We should look into what it will take to build a shim that would support 
existing listeners.  This would need to live in the metastore module rather 
than standalone-metastore, since it will need to reference HiveConf.

> Move remaining classes that HiveMetaStore depends on 
> -
>
> Key: HIVE-17812
> URL: https://issues.apache.org/jira/browse/HIVE-17812
> Project: Hive
>  Issue Type: Sub-task
>  Components: Metastore
>Reporter: Alan Gates
>Assignee: Alan Gates
>  Labels: pull-request-available
> Attachments: HIVE-17812.patch
>
>
> There are several remaining pieces that need moved before we can move 
> HiveMetaStore itself.  These include NotificationListener and 
> implementations, Events, AlterHandler, and a few other miscellaneous pieces.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HIVE-17812) Move remaining classes that HiveMetaStore depends on

[Alan Gates (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-17812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Gates updated HIVE-17812:
--
Status: Patch Available  (was: Open)

> Move remaining classes that HiveMetaStore depends on 
> -
>
> Key: HIVE-17812
> URL: https://issues.apache.org/jira/browse/HIVE-17812
> Project: Hive
>  Issue Type: Sub-task
>  Components: Metastore
>Reporter: Alan Gates
>Assignee: Alan Gates
>  Labels: pull-request-available
> Attachments: HIVE-17812.patch
>
>
> There are several remaining pieces that need moved before we can move 
> HiveMetaStore itself.  These include NotificationListener and 
> implementations, Events, AlterHandler, and a few other miscellaneous pieces.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17812) Move remaining classes that HiveMetaStore depends on

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204785#comment-16204785
 ] 

ASF GitHub Bot commented on HIVE-17812:
---

GitHub user alanfgates opened a pull request:

https://github.com/apache/hive/pull/261

HIVE-17812



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/alanfgates/hive hive17812

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hive/pull/261.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #261


commit 15ce382059ee985ece8621bb9849776184d4805b
Author: Alan Gates 
Date:   2017-08-28T21:18:53Z

Moved EventMessage and friends.

commit eb6e31f25d3af16e8d1f9ebe88f16e25bcb56eeb
Author: Alan Gates 
Date:   2017-08-28T21:39:27Z

Moved JSON message implementations.

commit 8150c599e49373b0d3e7d55423b653c2d1757fc9
Author: Alan Gates 
Date:   2017-08-28T23:56:09Z

Moved RetryingHMSHandler

commit ab945374b3ddb9fdd1e20c06c99c8154434295db
Author: Alan Gates 
Date:   2017-08-30T17:35:31Z

Moved events.  As a part of this I had to change events and listeners to 
take IHMSHandler interface rather than a direct reference to HMSHandler.  I 
believe this is an improvement as it gives a clean separation between the 
interface and the implementation.  But it will break existing listeners.  As 
part of this I had to add a few methods to IHMSHandler.  We may need to 
continue to add methods here to support existing external listeners.

commit fdde7842526d0ba7871333de72c5e38e3a2c2a58
Author: Alan Gates 
Date:   2017-08-31T18:00:44Z

Moved the listeners.  Also moved one event that either I missed or that 
snuck in after I moved the others.

commit 5cb1950fc2f91dc412e897c952e076348248d2f6
Author: Alan Gates 
Date:   2017-08-31T20:02:13Z

Moved AlterHandler and HiveAlterHandler.  Also did a partial cleanup of 
MetaStoreUtils.  Need to do more.

commit 7a7dde55e60a0b09db22becd7f839c9eaa25d0bd
Author: Alan Gates 
Date:   2017-09-01T19:11:50Z

Converted extenders of HouseKeeperService to instead implement 
RunnableConfigurable and use the general ThreadPool.

commit 5506814216dc212b0e79eefc458ec841dc2ca123
Author: Alan Gates 
Date:   2017-09-01T22:03:42Z

Moved FileMetadataManager.

commit 446be8e95a0399185ea9d23362bd878d325d0f63
Author: Alan Gates 
Date:   2017-10-14T19:02:27Z

Fixes after the rebase.  Also fixed acid house keeping threads to only open 
txn handler once instead of on every run.




> Move remaining classes that HiveMetaStore depends on 
> -
>
> Key: HIVE-17812
> URL: https://issues.apache.org/jira/browse/HIVE-17812
> Project: Hive
>  Issue Type: Sub-task
>  Components: Metastore
>Reporter: Alan Gates
>Assignee: Alan Gates
>  Labels: pull-request-available
>
> There are several remaining pieces that need moved before we can move 
> HiveMetaStore itself.  These include NotificationListener and 
> implementations, Events, AlterHandler, and a few other miscellaneous pieces.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HIVE-17812) Move remaining classes that HiveMetaStore depends on

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-17812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated HIVE-17812:
--
Labels: pull-request-available  (was: )

> Move remaining classes that HiveMetaStore depends on 
> -
>
> Key: HIVE-17812
> URL: https://issues.apache.org/jira/browse/HIVE-17812
> Project: Hive
>  Issue Type: Sub-task
>  Components: Metastore
>Reporter: Alan Gates
>Assignee: Alan Gates
>  Labels: pull-request-available
>
> There are several remaining pieces that need moved before we can move 
> HiveMetaStore itself.  These include NotificationListener and 
> implementations, Events, AlterHandler, and a few other miscellaneous pieces.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-16722) Converting bucketed non-acid table to acid should perform validation

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204784#comment-16204784
 ] 

Hive QA commented on HIVE-16722:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12892134/HIVE-16722.01.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 20 failed/errored test(s), 11233 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[llap_acid] (batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mm_conversions] 
(batchId=73)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_ppd_decimal] 
(batchId=9)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1] 
(batchId=145)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[optimize_nullscan]
 (batchId=163)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi] 
(batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_notin] 
(batchId=133)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_scalar] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_select] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_views] 
(batchId=108)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query16] 
(batchId=243)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query94] 
(batchId=243)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query16] 
(batchId=241)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query94] 
(batchId=241)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut 
(batchId=204)
org.apache.hadoop.hive.metastore.TestEmbeddedHiveMetaStore.testTransactionalValidation
 (batchId=209)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testTransactionalValidation
 (batchId=211)
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testTransactionalValidation
 (batchId=208)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testTransactionalValidation
 (batchId=206)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testTransactionalValidation
 (batchId=216)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7300/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7300/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7300/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 20 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12892134 - PreCommit-HIVE-Build

> Converting bucketed non-acid table to acid should perform validation
> 
>
> Key: HIVE-16722
> URL: https://issues.apache.org/jira/browse/HIVE-16722
> Project: Hive
>  Issue Type: Sub-task
>  Components: Transactions
>Affects Versions: 1.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
> Attachments: HIVE-16722.01.patch, HIVE-16722.WIP.patch
>
>
> Converting a non acid table to acid only performs metadata validation (in 
> _TransactionalValidationListener_).
> The data read code path only understands certain directory layouts and file 
> names and ignores (generally) files that don't match the expected format.
> In Hive, directory layout and bucket file naming (especially older releases) 
> is poorly enforced.
> Need to add a validation step on 
> {noformat}
> alter table T SET TBLPROPERTIES ('transactional'='true')
> {noformat}
> to 
> scan the file system and report any possible data loss scenarios.
> Currently Acid understands bucket files name like "0_0" and (with 
> HIVE-16177) 0_0_copy1" etc at the root of the partition.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (HIVE-17812) Move remaining classes that HiveMetaStore depends on

[Alan Gates (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-17812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Gates reassigned HIVE-17812:
-


> Move remaining classes that HiveMetaStore depends on 
> -
>
> Key: HIVE-17812
> URL: https://issues.apache.org/jira/browse/HIVE-17812
> Project: Hive
>  Issue Type: Sub-task
>  Components: Metastore
>Reporter: Alan Gates
>Assignee: Alan Gates
>
> There are several remaining pieces that need moved before we can move 
> HiveMetaStore itself.  These include NotificationListener and 
> implementations, Events, AlterHandler, and a few other miscellaneous pieces.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17765) expose Hive keywords

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204762#comment-16204762
 ] 

Hive QA commented on HIVE-17765:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12892133/HIVE-17765.02.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 11233 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[optimize_nullscan]
 (batchId=163)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2] 
(batchId=101)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi] 
(batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_notin] 
(batchId=133)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_scalar] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_select] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_views] 
(batchId=108)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query16] 
(batchId=243)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query94] 
(batchId=243)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query14] 
(batchId=241)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query16] 
(batchId=241)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query23] 
(batchId=241)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query94] 
(batchId=241)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut 
(batchId=204)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7299/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7299/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7299/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12892133 - PreCommit-HIVE-Build

> expose Hive keywords 
> -
>
> Key: HIVE-17765
> URL: https://issues.apache.org/jira/browse/HIVE-17765
> Project: Hive
>  Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-17765.01.patch, HIVE-17765.02.patch, 
> HIVE-17765.nogen.patch, HIVE-17765.patch
>
>
> This could be useful e.g. for BI tools (via ODBC/JDBC drivers) to decide on 
> SQL capabilities of Hive



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17371) Move tokenstores to metastore module

[Alan Gates (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204758#comment-16204758
 ] 

Alan Gates commented on HIVE-17371:
---

Nothing more from me.

> Move tokenstores to metastore module
> 
>
> Key: HIVE-17371
> URL: https://issues.apache.org/jira/browse/HIVE-17371
> Project: Hive
>  Issue Type: Sub-task
>  Components: Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
> Attachments: HIVE-17371.01.patch, HIVE-17371.02.patch, 
> HIVE-17371.03.patch, HIVE-17371.04.patch
>
>
> The {{getTokenStore}} method will not work for the {{DBTokenStore}} and 
> {{ZKTokenStore}} since they implement 
> {{org.apache.hadoop.hive.thrift.DelegationTokenStore}} instead of  
> {{org.apache.hadoop.hive.metastore.security.DelegationTokenStore}}
> {code}
> private DelegationTokenStore getTokenStore(Configuration conf) throws 
> IOException {
> String tokenStoreClassName =
> MetastoreConf.getVar(conf, 
> MetastoreConf.ConfVars.DELEGATION_TOKEN_STORE_CLS, "");
> // The second half of this if is to catch cases where users are passing 
> in a HiveConf for
> // configuration.  It will have set the default value of
> // "hive.cluster.delegation.token.store .class" to
> // "org.apache.hadoop.hive.thrift.MemoryTokenStore" as part of its 
> construction.  But this is
> // the hive-shims version of the memory store.  We want to convert this 
> to our default value.
> if (StringUtils.isBlank(tokenStoreClassName) ||
> 
> "org.apache.hadoop.hive.thrift.MemoryTokenStore".equals(tokenStoreClassName)) 
> {
>   return new MemoryTokenStore();
> }
> try {
>   Class storeClass =
>   
> Class.forName(tokenStoreClassName).asSubclass(DelegationTokenStore.class);
>   return ReflectionUtils.newInstance(storeClass, conf);
> } catch (ClassNotFoundException e) {
>   throw new IOException("Error initializing delegation token store: " + 
> tokenStoreClassName, e);
> }
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17792) Enable Bucket Map Join when there are extra keys other than bucketed columns

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204727#comment-16204727
 ] 

Hive QA commented on HIVE-17792:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12892130/HIVE-17792.2.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 54 failed/errored test(s), 11233 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_8] 
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_partlvl] 
(batchId=34)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_partlvl_dp] 
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[constant_prop_2] 
(batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_duplicate_key] 
(batchId=6)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_grouping_id3] 
(batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[order3] (batchId=62)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[quotedid_basic] 
(batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[semijoin2] (batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_in_explain_rewrite]
 (batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_windowing_gby2] 
(batchId=64)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_windowing_gby] 
(batchId=76)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[windowing_gby2] 
(batchId=34)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1] 
(batchId=145)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[autoColumnStats_1]
 (batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[autoColumnStats_2]
 (batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[columnStatsUpdateForStatsOptimizer_1]
 (batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[column_table_stats]
 (batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[column_table_stats_orc]
 (batchId=148)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[columnstats_part_coltype]
 (batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[drop_partition_with_stats]
 (batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynpart_sort_optimization_acid]
 (batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[extrapolate_part_stats_partial_ndv]
 (batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[metadata_only_queries]
 (batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[metadata_only_queries_with_filters]
 (batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[optimize_nullscan]
 (batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[semijoin] 
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[special_character_in_tabnames_1]
 (batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[stats_only_null]
 (batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[table_access_keys_stats]
 (batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_join] 
(batchId=155)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_main]
 (batchId=153)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_groupby_grouping_id3]
 (batchId=155)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_leftsemi_mapjoin]
 (batchId=149)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[windowing_gby]
 (batchId=156)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2] 
(batchId=101)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[bucket_map_join_tez1]
 (batchId=139)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[metadata_only_queries]
 (batchId=116)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[metadata_only_queries_with_filters]
 (batchId=132)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[semijoin] 
(batchId=112)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[stats_only_null] 
(batchId=114)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi] 
(batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_notin] 
(batchId=133)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_scalar] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_select] 
(batchId=119)

[jira] [Commented] (HIVE-17692) Block HCat on Acid tables

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204681#comment-16204681
 ] 

Hive QA commented on HIVE-17692:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12892124/HIVE-17692.02.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 12 failed/errored test(s), 11233 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[optimize_nullscan]
 (batchId=163)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi] 
(batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_notin] 
(batchId=133)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_scalar] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_select] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_views] 
(batchId=108)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query16] 
(batchId=242)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query94] 
(batchId=242)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query14] 
(batchId=240)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query16] 
(batchId=240)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query94] 
(batchId=240)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut 
(batchId=203)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7297/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7297/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7297/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 12 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12892124 - PreCommit-HIVE-Build

> Block HCat on Acid tables
> -
>
> Key: HIVE-17692
> URL: https://issues.apache.org/jira/browse/HIVE-17692
> Project: Hive
>  Issue Type: Sub-task
>  Components: Transactions
>Affects Versions: 1.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Critical
> Attachments: HIVE-17692.01.patch, HIVE-17692.02.patch
>
>
> See _DDLSemanticAnalzyer.analyzeAlterTablePartMergeFiles(ASTNode ast, String 
> tableName, HashMap partSpec)_
> This was fine before due to 
> {noformat}
>   // throw a HiveException if the table/partition is bucketized
>   if (bucketCols != null && bucketCols.size() > 0) {
> throw new 
> SemanticException(ErrorMsg.CONCATENATE_UNSUPPORTED_TABLE_BUCKETED.getMsg());
>   }
> {noformat}
> but now that we support unbucketed acid tables



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-15016) Run tests with Hadoop 3.0.0-beta1

[Aihua Xu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204669#comment-16204669
 ] 

Aihua Xu commented on HIVE-15016:
-

[~ashutoshc] Just found out tez 0.9 is not compatible with hadoop 3.0 since 
hadoop made the change from {{org.apache.log4j.LevelDEFAULT_LOG_LEVEL}} 
to {{String DEFAULT_LOG_LEVEL}}. Tez needs to recompile against hadoop 3.0.

> Run tests with Hadoop 3.0.0-beta1
> -
>
> Key: HIVE-15016
> URL: https://issues.apache.org/jira/browse/HIVE-15016
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Sergio Peña
>Assignee: Aihua Xu
> Attachments: HIVE-15016.2.patch, HIVE-15016.3.patch, 
> HIVE-15016.4.patch, HIVE-15016.5.patch, HIVE-15016.patch, 
> Hadoop3Upstream.patch
>
>
> Hadoop 3.0.0-alpha1 was released back on Sep/16 to allow other components run 
> tests against this new version before GA.
> We should start running tests with Hive to validate compatibility against 
> Hadoop 3.0.
> NOTE: The patch used to test must not be committed to Hive until Hadoop 3.0 
> GA is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HIVE-17733) Move RawStore to standalone metastore

[Alan Gates (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-17733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Gates updated HIVE-17733:
--
   Resolution: Fixed
Fix Version/s: 3.0.0
   Status: Resolved  (was: Patch Available)

Patch 6 committed.  Thanks Sergey, Zoltan, and Vihang for the reviews and 
feedback.

> Move RawStore to standalone metastore
> -
>
> Key: HIVE-17733
> URL: https://issues.apache.org/jira/browse/HIVE-17733
> Project: Hive
>  Issue Type: Sub-task
>  Components: Metastore
>Reporter: Alan Gates
>Assignee: Alan Gates
>  Labels: pull-request-available
> Fix For: 3.0.0
>
> Attachments: HIVE-17733.2.patch, HIVE-17733.3.patch, 
> HIVE-17733.4.patch, HIVE-17733.5.patch, HIVE-17733.6.patch, HIVE-17733.patch
>
>
> This includes moving implementations of RawStore (like ObjectStore), 
> MetastoreDirectSql, and stats related classes like ColumnStatsAggregator and 
> the NDV classes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17733) Move RawStore to standalone metastore

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204651#comment-16204651
 ] 

ASF GitHub Bot commented on HIVE-17733:
---

Github user asfgit closed the pull request at:

https://github.com/apache/hive/pull/258


> Move RawStore to standalone metastore
> -
>
> Key: HIVE-17733
> URL: https://issues.apache.org/jira/browse/HIVE-17733
> Project: Hive
>  Issue Type: Sub-task
>  Components: Metastore
>Reporter: Alan Gates
>Assignee: Alan Gates
>  Labels: pull-request-available
> Attachments: HIVE-17733.2.patch, HIVE-17733.3.patch, 
> HIVE-17733.4.patch, HIVE-17733.5.patch, HIVE-17733.6.patch, HIVE-17733.patch
>
>
> This includes moving implementations of RawStore (like ObjectStore), 
> MetastoreDirectSql, and stats related classes like ColumnStatsAggregator and 
> the NDV classes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17635) Add unit tests to CompactionTxnHandler and use PreparedStatements for queries

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204647#comment-16204647
 ] 

Hive QA commented on HIVE-17635:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12892117/HIVE-17635.2.patch

{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 11234 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[optimize_nullscan]
 (batchId=163)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi] 
(batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_notin] 
(batchId=133)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_scalar] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_select] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_views] 
(batchId=108)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query16] 
(batchId=242)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query94] 
(batchId=242)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query14] 
(batchId=240)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query16] 
(batchId=240)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query23] 
(batchId=240)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query94] 
(batchId=240)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut 
(batchId=203)
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testHttpRetryOnServerIdleTimeout 
(batchId=229)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7296/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7296/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7296/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12892117 - PreCommit-HIVE-Build

> Add unit tests to CompactionTxnHandler and use PreparedStatements for queries
> -
>
> Key: HIVE-17635
> URL: https://issues.apache.org/jira/browse/HIVE-17635
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Andrew Sherman
>Assignee: Andrew Sherman
> Attachments: HIVE-17635.1.patch, HIVE-17635.2.patch
>
>
> It is better for jdbc code that runs against the HMS database to use 
> PreparedStatements. Convert CompactionTxnHandler queries to use 
> PreparedStatement and add tests to TestCompactionTxnHandler to test these 
> queries, and improve code coverage.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (HIVE-16828) With CBO enabled, Query on partitioned views throws IndexOutOfBoundException

[Sanjay Mishra (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204643#comment-16204643
 ] 

Sanjay Mishra edited comment on HIVE-16828 at 10/14/17 2:21 PM:


Facing the same issue in Hive 2.1.0. 
The issue is with all types of views. Tried this:
Authorization is enabled.
Create table test_table(id string);
Create view test_view as select * from test_table;
Select * from test_view; -> Throws the mentioned error.

Is there a plan to release the fix for the same anytime soon.?


was (Author: sanjurm16):
Facing the same issue in Hive 2.1.0. 
The issue is with all types of views. Tried this:
Authorization is enabled.
Create table test_table(id string);
Create view test_view as select * from test_table;
Select * from test_view; -> Throws the mentioned error.

Is there are plan to release the fix for the same anytime soon.?

> With CBO enabled, Query on partitioned views throws IndexOutOfBoundException
> 
>
> Key: HIVE-16828
> URL: https://issues.apache.org/jira/browse/HIVE-16828
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Affects Versions: 1.2.0, 2.1.1, 2.2.0
>Reporter: Adesh Kumar Rao
>Assignee: Vineet Garg
> Attachments: HIVE-16828.patch
>
>
> {code:java}
> Caused by: java.lang.AssertionError: Internal error: While invoking method 
> 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult 
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Filter,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'
> at org.apache.calcite.util.Util.newInternal(Util.java:789)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:534)
> at 
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:270)
> at 
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:213)
> at 
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:374)
> at 
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(HiveRelFieldTrimmer.java:394)
> ... 98 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531)
> ... 102 more
> Caused by: java.lang.AssertionError: Internal error: While invoking method 
> 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult 
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Project,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'
> at org.apache.calcite.util.Util.newInternal(Util.java:789)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:534)
> at 
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:270)
> at 
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:213)
> at 
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:466)
> ... 107 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531)
> ... 110 more
> Caused by: java.lang.IndexOutOfBoundsException: Index: 94, Size: 94
> at java.util.ArrayList.rangeCheck(ArrayList.java:653)
> at java.util.ArrayList.get(ArrayList.java:429)
> at 
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(HiveRelFieldTrimmer.java:390)
> {code}
> Reproducible case:
> 1) set "hive.security.authorization.enabled" to "true" in "hive-site.xml".
> 2) {code:java}
> CREATE TABLE table1 (id int) PARTITIONED BY (year int)
> -- create partitioned view
> CREATE VIEW view1 partitioned on (year) as select id, year from table1;
> set hive.cbo.enable=true;
> select year from view1;
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (HIVE-16828) With CBO enabled, Query on partitioned views throws IndexOutOfBoundException

[Sanjay Mishra (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204643#comment-16204643
 ] 

Sanjay Mishra edited comment on HIVE-16828 at 10/14/17 2:21 PM:


Facing the same issue in Hive 2.1.0. 
The issue is with all types of views. Tried this:
Authorization is enabled.
Create table test_table(id string);
Create view test_view as select * from test_table;
Select * from test_view; -> Throws the mentioned error.

Is there are plan to release the fix for the same anytime soon.?


was (Author: sanjurm16):
Facing the same issue in Hive 2.1.0. Is there are plan to release the fix for 
the same anytime soon.?

> With CBO enabled, Query on partitioned views throws IndexOutOfBoundException
> 
>
> Key: HIVE-16828
> URL: https://issues.apache.org/jira/browse/HIVE-16828
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Affects Versions: 1.2.0, 2.1.1, 2.2.0
>Reporter: Adesh Kumar Rao
>Assignee: Vineet Garg
> Attachments: HIVE-16828.patch
>
>
> {code:java}
> Caused by: java.lang.AssertionError: Internal error: While invoking method 
> 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult 
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Filter,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'
> at org.apache.calcite.util.Util.newInternal(Util.java:789)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:534)
> at 
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:270)
> at 
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:213)
> at 
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:374)
> at 
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(HiveRelFieldTrimmer.java:394)
> ... 98 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531)
> ... 102 more
> Caused by: java.lang.AssertionError: Internal error: While invoking method 
> 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult 
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Project,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'
> at org.apache.calcite.util.Util.newInternal(Util.java:789)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:534)
> at 
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:270)
> at 
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:213)
> at 
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:466)
> ... 107 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531)
> ... 110 more
> Caused by: java.lang.IndexOutOfBoundsException: Index: 94, Size: 94
> at java.util.ArrayList.rangeCheck(ArrayList.java:653)
> at java.util.ArrayList.get(ArrayList.java:429)
> at 
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(HiveRelFieldTrimmer.java:390)
> {code}
> Reproducible case:
> 1) set "hive.security.authorization.enabled" to "true" in "hive-site.xml".
> 2) {code:java}
> CREATE TABLE table1 (id int) PARTITIONED BY (year int)
> -- create partitioned view
> CREATE VIEW view1 partitioned on (year) as select id, year from table1;
> set hive.cbo.enable=true;
> select year from view1;
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-16828) With CBO enabled, Query on partitioned views throws IndexOutOfBoundException

[Sanjay Mishra (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204643#comment-16204643
 ] 

Sanjay Mishra commented on HIVE-16828:
--

Facing the same issue in Hive 2.1.0. Is there are plan to release the fix for 
the same anytime soon.?

> With CBO enabled, Query on partitioned views throws IndexOutOfBoundException
> 
>
> Key: HIVE-16828
> URL: https://issues.apache.org/jira/browse/HIVE-16828
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Affects Versions: 1.2.0, 2.1.1, 2.2.0
>Reporter: Adesh Kumar Rao
>Assignee: Vineet Garg
> Attachments: HIVE-16828.patch
>
>
> {code:java}
> Caused by: java.lang.AssertionError: Internal error: While invoking method 
> 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult 
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Filter,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'
> at org.apache.calcite.util.Util.newInternal(Util.java:789)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:534)
> at 
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:270)
> at 
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:213)
> at 
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:374)
> at 
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(HiveRelFieldTrimmer.java:394)
> ... 98 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531)
> ... 102 more
> Caused by: java.lang.AssertionError: Internal error: While invoking method 
> 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult 
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Project,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'
> at org.apache.calcite.util.Util.newInternal(Util.java:789)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:534)
> at 
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:270)
> at 
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:213)
> at 
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:466)
> ... 107 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531)
> ... 110 more
> Caused by: java.lang.IndexOutOfBoundsException: Index: 94, Size: 94
> at java.util.ArrayList.rangeCheck(ArrayList.java:653)
> at java.util.ArrayList.get(ArrayList.java:429)
> at 
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(HiveRelFieldTrimmer.java:390)
> {code}
> Reproducible case:
> 1) set "hive.security.authorization.enabled" to "true" in "hive-site.xml".
> 2) {code:java}
> CREATE TABLE table1 (id int) PARTITIONED BY (year int)
> -- create partitioned view
> CREATE VIEW view1 partitioned on (year) as select id, year from table1;
> set hive.cbo.enable=true;
> select year from view1;
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17801) OpenCSVserde should store schema in metastore

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204631#comment-16204631
 ] 

Hive QA commented on HIVE-17801:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12892103/HIVE-17801.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 12 failed/errored test(s), 11233 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[optimize_nullscan]
 (batchId=163)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi] 
(batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_notin] 
(batchId=133)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_scalar] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_select] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_views] 
(batchId=108)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query16] 
(batchId=242)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query94] 
(batchId=242)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query14] 
(batchId=240)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query16] 
(batchId=240)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query94] 
(batchId=240)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut 
(batchId=203)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7295/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7295/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7295/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 12 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12892103 - PreCommit-HIVE-Build

> OpenCSVserde should store schema in metastore
> -
>
> Key: HIVE-17801
> URL: https://issues.apache.org/jira/browse/HIVE-17801
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore, Serializers/Deserializers
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-17801.patch
>
>
> Just need to add opencsv serde in config.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17787) Apply more filters on the BeeLine test output files (follow-up on HIVE-17569)

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204602#comment-16204602
 ] 

Hive QA commented on HIVE-17787:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12892085/HIVE-17787.2.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 13 failed/errored test(s), 11233 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[optimize_nullscan]
 (batchId=163)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi] 
(batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_notin] 
(batchId=133)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_scalar] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_select] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_views] 
(batchId=108)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query16] 
(batchId=242)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query94] 
(batchId=242)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query14] 
(batchId=240)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query16] 
(batchId=240)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query94] 
(batchId=240)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut 
(batchId=203)
org.apache.hive.jdbc.TestJdbcDriver2.testSelectExecAsync2 (batchId=227)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7294/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7294/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7294/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 13 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12892085 - PreCommit-HIVE-Build

> Apply more filters on the BeeLine test output files (follow-up on HIVE-17569)
> -
>
> Key: HIVE-17787
> URL: https://issues.apache.org/jira/browse/HIVE-17787
> Project: Hive
>  Issue Type: Improvement
>  Components: Testing Infrastructure
>Affects Versions: 3.0.0
>Reporter: Marta Kuczora
>Assignee: Marta Kuczora
>Priority: Minor
> Attachments: HIVE-17787.1.patch, HIVE-17787.2.patch
>
>
> When running the q tests with BeeLine, some known differences came up which 
> should be filtered out if the "test.beeline.compare.portable" parameter is 
> set to true.
> The result of the following commands can be different when running them via 
> BeeLine then in the golden out file:
> - DESCRIBE
> - SHOW TABLES
> - SHOW FORMATTED TABLES
> - SHOW DATABASES TABLES
> Also the join warnings and the mapreduce jobtracker address can be different 
> so it would make sense to filter them out.
> For example:
> {noformat}
> Warning: Map Join MAPJOIN[13][bigTable=?] in task 'Stage-3:MAPRED' is a cross 
> product
> Warning: MASKED is a cross product
> {noformat}
> {noformat}
> mapreduce.jobtracker.address=local
> mapreduce.jobtracker.address=MASKED
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-16688) Make sure Alter Table to set transaction=true acquires X lock

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204577#comment-16204577
 ] 

Hive QA commented on HIVE-16688:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12892082/HIVE-16688.02.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 13 failed/errored test(s), 11234 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[update_all_non_partitioned]
 (batchId=7)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[optimize_nullscan]
 (batchId=163)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi] 
(batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_notin] 
(batchId=133)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_scalar] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_select] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_views] 
(batchId=108)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query16] 
(batchId=242)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query94] 
(batchId=242)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query14] 
(batchId=240)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query16] 
(batchId=240)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query94] 
(batchId=240)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut 
(batchId=203)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7293/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7293/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7293/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 13 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12892082 - PreCommit-HIVE-Build

> Make sure Alter Table to set transaction=true acquires X lock
> -
>
> Key: HIVE-16688
> URL: https://issues.apache.org/jira/browse/HIVE-16688
> Project: Hive
>  Issue Type: Sub-task
>  Components: Transactions
>Affects Versions: 1.3.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Critical
> Attachments: HIVE-16688.01.patch, HIVE-16688.02.patch
>
>
> suppose we have non-acid table with some data
> An insert op starts (long running)  (with hive.txn.strict.locking.mode=false 
> this takes shared lock)
> An alter table runs to add (transactional=true)
> An update is run which will read the list of "original" files and assign IDs 
> on the fly which are written to a delta file.
> The long running insert completes.
> Another update is run which now sees a different set of "original" files and 
> will (most likely) assign different IDs.
> Need to make sure to mutex this



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17799) Add Ellipsis For Truncated Query In Hive Lock

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204557#comment-16204557
 ] 

Hive QA commented on HIVE-17799:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12892083/HIVE-17799.1.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 13 failed/errored test(s), 11233 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[optimize_nullscan]
 (batchId=163)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi] 
(batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_notin] 
(batchId=133)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_scalar] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_select] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_views] 
(batchId=108)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query16] 
(batchId=242)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query94] 
(batchId=242)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query14] 
(batchId=240)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query16] 
(batchId=240)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query23] 
(batchId=240)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query94] 
(batchId=240)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut 
(batchId=203)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7292/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7292/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7292/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 13 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12892083 - PreCommit-HIVE-Build

> Add Ellipsis For Truncated Query In Hive Lock
> -
>
> Key: HIVE-17799
> URL: https://issues.apache.org/jira/browse/HIVE-17799
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Affects Versions: 3.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Attachments: HIVE-17799.1.patch
>
>
> [HIVE-16334] introduced truncation for storing queries in ZK lock nodes.  
> This Jira is to add ellipsis into the query to let the operator know that 
> truncation has occurred and therefore they will not find the specific query 
> in their logs, only a prefix match will work.
> {code:sql}
> -- Truncation of query may be confusing to operator
> -- Without truncation
> SELECT * FROM TABLE WHERE COL=1
> -- With truncation (operator will not find this query in workload)
> SELECT * FROM TABLE
> -- With truncation (operator will know this is only a prefix match)
> SELECT * FROM TABLE...
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17733) Move RawStore to standalone metastore

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204541#comment-16204541
 ] 

Hive QA commented on HIVE-17733:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12892081/HIVE-17733.6.patch

{color:green}SUCCESS:{color} +1 due to 14 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 11233 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[update_all_non_partitioned]
 (batchId=7)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[optimize_nullscan]
 (batchId=163)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2] 
(batchId=101)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi] 
(batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_notin] 
(batchId=133)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_scalar] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_select] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_views] 
(batchId=108)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query16] 
(batchId=243)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query94] 
(batchId=243)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query14] 
(batchId=241)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query16] 
(batchId=241)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query94] 
(batchId=241)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut 
(batchId=204)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7291/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7291/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7291/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12892081 - PreCommit-HIVE-Build

> Move RawStore to standalone metastore
> -
>
> Key: HIVE-17733
> URL: https://issues.apache.org/jira/browse/HIVE-17733
> Project: Hive
>  Issue Type: Sub-task
>  Components: Metastore
>Reporter: Alan Gates
>Assignee: Alan Gates
>  Labels: pull-request-available
> Attachments: HIVE-17733.2.patch, HIVE-17733.3.patch, 
> HIVE-17733.4.patch, HIVE-17733.5.patch, HIVE-17733.6.patch, HIVE-17733.patch
>
>
> This includes moving implementations of RawStore (like ObjectStore), 
> MetastoreDirectSql, and stats related classes like ColumnStatsAggregator and 
> the NDV classes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17798) When replacing the src table names in BeeLine testing, the table names shouldn't be changed to lower case

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204518#comment-16204518
 ] 

Hive QA commented on HIVE-17798:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12892079/HIVE-17798.1.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 13 failed/errored test(s), 11233 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[optimize_nullscan]
 (batchId=163)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2] 
(batchId=101)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi] 
(batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_notin] 
(batchId=133)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_scalar] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_select] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_views] 
(batchId=108)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query16] 
(batchId=242)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query94] 
(batchId=242)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query14] 
(batchId=240)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query16] 
(batchId=240)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query94] 
(batchId=240)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut 
(batchId=203)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7290/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7290/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7290/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 13 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12892079 - PreCommit-HIVE-Build

> When replacing the src table names in BeeLine testing, the table names 
> shouldn't be changed to lower case
> -
>
> Key: HIVE-17798
> URL: https://issues.apache.org/jira/browse/HIVE-17798
> Project: Hive
>  Issue Type: Bug
>  Components: Testing Infrastructure
>Affects Versions: 3.0.0
>Reporter: Marta Kuczora
>Assignee: Marta Kuczora
>Priority: Minor
> Attachments: HIVE-17798.1.patch
>
>
> When running the q tests with BeeLine, the name of the src tables are changed 
> in all queries to have the database name as prefix, like src -> default.src, 
> srcpart -> default.srcpart
> This renaming mechanism changes the table names to lower case. For example 
> the query "SELECT * FROM SRC" will be "SELECT * FROM src" after the rewite. 
> This will cause failure during the comparison of the out files.
> Change the QFile.replaceTableNames method to keep the upper case letters 
> unchanged.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17793) Parameterize Logging Messages

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204501#comment-16204501
 ] 

Hive QA commented on HIVE-17793:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12892078/HIVE-17793.2.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 13 failed/errored test(s), 11233 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[optimize_nullscan]
 (batchId=163)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2] 
(batchId=101)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi] 
(batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_notin] 
(batchId=133)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_scalar] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_select] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_views] 
(batchId=108)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query16] 
(batchId=242)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query94] 
(batchId=242)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query14] 
(batchId=240)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query16] 
(batchId=240)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query94] 
(batchId=240)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut 
(batchId=203)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7289/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7289/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7289/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 13 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12892078 - PreCommit-HIVE-Build

> Parameterize Logging Messages
> -
>
> Key: HIVE-17793
> URL: https://issues.apache.org/jira/browse/HIVE-17793
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Affects Versions: 3.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Attachments: HIVE-17793.1.patch, HIVE-17793.2.patch
>
>
> * Use SLF4J parameterized logging
> * Remove use of archaic Util's "stringifyException" and simply allow logging 
> framework to handle formatting of output.  Also saves having to create the 
> error message and then throwing it away when the logging level is set higher 
> than the logging message
> * Add some {{LOG.isDebugEnabled}} around complex debug messages



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)