[jira] [Reopened] (HIVE-20470) Runtime of QueryInfo should be initialized
[ https://issues.apache.org/jira/browse/HIVE-20470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alice Fan reopened HIVE-20470: -- > Runtime of QueryInfo should be initialized > -- > > Key: HIVE-20470 > URL: https://issues.apache.org/jira/browse/HIVE-20470 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 4.0.0 >Reporter: Alice Fan >Assignee: Alice Fan >Priority: Minor > Fix For: 4.0.0 > > > Runtime of QueryInfo should be initialized to prevent NPE at hiveserver2_jsp -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-12812) Enable mapred.input.dir.recursive by default to support union with aggregate function
[ https://issues.apache.org/jira/browse/HIVE-12812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alice Fan updated HIVE-12812: - Attachment: HIVE-12812.1.patch Status: Patch Available (was: In Progress) > Enable mapred.input.dir.recursive by default to support union with aggregate > function > - > > Key: HIVE-12812 > URL: https://issues.apache.org/jira/browse/HIVE-12812 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.1.0, 1.2.1 >Reporter: Chaoyu Tang >Assignee: Alice Fan >Priority: Major > Attachments: HIVE-12812.1.patch, HIVE-12812.patch, HIVE-12812.patch, > HIVE-12812.patch > > > When union remove optimization is enabled, union query with aggregate > function writes its subquery intermediate results to subdirs which needs > mapred.input.dir.recursive to be enabled in order to be fetched. This > property is not defined by default in Hive and often ignored by user, which > causes the query failure and is hard to be debugged. > So we need set mapred.input.dir.recursive to true whenever union remove > optimization is enabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-12812) Enable mapred.input.dir.recursive by default to support union with aggregate function
[ https://issues.apache.org/jira/browse/HIVE-12812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alice Fan updated HIVE-12812: - Attachment: (was: HIVE-12812.1.patch) > Enable mapred.input.dir.recursive by default to support union with aggregate > function > - > > Key: HIVE-12812 > URL: https://issues.apache.org/jira/browse/HIVE-12812 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 1.2.1, 2.1.0 >Reporter: Chaoyu Tang >Assignee: Alice Fan >Priority: Major > Attachments: HIVE-12812.1.patch, HIVE-12812.patch, HIVE-12812.patch, > HIVE-12812.patch > > > When union remove optimization is enabled, union query with aggregate > function writes its subquery intermediate results to subdirs which needs > mapred.input.dir.recursive to be enabled in order to be fetched. This > property is not defined by default in Hive and often ignored by user, which > causes the query failure and is hard to be debugged. > So we need set mapred.input.dir.recursive to true whenever union remove > optimization is enabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HIVE-12812) Enable mapred.input.dir.recursive by default to support union with aggregate function
[ https://issues.apache.org/jira/browse/HIVE-12812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-12812 started by Alice Fan. > Enable mapred.input.dir.recursive by default to support union with aggregate > function > - > > Key: HIVE-12812 > URL: https://issues.apache.org/jira/browse/HIVE-12812 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 1.2.1, 2.1.0 >Reporter: Chaoyu Tang >Assignee: Alice Fan >Priority: Major > Attachments: HIVE-12812.1.patch, HIVE-12812.patch, HIVE-12812.patch, > HIVE-12812.patch > > > When union remove optimization is enabled, union query with aggregate > function writes its subquery intermediate results to subdirs which needs > mapred.input.dir.recursive to be enabled in order to be fetched. This > property is not defined by default in Hive and often ignored by user, which > causes the query failure and is hard to be debugged. > So we need set mapred.input.dir.recursive to true whenever union remove > optimization is enabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-12812) Enable mapred.input.dir.recursive by default to support union with aggregate function
[ https://issues.apache.org/jira/browse/HIVE-12812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alice Fan updated HIVE-12812: - Attachment: HIVE-12812.1.patch > Enable mapred.input.dir.recursive by default to support union with aggregate > function > - > > Key: HIVE-12812 > URL: https://issues.apache.org/jira/browse/HIVE-12812 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 1.2.1, 2.1.0 >Reporter: Chaoyu Tang >Assignee: Alice Fan >Priority: Major > Attachments: HIVE-12812.1.patch, HIVE-12812.patch, HIVE-12812.patch, > HIVE-12812.patch > > > When union remove optimization is enabled, union query with aggregate > function writes its subquery intermediate results to subdirs which needs > mapred.input.dir.recursive to be enabled in order to be fetched. This > property is not defined by default in Hive and often ignored by user, which > causes the query failure and is hard to be debugged. > So we need set mapred.input.dir.recursive to true whenever union remove > optimization is enabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-12812) Enable mapred.input.dir.recursive by default to support union with aggregate function
[ https://issues.apache.org/jira/browse/HIVE-12812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alice Fan reassigned HIVE-12812: Assignee: Alice Fan > Enable mapred.input.dir.recursive by default to support union with aggregate > function > - > > Key: HIVE-12812 > URL: https://issues.apache.org/jira/browse/HIVE-12812 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 1.2.1, 2.1.0 >Reporter: Chaoyu Tang >Assignee: Alice Fan >Priority: Major > Attachments: HIVE-12812.patch, HIVE-12812.patch, HIVE-12812.patch > > > When union remove optimization is enabled, union query with aggregate > function writes its subquery intermediate results to subdirs which needs > mapred.input.dir.recursive to be enabled in order to be fetched. This > property is not defined by default in Hive and often ignored by user, which > causes the query failure and is hard to be debugged. > So we need set mapred.input.dir.recursive to true whenever union remove > optimization is enabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20556) Expose an API to retrieve the TBL_ID from TBLS in the metastore tables
[ https://issues.apache.org/jira/browse/HIVE-20556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M updated HIVE-20556: --- Attachment: HIVE-20556.14.patch Status: Patch Available (was: Open) > Expose an API to retrieve the TBL_ID from TBLS in the metastore tables > -- > > Key: HIVE-20556 > URL: https://issues.apache.org/jira/browse/HIVE-20556 > Project: Hive > Issue Type: New Feature > Components: Metastore, Standalone Metastore >Reporter: Jaume M >Assignee: Jaume M >Priority: Major > Attachments: HIVE-20556.1.patch, HIVE-20556.10.patch, > HIVE-20556.11.patch, HIVE-20556.12.patch, HIVE-20556.13.patch, > HIVE-20556.14.patch, HIVE-20556.2.patch, HIVE-20556.3.patch, > HIVE-20556.4.patch, HIVE-20556.5.patch, HIVE-20556.6.patch, > HIVE-20556.7.patch, HIVE-20556.8.patch, HIVE-20556.9.patch > > > We have two options to do this > 1) Use the current MTable and add a field for this value > 2) Add an independent API call to the metastore that would return the TBL_ID. > Option 1 is preferable. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20556) Expose an API to retrieve the TBL_ID from TBLS in the metastore tables
[ https://issues.apache.org/jira/browse/HIVE-20556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M updated HIVE-20556: --- Status: Open (was: Patch Available) > Expose an API to retrieve the TBL_ID from TBLS in the metastore tables > -- > > Key: HIVE-20556 > URL: https://issues.apache.org/jira/browse/HIVE-20556 > Project: Hive > Issue Type: New Feature > Components: Metastore, Standalone Metastore >Reporter: Jaume M >Assignee: Jaume M >Priority: Major > Attachments: HIVE-20556.1.patch, HIVE-20556.10.patch, > HIVE-20556.11.patch, HIVE-20556.12.patch, HIVE-20556.13.patch, > HIVE-20556.2.patch, HIVE-20556.3.patch, HIVE-20556.4.patch, > HIVE-20556.5.patch, HIVE-20556.6.patch, HIVE-20556.7.patch, > HIVE-20556.8.patch, HIVE-20556.9.patch > > > We have two options to do this > 1) Use the current MTable and add a field for this value > 2) Add an independent API call to the metastore that would return the TBL_ID. > Option 1 is preferable. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20607) TxnHandler should use PreparedStatement to execute direct SQL queries.
[
https://issues.apache.org/jira/browse/HIVE-20607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625373#comment-16625373
]
Hive QA commented on HIVE-20607:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12940997/HIVE-20607.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14994 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14009/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14009/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14009/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12940997 - PreCommit-HIVE-Build
> TxnHandler should use PreparedStatement to execute direct SQL queries.
> --
>
> Key: HIVE-20607
> URL: https://issues.apache.org/jira/browse/HIVE-20607
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore, Transactions
>Affects Versions: 4.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
> Labels: ACID, pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-20607.01.patch
>
>
> TxnHandler uses direct SQL queries to operate on Txn related databases/tables
> in Hive metastore RDBMS.
> Most of the methods are direct calls from Metastore api which should be
> directly append input string arguments to the SQL string.
> Need to use parameterised PreparedStatement object to set these arguments.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18778) Needs to capture input/output entities in explain
[
https://issues.apache.org/jira/browse/HIVE-18778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated HIVE-18778:
--
Attachment: HIVE-18778.9.patch
> Needs to capture input/output entities in explain
> -
>
> Key: HIVE-18778
> URL: https://issues.apache.org/jira/browse/HIVE-18778
> Project: Hive
> Issue Type: Bug
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-18778-SparkPositive.patch, HIVE-18778.1.patch,
> HIVE-18778.2.patch, HIVE-18778.3.patch, HIVE-18778.4.patch,
> HIVE-18778.5.patch, HIVE-18778.6.patch, HIVE-18778.7.patch,
> HIVE-18778.8.patch, HIVE-18778.9.patch, HIVE-18778_TestCliDriver.patch,
> HIVE-18788_SparkNegative.patch, HIVE-18788_SparkPerf.patch
>
>
> With Sentry enabled, commands like explain drop table foo fail with {{explain
> drop table foo;}}
> {code}
> Error: Error while compiling statement: FAILED: SemanticException No valid
> privileges
> Required privilege( Table) not available in input privileges
> The required privileges: (state=42000,code=4)
> {code}
> Sentry fails to authorize because the ExplainSemanticAnalyzer uses an
> instance of DDLSemanticAnalyzer to analyze the explain query.
> {code}
> BaseSemanticAnalyzer sem = SemanticAnalyzerFactory.get(conf, input);
> sem.analyze(input, ctx);
> sem.validate()
> {code}
> The inputs/outputs entities for this query are set in the above code.
> However, these are never set on the instance of ExplainSemanticAnalyzer
> itself and thus is not propagated into the HookContext in the calling Driver
> code.
> {code}
> sem.analyze(tree, ctx); --> this results in calling the above code that uses
> DDLSA
> hookCtx.update(sem); --> sem is an instance of ExplainSemanticAnalyzer, this
> code attempts to update the HookContext with the input/output info from ESA
> which is never set.
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20607) TxnHandler should use PreparedStatement to execute direct SQL queries.
[
https://issues.apache.org/jira/browse/HIVE-20607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625358#comment-16625358
]
Hive QA commented on HIVE-20607:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
24s{color} | {color:blue} hcatalog/server-extensions in master has 2 extant
Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
47s{color} | {color:blue} ql in master has 2326 extant Findbugs warnings.
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
13s{color} | {color:red} metastore-server in master failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
23s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
17s{color} | {color:red} server-extensions in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
18s{color} | {color:red} server-extensions in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 18s{color}
| {color:red} server-extensions in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
17s{color} | {color:red} server-extensions in the patch failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
12s{color} | {color:red} metastore-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 0s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14009/dev-support/hive-personality.sh
|
| git revision | master / 0f7163f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14009/yetus/branch-findbugs-standalone-metastore_metastore-server.txt
|
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14009/yetus/patch-mvninstall-hcatalog_server-extensions.txt
|
| compile |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14009/yetus/patch-compile-hcatalog_server-extensions.txt
|
| javac |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14009/yetus/patch-compile-hcatalog_server-extensions.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14009/yetus/patch-findbugs-hcatalog_server-extensions.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14009/yetus/patch-findbugs-standalone-metastore_metastore-server.txt
|
| modules | C: hcatalog/server-extensions ql
standalone-metastore/metastore-server U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14009/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> TxnHandler should use PreparedStatement to execute direct SQL queries.
[jira] [Commented] (HIVE-20541) REPL DUMP on external table with add partition event throws NoSuchElementException.
[ https://issues.apache.org/jira/browse/HIVE-20541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625349#comment-16625349 ] ASF GitHub Bot commented on HIVE-20541: --- Github user sankarh closed the pull request at: https://github.com/apache/hive/pull/432 > REPL DUMP on external table with add partition event throws > NoSuchElementException. > --- > > Key: HIVE-20541 > URL: https://issues.apache.org/jira/browse/HIVE-20541 > Project: Hive > Issue Type: Bug > Components: repl >Affects Versions: 4.0.0, 3.2.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: DR, Replication, pull-request-available > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-20541.01.patch > > > REPL dump on an external table with add partition event throws > NoSuchElementException. Need to check if file iterator list hasNext before > accessing it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20607) TxnHandler should use PreparedStatement to execute direct SQL queries.
[ https://issues.apache.org/jira/browse/HIVE-20607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-20607: Attachment: HIVE-20607.01.patch > TxnHandler should use PreparedStatement to execute direct SQL queries. > -- > > Key: HIVE-20607 > URL: https://issues.apache.org/jira/browse/HIVE-20607 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore, Transactions >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: ACID, pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-20607.01.patch > > > TxnHandler uses direct SQL queries to operate on Txn related databases/tables > in Hive metastore RDBMS. > Most of the methods are direct calls from Metastore api which should be > directly append input string arguments to the SQL string. > Need to use parameterised PreparedStatement object to set these arguments. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20607) TxnHandler should use PreparedStatement to execute direct SQL queries.
[ https://issues.apache.org/jira/browse/HIVE-20607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-20607: Status: Open (was: Patch Available) > TxnHandler should use PreparedStatement to execute direct SQL queries. > -- > > Key: HIVE-20607 > URL: https://issues.apache.org/jira/browse/HIVE-20607 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore, Transactions >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: ACID, pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-20607.01.patch > > > TxnHandler uses direct SQL queries to operate on Txn related databases/tables > in Hive metastore RDBMS. > Most of the methods are direct calls from Metastore api which should be > directly append input string arguments to the SQL string. > Need to use parameterised PreparedStatement object to set these arguments. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20607) TxnHandler should use PreparedStatement to execute direct SQL queries.
[ https://issues.apache.org/jira/browse/HIVE-20607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-20607: Status: Patch Available (was: Open) Re-attaching same patch as test failures are irrelevant. > TxnHandler should use PreparedStatement to execute direct SQL queries. > -- > > Key: HIVE-20607 > URL: https://issues.apache.org/jira/browse/HIVE-20607 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore, Transactions >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: ACID, pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-20607.01.patch > > > TxnHandler uses direct SQL queries to operate on Txn related databases/tables > in Hive metastore RDBMS. > Most of the methods are direct calls from Metastore api which should be > directly append input string arguments to the SQL string. > Need to use parameterised PreparedStatement object to set these arguments. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20607) TxnHandler should use PreparedStatement to execute direct SQL queries.
[ https://issues.apache.org/jira/browse/HIVE-20607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-20607: Attachment: (was: HIVE-20607.01.patch) > TxnHandler should use PreparedStatement to execute direct SQL queries. > -- > > Key: HIVE-20607 > URL: https://issues.apache.org/jira/browse/HIVE-20607 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore, Transactions >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: ACID, pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-20607.01.patch > > > TxnHandler uses direct SQL queries to operate on Txn related databases/tables > in Hive metastore RDBMS. > Most of the methods are direct calls from Metastore api which should be > directly append input string arguments to the SQL string. > Need to use parameterised PreparedStatement object to set these arguments. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20604) Minor compaction disables ORC column stats
[
https://issues.apache.org/jira/browse/HIVE-20604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625294#comment-16625294
]
Prasanth Jayachandran commented on HIVE-20604:
--
Yes. But doing PPD on ROW__ID would require row index.
> Minor compaction disables ORC column stats
> --
>
> Key: HIVE-20604
> URL: https://issues.apache.org/jira/browse/HIVE-20604
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Affects Versions: 1.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Major
> Fix For: 4.0.0
>
>
> {noformat}
> @Override
> public org.apache.hadoop.hive.ql.exec.FileSinkOperator.RecordWriter
> getRawRecordWriter(Path path, Options options) throws IOException {
> final Path filename = AcidUtils.createFilename(path, options);
> final OrcFile.WriterOptions opts =
> OrcFile.writerOptions(options.getTableProperties(),
> options.getConfiguration());
> if (!options.isWritingBase()) {
> opts.bufferSize(OrcRecordUpdater.DELTA_BUFFER_SIZE)
> .stripeSize(OrcRecordUpdater.DELTA_STRIPE_SIZE)
> .blockPadding(false)
> .compress(CompressionKind.NONE)
> .rowIndexStride(0)
> ;
> }
> {noformat}
> {{rowIndexStride(0)}} makes {{StripeStatistics.getColumnStatistics()}} return
> objects but with meaningless values, like min/max for
> {{IntegerColumnStatistics}} set to MIN_LONG/MAX_LONG.
> This interferes with ability to infer min ROW_ID for a split but also creates
> inefficient files.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17043) Remove non unique columns from group by keys if not referenced later
[
https://issues.apache.org/jira/browse/HIVE-17043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625288#comment-16625288
]
Hive QA commented on HIVE-17043:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12940852/HIVE-17043.5.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14008/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14008/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14008/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12940852/HIVE-17043.5.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12940852 - PreCommit-HIVE-Build
> Remove non unique columns from group by keys if not referenced later
>
>
> Key: HIVE-17043
> URL: https://issues.apache.org/jira/browse/HIVE-17043
> Project: Hive
> Issue Type: Sub-task
> Components: Logical Optimizer
>Affects Versions: 3.0.0
>Reporter: Ashutosh Chauhan
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-17043.1.patch, HIVE-17043.2.patch,
> HIVE-17043.3.patch, HIVE-17043.4.patch, HIVE-17043.5.patch
>
>
> Group by keys may be a mix of unique (or primary) keys and regular columns.
> In such cases presence of regular column won't alter cardinality of groups.
> So, if regular columns are not referenced later, they can be dropped from
> group by keys. Depending on operator tree may result in those columns not
> being read at all from disk in best case. In worst case, we will avoid
> shuffling and sorting regular columns from mapper to reducer, which still
> could be substantial CPU and network savings.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20523) Improve table statistics for Parquet format
[
https://issues.apache.org/jira/browse/HIVE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625287#comment-16625287
]
Hive QA commented on HIVE-20523:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12940982/HIVE-20523.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 90 failed/errored test(s), 14993 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=194)
[druidmini_masking.q,druidmini_test1.q,druidkafkamini_basic.q,druidmini_joins.q,druid_timestamptz.q]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_update_status]
(batchId=85)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nested_column_pruning]
(batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_analyze]
(batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_array_null_element]
(batchId=79)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_complex_types_vectorization]
(batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_join]
(batchId=21)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_map_null]
(batchId=90)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_map_type_vectorization]
(batchId=89)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_no_row_serde]
(batchId=74)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_read_backward_compatible_files]
(batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_schema_evolution]
(batchId=80)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_struct_type_vectorization]
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_types_non_dictionary_encoding_vectorization]
(batchId=91)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_types_vectorization]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_0]
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_10]
(batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_11]
(batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_12]
(batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_13]
(batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_14]
(batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_15]
(batchId=92)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_16]
(batchId=87)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_17]
(batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_1]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_2]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_3]
(batchId=82)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_4]
(batchId=46)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_5]
(batchId=75)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_6]
(batchId=44)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_7]
(batchId=90)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_8]
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_9]
(batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_decimal_date]
(batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_div0]
(batchId=82)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_limit]
(batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_nested_udf]
(batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_not]
(batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_offset_limit]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_part]
(batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_part_project]
(batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_part_varchar]
(batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_pushdown]
(batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorization_numeric_overflows]
(batchId=73)
[jira] [Commented] (HIVE-20523) Improve table statistics for Parquet format
[
https://issues.apache.org/jira/browse/HIVE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625274#comment-16625274
]
Hive QA commented on HIVE-20523:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
52s{color} | {color:blue} ql in master has 2326 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
36s{color} | {color:red} ql: The patch generated 1 new + 5 unchanged - 1 fixed
= 6 total (was 6) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 19s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14007/dev-support/hive-personality.sh
|
| git revision | master / 0f7163f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14007/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14007/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Improve table statistics for Parquet format
> ---
>
> Key: HIVE-20523
> URL: https://issues.apache.org/jira/browse/HIVE-20523
> Project: Hive
> Issue Type: Improvement
> Components: Physical Optimizer
>Reporter: George Pachitariu
>Assignee: George Pachitariu
>Priority: Minor
> Attachments: HIVE-20523.1.patch, HIVE-20523.2.patch,
> HIVE-20523.3.patch, HIVE-20523.patch
>
>
> Right now, in the table basic statistics, the *raw data size* for a row with
> any data type in the Parquet format is 1. This is an underestimated value
> when columns are complex data structures, like arrays.
> Having tables with underestimated raw data size makes Hive assign less
> containers (mappers/reducers) to it, making the overall query slower.
> Heavy underestimation also makes Hive choose MapJoin instead of the
> ShuffleJoin that can fail with OOM errors.
> In this patch, I compute the columns data size better, taking into account
> complex structures. I followed the Writer implementation for the ORC format.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20523) Improve table statistics for Parquet format
[
https://issues.apache.org/jira/browse/HIVE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625265#comment-16625265
]
Hive QA commented on HIVE-20523:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12940980/HIVE-20523.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 98 failed/errored test(s), 14995 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nested_column_pruning]
(batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_analyze]
(batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_array_map_emptynullvals]
(batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_array_null_element]
(batchId=79)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_complex_types_vectorization]
(batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_create]
(batchId=94)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_decimal1]
(batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_join]
(batchId=21)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_map_null]
(batchId=90)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_map_of_arrays_of_ints]
(batchId=10)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_map_of_maps]
(batchId=70)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_map_type_vectorization]
(batchId=89)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_nested_complex]
(batchId=94)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_no_row_serde]
(batchId=74)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_read_backward_compatible_files]
(batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_schema_evolution]
(batchId=80)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_struct_type_vectorization]
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_type_promotion]
(batchId=91)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_types_non_dictionary_encoding_vectorization]
(batchId=91)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_types_vectorization]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_0]
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_10]
(batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_11]
(batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_12]
(batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_13]
(batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_14]
(batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_15]
(batchId=92)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_16]
(batchId=87)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_17]
(batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_1]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_2]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_3]
(batchId=82)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_4]
(batchId=46)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_5]
(batchId=75)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_6]
(batchId=44)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_7]
(batchId=90)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_8]
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_9]
(batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_decimal_date]
(batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_div0]
(batchId=82)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_limit]
(batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_nested_udf]
(batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_not]
(batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_offset_limit]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_part]
(batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_part_project]
(batchId=37)
[jira] [Updated] (HIVE-20523) Improve table statistics for Parquet format
[ https://issues.apache.org/jira/browse/HIVE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] George Pachitariu updated HIVE-20523: - Attachment: HIVE-20523.3.patch Status: Patch Available (was: Open) > Improve table statistics for Parquet format > --- > > Key: HIVE-20523 > URL: https://issues.apache.org/jira/browse/HIVE-20523 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Reporter: George Pachitariu >Assignee: George Pachitariu >Priority: Minor > Attachments: HIVE-20523.1.patch, HIVE-20523.2.patch, > HIVE-20523.3.patch, HIVE-20523.patch > > > Right now, in the table basic statistics, the *raw data size* for a row with > any data type in the Parquet format is 1. This is an underestimated value > when columns are complex data structures, like arrays. > Having tables with underestimated raw data size makes Hive assign less > containers (mappers/reducers) to it, making the overall query slower. > Heavy underestimation also makes Hive choose MapJoin instead of the > ShuffleJoin that can fail with OOM errors. > In this patch, I compute the columns data size better, taking into account > complex structures. I followed the Writer implementation for the ORC format. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20523) Improve table statistics for Parquet format
[
https://issues.apache.org/jira/browse/HIVE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625256#comment-16625256
]
Hive QA commented on HIVE-20523:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
50s{color} | {color:blue} ql in master has 2326 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} ql: The patch generated 0 new + 5 unchanged - 1
fixed = 5 total (was 6) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 47s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14006/dev-support/hive-personality.sh
|
| git revision | master / 0f7163f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14006/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Improve table statistics for Parquet format
> ---
>
> Key: HIVE-20523
> URL: https://issues.apache.org/jira/browse/HIVE-20523
> Project: Hive
> Issue Type: Improvement
> Components: Physical Optimizer
>Reporter: George Pachitariu
>Assignee: George Pachitariu
>Priority: Minor
> Attachments: HIVE-20523.1.patch, HIVE-20523.2.patch, HIVE-20523.patch
>
>
> Right now, in the table basic statistics, the *raw data size* for a row with
> any data type in the Parquet format is 1. This is an underestimated value
> when columns are complex data structures, like arrays.
> Having tables with underestimated raw data size makes Hive assign less
> containers (mappers/reducers) to it, making the overall query slower.
> Heavy underestimation also makes Hive choose MapJoin instead of the
> ShuffleJoin that can fail with OOM errors.
> In this patch, I compute the columns data size better, taking into account
> complex structures. I followed the Writer implementation for the ORC format.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20523) Improve table statistics for Parquet format
[ https://issues.apache.org/jira/browse/HIVE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] George Pachitariu updated HIVE-20523: - Status: Open (was: Patch Available) > Improve table statistics for Parquet format > --- > > Key: HIVE-20523 > URL: https://issues.apache.org/jira/browse/HIVE-20523 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Reporter: George Pachitariu >Assignee: George Pachitariu >Priority: Minor > Attachments: HIVE-20523.1.patch, HIVE-20523.2.patch, HIVE-20523.patch > > > Right now, in the table basic statistics, the *raw data size* for a row with > any data type in the Parquet format is 1. This is an underestimated value > when columns are complex data structures, like arrays. > Having tables with underestimated raw data size makes Hive assign less > containers (mappers/reducers) to it, making the overall query slower. > Heavy underestimation also makes Hive choose MapJoin instead of the > ShuffleJoin that can fail with OOM errors. > In this patch, I compute the columns data size better, taking into account > complex structures. I followed the Writer implementation for the ORC format. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18778) Needs to capture input/output entities in explain
[
https://issues.apache.org/jira/browse/HIVE-18778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625252#comment-16625252
]
Hive QA commented on HIVE-18778:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12940976/HIVE-18778.8.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 14995 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_show_grant]
(batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[constant_prop_2]
(batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nonmr_fetch] (batchId=21)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=186)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidkafkamini_basic]
(batchId=194)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test1]
(batchId=194)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[external_jdbc_table]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3]
(batchId=107)
org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver.testCliDriver[spark_task_failure]
(batchId=266)
org.apache.hadoop.hive.ql.exec.spark.TestSparkSessionTimeout.testMultiSparkSessionTimeout
(batchId=245)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14005/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14005/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14005/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12940976 - PreCommit-HIVE-Build
> Needs to capture input/output entities in explain
> -
>
> Key: HIVE-18778
> URL: https://issues.apache.org/jira/browse/HIVE-18778
> Project: Hive
> Issue Type: Bug
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-18778-SparkPositive.patch, HIVE-18778.1.patch,
> HIVE-18778.2.patch, HIVE-18778.3.patch, HIVE-18778.4.patch,
> HIVE-18778.5.patch, HIVE-18778.6.patch, HIVE-18778.7.patch,
> HIVE-18778.8.patch, HIVE-18778_TestCliDriver.patch,
> HIVE-18788_SparkNegative.patch, HIVE-18788_SparkPerf.patch
>
>
> With Sentry enabled, commands like explain drop table foo fail with {{explain
> drop table foo;}}
> {code}
> Error: Error while compiling statement: FAILED: SemanticException No valid
> privileges
> Required privilege( Table) not available in input privileges
> The required privileges: (state=42000,code=4)
> {code}
> Sentry fails to authorize because the ExplainSemanticAnalyzer uses an
> instance of DDLSemanticAnalyzer to analyze the explain query.
> {code}
> BaseSemanticAnalyzer sem = SemanticAnalyzerFactory.get(conf, input);
> sem.analyze(input, ctx);
> sem.validate()
> {code}
> The inputs/outputs entities for this query are set in the above code.
> However, these are never set on the instance of ExplainSemanticAnalyzer
> itself and thus is not propagated into the HookContext in the calling Driver
> code.
> {code}
> sem.analyze(tree, ctx); --> this results in calling the above code that uses
> DDLSA
> hookCtx.update(sem); --> sem is an instance of ExplainSemanticAnalyzer, this
> code attempts to update the HookContext with the input/output info from ESA
> which is never set.
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20612) Create new join multi-key correlation flag for CBO
[ https://issues.apache.org/jira/browse/HIVE-20612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20612: --- Resolution: Fixed Fix Version/s: 3.2.0 4.0.0 Status: Resolved (was: Patch Available) Pushed to master, branch-3. Thanks for reviewing [~ashutoshc] > Create new join multi-key correlation flag for CBO > -- > > Key: HIVE-20612 > URL: https://issues.apache.org/jira/browse/HIVE-20612 > Project: Hive > Issue Type: Bug > Components: CBO >Affects Versions: 4.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-20612.01.patch, HIVE-20612.patch > > > Currently we reuse the flag in Hive side. It would be good to have the flag > separated for debugging purposes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20625) Regex patterns not working in SHOW MATERIALIZED VIEWS ''
[ https://issues.apache.org/jira/browse/HIVE-20625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20625: --- Resolution: Fixed Fix Version/s: 3.2.0 4.0.0 Status: Resolved (was: Patch Available) Pushed to master, branch-3. Thanks for reviewing [~ashutoshc] > Regex patterns not working in SHOW MATERIALIZED VIEWS '' > - > > Key: HIVE-20625 > URL: https://issues.apache.org/jira/browse/HIVE-20625 > Project: Hive > Issue Type: Bug > Components: Materialized views >Reporter: kristine hahn >Assignee: Jesus Camacho Rodriguez >Priority: Major > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-20625.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20523) Improve table statistics for Parquet format
[ https://issues.apache.org/jira/browse/HIVE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] George Pachitariu updated HIVE-20523: - Attachment: HIVE-20523.2.patch Status: Patch Available (was: Open) > Improve table statistics for Parquet format > --- > > Key: HIVE-20523 > URL: https://issues.apache.org/jira/browse/HIVE-20523 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Reporter: George Pachitariu >Assignee: George Pachitariu >Priority: Minor > Attachments: HIVE-20523.1.patch, HIVE-20523.2.patch, HIVE-20523.patch > > > Right now, in the table basic statistics, the *raw data size* for a row with > any data type in the Parquet format is 1. This is an underestimated value > when columns are complex data structures, like arrays. > Having tables with underestimated raw data size makes Hive assign less > containers (mappers/reducers) to it, making the overall query slower. > Heavy underestimation also makes Hive choose MapJoin instead of the > ShuffleJoin that can fail with OOM errors. > In this patch, I compute the columns data size better, taking into account > complex structures. I followed the Writer implementation for the ORC format. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20523) Improve table statistics for Parquet format
[ https://issues.apache.org/jira/browse/HIVE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] George Pachitariu updated HIVE-20523: - Status: Open (was: Patch Available) > Improve table statistics for Parquet format > --- > > Key: HIVE-20523 > URL: https://issues.apache.org/jira/browse/HIVE-20523 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Reporter: George Pachitariu >Assignee: George Pachitariu >Priority: Minor > Attachments: HIVE-20523.1.patch, HIVE-20523.patch > > > Right now, in the table basic statistics, the *raw data size* for a row with > any data type in the Parquet format is 1. This is an underestimated value > when columns are complex data structures, like arrays. > Having tables with underestimated raw data size makes Hive assign less > containers (mappers/reducers) to it, making the overall query slower. > Heavy underestimation also makes Hive choose MapJoin instead of the > ShuffleJoin that can fail with OOM errors. > In this patch, I compute the columns data size better, taking into account > complex structures. I followed the Writer implementation for the ORC format. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18778) Needs to capture input/output entities in explain
[
https://issues.apache.org/jira/browse/HIVE-18778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625242#comment-16625242
]
Hive QA commented on HIVE-18778:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:red}-1{color} | {color:red} @author {color} | {color:red} 0m
0s{color} | {color:red} The patch appears to contain 6 @author tags which the
community has agreed to not allow in code contributions. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
45s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
28s{color} | {color:blue} accumulo-handler in master has 21 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
22s{color} | {color:blue} contrib in master has 13 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
27s{color} | {color:blue} hbase-handler in master has 15 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
44s{color} | {color:blue} ql in master has 2326 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
47s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
49s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 43m 13s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14005/dev-support/hive-personality.sh
|
| git revision | master / 80fd0ec |
| Default Java | 1.8.0_111 |
| @author |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14005/yetus/author-tags.txt |
| findbugs | v3.0.0 |
| modules | C: accumulo-handler contrib hbase-handler itests/hive-blobstore ql
U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14005/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Needs to capture input/output entities in explain
> -
>
> Key: HIVE-18778
> URL: https://issues.apache.org/jira/browse/HIVE-18778
> Project: Hive
> Issue Type: Bug
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-18778-SparkPositive.patch, HIVE-18778.1.patch,
> HIVE-18778.2.patch, HIVE-18778.3.patch, HIVE-18778.4.patch,
> HIVE-18778.5.patch, HIVE-18778.6.patch, HIVE-18778.7.patch,
> HIVE-18778.8.patch, HIVE-18778_TestCliDriver.patch,
> HIVE-18788_SparkNegative.patch, HIVE-18788_SparkPerf.patch
>
>
> With Sentry enabled, commands
[jira] [Commented] (HIVE-20625) Regex patterns not working in SHOW MATERIALIZED VIEWS ''
[
https://issues.apache.org/jira/browse/HIVE-20625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625233#comment-16625233
]
Hive QA commented on HIVE-20625:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12940973/HIVE-20625.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14994 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14004/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14004/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14004/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12940973 - PreCommit-HIVE-Build
> Regex patterns not working in SHOW MATERIALIZED VIEWS ''
> -
>
> Key: HIVE-20625
> URL: https://issues.apache.org/jira/browse/HIVE-20625
> Project: Hive
> Issue Type: Bug
> Components: Materialized views
>Reporter: kristine hahn
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20625.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20625) Regex patterns not working in SHOW MATERIALIZED VIEWS ''
[
https://issues.apache.org/jira/browse/HIVE-20625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625225#comment-16625225
]
Hive QA commented on HIVE-20625:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
50s{color} | {color:blue} ql in master has 2326 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 7s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14004/dev-support/hive-personality.sh
|
| git revision | master / 80fd0ec |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14004/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Regex patterns not working in SHOW MATERIALIZED VIEWS ''
> -
>
> Key: HIVE-20625
> URL: https://issues.apache.org/jira/browse/HIVE-20625
> Project: Hive
> Issue Type: Bug
> Components: Materialized views
>Reporter: kristine hahn
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20625.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20625) Regex patterns not working in SHOW MATERIALIZED VIEWS ''
[ https://issues.apache.org/jira/browse/HIVE-20625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625222#comment-16625222 ] Ashutosh Chauhan commented on HIVE-20625: - +1 pending tests > Regex patterns not working in SHOW MATERIALIZED VIEWS '' > - > > Key: HIVE-20625 > URL: https://issues.apache.org/jira/browse/HIVE-20625 > Project: Hive > Issue Type: Bug > Components: Materialized views >Reporter: kristine hahn >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20625.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20612) Create new join multi-key correlation flag for CBO
[ https://issues.apache.org/jira/browse/HIVE-20612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625216#comment-16625216 ] Ashutosh Chauhan commented on HIVE-20612: - +1 > Create new join multi-key correlation flag for CBO > -- > > Key: HIVE-20612 > URL: https://issues.apache.org/jira/browse/HIVE-20612 > Project: Hive > Issue Type: Bug > Components: CBO >Affects Versions: 4.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20612.01.patch, HIVE-20612.patch > > > Currently we reuse the flag in Hive side. It would be good to have the flag > separated for debugging purposes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20599) CAST(INTERVAL_DAY_TIME AS STRING) is throwing SemanticException
[
https://issues.apache.org/jira/browse/HIVE-20599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625213#comment-16625213
]
Hive QA commented on HIVE-20599:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12940955/HIVE-20599.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14994 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.miniHS2.TestHs2ConnectionMetricsBinary.testOpenConnectionMetrics
(batchId=255)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14003/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14003/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14003/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12940955 - PreCommit-HIVE-Build
> CAST(INTERVAL_DAY_TIME AS STRING) is throwing SemanticException
> ---
>
> Key: HIVE-20599
> URL: https://issues.apache.org/jira/browse/HIVE-20599
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 3.1.0
>Reporter: Naresh P R
>Assignee: Naresh P R
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-20599-branch-3.patch,
> HIVE-20599.1-branch-3.1.patch, HIVE-20599.1-branch-3.patch,
> HIVE-20599.1.patch, HIVE-20599.2.patch
>
>
> SELECT CAST(from_utc_timestamp(timestamp '2018-05-02 15:30:30', 'PST') -
> from_utc_timestamp(timestamp '1970-01-30 16:00:00', 'PST') AS STRING);
> throws below Exception
> {code:java}
> Error: Error while compiling statement: FAILED: SemanticException Line 0:-1
> Wrong arguments ''PST'': No matching method for class
> org.apache.hadoop.hive.ql.udf.UDFToString with (interval_day_time). Possible
> choices: _FUNC_(bigint) _FUNC_(binary) _FUNC_(boolean) _FUNC_(date)
> _FUNC_(decimal(38,18)) _FUNC_(double) _FUNC_(float) _FUNC_(int)
> _FUNC_(smallint) _FUNC_(string) _FUNC_(timestamp) _FUNC_(tinyint)
> _FUNC_(void) (state=42000,code=4){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18778) Needs to capture input/output entities in explain
[
https://issues.apache.org/jira/browse/HIVE-18778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated HIVE-18778:
--
Attachment: HIVE-18778.8.patch
> Needs to capture input/output entities in explain
> -
>
> Key: HIVE-18778
> URL: https://issues.apache.org/jira/browse/HIVE-18778
> Project: Hive
> Issue Type: Bug
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-18778-SparkPositive.patch, HIVE-18778.1.patch,
> HIVE-18778.2.patch, HIVE-18778.3.patch, HIVE-18778.4.patch,
> HIVE-18778.5.patch, HIVE-18778.6.patch, HIVE-18778.7.patch,
> HIVE-18778.8.patch, HIVE-18778_TestCliDriver.patch,
> HIVE-18788_SparkNegative.patch, HIVE-18788_SparkPerf.patch
>
>
> With Sentry enabled, commands like explain drop table foo fail with {{explain
> drop table foo;}}
> {code}
> Error: Error while compiling statement: FAILED: SemanticException No valid
> privileges
> Required privilege( Table) not available in input privileges
> The required privileges: (state=42000,code=4)
> {code}
> Sentry fails to authorize because the ExplainSemanticAnalyzer uses an
> instance of DDLSemanticAnalyzer to analyze the explain query.
> {code}
> BaseSemanticAnalyzer sem = SemanticAnalyzerFactory.get(conf, input);
> sem.analyze(input, ctx);
> sem.validate()
> {code}
> The inputs/outputs entities for this query are set in the above code.
> However, these are never set on the instance of ExplainSemanticAnalyzer
> itself and thus is not propagated into the HookContext in the calling Driver
> code.
> {code}
> sem.analyze(tree, ctx); --> this results in calling the above code that uses
> DDLSA
> hookCtx.update(sem); --> sem is an instance of ExplainSemanticAnalyzer, this
> code attempts to update the HookContext with the input/output info from ESA
> which is never set.
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20625) Regex patterns not working in SHOW MATERIALIZED VIEWS ''
[ https://issues.apache.org/jira/browse/HIVE-20625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20625: --- Attachment: HIVE-20625.patch > Regex patterns not working in SHOW MATERIALIZED VIEWS '' > - > > Key: HIVE-20625 > URL: https://issues.apache.org/jira/browse/HIVE-20625 > Project: Hive > Issue Type: Bug > Components: Materialized views >Reporter: kristine hahn >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20625.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HIVE-20625) Regex patterns not working in SHOW MATERIALIZED VIEWS ''
[ https://issues.apache.org/jira/browse/HIVE-20625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-20625 started by Jesus Camacho Rodriguez. -- > Regex patterns not working in SHOW MATERIALIZED VIEWS '' > - > > Key: HIVE-20625 > URL: https://issues.apache.org/jira/browse/HIVE-20625 > Project: Hive > Issue Type: Bug > Components: Materialized views >Reporter: kristine hahn >Assignee: Jesus Camacho Rodriguez >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20625) Regex patterns not working in SHOW MATERIALIZED VIEWS ''
[ https://issues.apache.org/jira/browse/HIVE-20625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20625: --- Status: Patch Available (was: In Progress) > Regex patterns not working in SHOW MATERIALIZED VIEWS '' > - > > Key: HIVE-20625 > URL: https://issues.apache.org/jira/browse/HIVE-20625 > Project: Hive > Issue Type: Bug > Components: Materialized views >Reporter: kristine hahn >Assignee: Jesus Camacho Rodriguez >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-20625) Regex patterns not working in SHOW MATERIALIZED VIEWS ''
[ https://issues.apache.org/jira/browse/HIVE-20625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez reassigned HIVE-20625: -- > Regex patterns not working in SHOW MATERIALIZED VIEWS '' > - > > Key: HIVE-20625 > URL: https://issues.apache.org/jira/browse/HIVE-20625 > Project: Hive > Issue Type: Bug > Components: Materialized views >Reporter: kristine hahn >Assignee: Jesus Camacho Rodriguez >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20599) CAST(INTERVAL_DAY_TIME AS STRING) is throwing SemanticException
[
https://issues.apache.org/jira/browse/HIVE-20599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625180#comment-16625180
]
Hive QA commented on HIVE-20599:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
54s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 1m 31s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14003/dev-support/hive-personality.sh
|
| git revision | master / 80fd0ec |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14003/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> CAST(INTERVAL_DAY_TIME AS STRING) is throwing SemanticException
> ---
>
> Key: HIVE-20599
> URL: https://issues.apache.org/jira/browse/HIVE-20599
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 3.1.0
>Reporter: Naresh P R
>Assignee: Naresh P R
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-20599-branch-3.patch,
> HIVE-20599.1-branch-3.1.patch, HIVE-20599.1-branch-3.patch,
> HIVE-20599.1.patch, HIVE-20599.2.patch
>
>
> SELECT CAST(from_utc_timestamp(timestamp '2018-05-02 15:30:30', 'PST') -
> from_utc_timestamp(timestamp '1970-01-30 16:00:00', 'PST') AS STRING);
> throws below Exception
> {code:java}
> Error: Error while compiling statement: FAILED: SemanticException Line 0:-1
> Wrong arguments ''PST'': No matching method for class
> org.apache.hadoop.hive.ql.udf.UDFToString with (interval_day_time). Possible
> choices: _FUNC_(bigint) _FUNC_(binary) _FUNC_(boolean) _FUNC_(date)
> _FUNC_(decimal(38,18)) _FUNC_(double) _FUNC_(float) _FUNC_(int)
> _FUNC_(smallint) _FUNC_(string) _FUNC_(timestamp) _FUNC_(tinyint)
> _FUNC_(void) (state=42000,code=4){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18778) Needs to capture input/output entities in explain
[
https://issues.apache.org/jira/browse/HIVE-18778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625179#comment-16625179
]
Hive QA commented on HIVE-18778:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12940952/HIVE-18778.7.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 35 failed/errored test(s), 14992 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=194)
[druidmini_masking.q,druidmini_test1.q,druidkafkamini_basic.q,druidmini_joins.q,druid_timestamptz.q]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_show_grant]
(batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[constant_prop_2]
(batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nonmr_fetch] (batchId=21)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=186)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_dynamic_partition]
(batchId=193)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_expressions]
(batchId=193)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_extractTime]
(batchId=192)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_floorTime]
(batchId=192)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_mv]
(batchId=192)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test_insert]
(batchId=193)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test_ts]
(batchId=193)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3]
(batchId=107)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[groupby9]
(batchId=112)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[groupby_complex_types]
(batchId=140)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[groupby_complex_types_multi_single_reducer]
(batchId=150)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[groupby_cube1]
(batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[groupby_map_ppr]
(batchId=113)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[groupby_map_ppr_multi_distinct]
(batchId=132)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[groupby_multi_insert_common_distinct]
(batchId=143)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[groupby_multi_single_reducer2]
(batchId=118)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[groupby_multi_single_reducer3]
(batchId=131)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_7]
(batchId=148)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_8]
(batchId=115)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_9]
(batchId=123)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_decimal_date]
(batchId=123)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_div0]
(batchId=145)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_limit]
(batchId=120)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_offset_limit]
(batchId=124)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_part_project]
(batchId=125)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_pushdown]
(batchId=124)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union] (batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[windowing]
(batchId=133)
org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver.testCliDriver[spark_task_failure]
(batchId=266)
org.apache.hive.jdbc.TestJdbcWithMiniLlapArrow.testKillQuery (batchId=251)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14002/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14002/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14002/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 35 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12940952 - PreCommit-HIVE-Build
> Needs to capture input/output entities in explain
> -
>
> Key: HIVE-18778
>
[jira] [Commented] (HIVE-18778) Needs to capture input/output entities in explain
[
https://issues.apache.org/jira/browse/HIVE-18778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625170#comment-16625170
]
Hive QA commented on HIVE-18778:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:red}-1{color} | {color:red} @author {color} | {color:red} 0m
0s{color} | {color:red} The patch appears to contain 6 @author tags which the
community has agreed to not allow in code contributions. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
44s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
29s{color} | {color:blue} accumulo-handler in master has 21 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
23s{color} | {color:blue} contrib in master has 13 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
33s{color} | {color:blue} hbase-handler in master has 15 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
55s{color} | {color:blue} ql in master has 2326 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 43m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14002/dev-support/hive-personality.sh
|
| git revision | master / 80fd0ec |
| Default Java | 1.8.0_111 |
| @author |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14002/yetus/author-tags.txt |
| findbugs | v3.0.0 |
| modules | C: accumulo-handler contrib hbase-handler itests/hive-blobstore ql
U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14002/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Needs to capture input/output entities in explain
> -
>
> Key: HIVE-18778
> URL: https://issues.apache.org/jira/browse/HIVE-18778
> Project: Hive
> Issue Type: Bug
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-18778-SparkPositive.patch, HIVE-18778.1.patch,
> HIVE-18778.2.patch, HIVE-18778.3.patch, HIVE-18778.4.patch,
> HIVE-18778.5.patch, HIVE-18778.6.patch, HIVE-18778.7.patch,
> HIVE-18778_TestCliDriver.patch, HIVE-18788_SparkNegative.patch,
> HIVE-18788_SparkPerf.patch
>
>
> With Sentry enabled, commands like explain drop
[jira] [Updated] (HIVE-20095) Fix feature to push computation to jdbc external tables
[ https://issues.apache.org/jira/browse/HIVE-20095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20095: --- Resolution: Fixed Fix Version/s: 3.2.0 4.0.0 Status: Resolved (was: Patch Available) Pushed to master, branch-3. Thanks [~msydoron] > Fix feature to push computation to jdbc external tables > --- > > Key: HIVE-20095 > URL: https://issues.apache.org/jira/browse/HIVE-20095 > Project: Hive > Issue Type: Bug >Reporter: Jonathan Doron >Assignee: Jonathan Doron >Priority: Major > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-20095.1.patch, HIVE-20095.10.patch, > HIVE-20095.2.patch, HIVE-20095.3.patch, HIVE-20095.4.patch, > HIVE-20095.5.patch, HIVE-20095.6.patch, HIVE-20095.7.patch, > HIVE-20095.7.patch, HIVE-20095.8.patch, HIVE-20095.8.patch, HIVE-20095.9.patch > > > It seems like the committed code for HIVE-19161 > (7584b3276bebf64aa006eaa162c0a6264d8fcb56) reverted some of HIVE-18423 > updates, and therefore some of the external table queries are not working > correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20612) Create new join multi-key correlation flag for CBO
[ https://issues.apache.org/jira/browse/HIVE-20612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625149#comment-16625149 ] Jesus Camacho Rodriguez commented on HIVE-20612: [~ashutoshc], could you take a look? Thanks > Create new join multi-key correlation flag for CBO > -- > > Key: HIVE-20612 > URL: https://issues.apache.org/jira/browse/HIVE-20612 > Project: Hive > Issue Type: Bug > Components: CBO >Affects Versions: 4.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20612.01.patch, HIVE-20612.patch > > > Currently we reuse the flag in Hive side. It would be good to have the flag > separated for debugging purposes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20095) Fix feature to push computation to jdbc external tables
[ https://issues.apache.org/jira/browse/HIVE-20095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20095: --- Summary: Fix feature to push computation to jdbc external tables (was: Fix jdbc external table feature) > Fix feature to push computation to jdbc external tables > --- > > Key: HIVE-20095 > URL: https://issues.apache.org/jira/browse/HIVE-20095 > Project: Hive > Issue Type: Bug >Reporter: Jonathan Doron >Assignee: Jonathan Doron >Priority: Major > Attachments: HIVE-20095.1.patch, HIVE-20095.10.patch, > HIVE-20095.2.patch, HIVE-20095.3.patch, HIVE-20095.4.patch, > HIVE-20095.5.patch, HIVE-20095.6.patch, HIVE-20095.7.patch, > HIVE-20095.7.patch, HIVE-20095.8.patch, HIVE-20095.8.patch, HIVE-20095.9.patch > > > It seems like the committed code for HIVE-19161 > (7584b3276bebf64aa006eaa162c0a6264d8fcb56) reverted some of HIVE-18423 > updates, and therefore some of the external table queries are not working > correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20623) Shared work: Extend sharing of map-join cache entries in LLAP
[
https://issues.apache.org/jira/browse/HIVE-20623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625146#comment-16625146
]
Hive QA commented on HIVE-20623:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12940945/HIVE-20623.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 14993 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.metadata.TestHiveRemote.testPartition (batchId=296)
org.apache.hive.jdbc.TestJdbcWithMiniLlapArrow.testKillQuery (batchId=251)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14001/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14001/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14001/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12940945 - PreCommit-HIVE-Build
> Shared work: Extend sharing of map-join cache entries in LLAP
> -
>
> Key: HIVE-20623
> URL: https://issues.apache.org/jira/browse/HIVE-20623
> Project: Hive
> Issue Type: Improvement
> Components: llap, Logical Optimizer
>Reporter: Gopal V
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20623.patch, hash-shared-work.json.txt,
> hash-shared-work.svg
>
>
> For a query like this
> {code}
> with all_sales as (
> select ss_customer_sk as customer_sk, ss_ext_list_price-ss_ext_discount_amt
> as ext_price from store_sales
> UNION ALL
> select ws_bill_customer_sk as customer_sk,
> ws_ext_list_price-ws_ext_discount_amt as ext_price from web_sales
> UNION ALL
> select cs_bill_customer_sk as customer_sk, cs_ext_sales_price -
> cs_ext_discount_amt as ext_price from catalog_sales)
> select sum(ext_price) total_price, c_customer_id from all_sales, customer
> where customer_sk = c_customer_sk
> group by c_customer_id
> order by total_price desc
> limit 100;
> {code}
> The hashtable used for all 3 joins are identical, which is loaded 3x times in
> the same LLAP instance because they are named.
> {code}
> cacheKey = "HASH_MAP_" + this.getOperatorId() + "_container";
> {code}
> in the cache.
> If those are identical in nature (i.e vectorization, hashtable type etc),
> then the duplication is just wasted CPU, memory and network - using the cache
> name for hashtables which will be identical in layout would be extremely
> useful.
> In cases where the join is pushed through a UNION, those are identical.
> This optimization can only be done without concern for accidental delays when
> the same upstream task is generating all of these hashtables, which is what
> is achieved by the shared scan optimizer already.
> In case the shared work is not present, this has potential downsides - in
> case two customer broadcasts were sourced from "Map 1" and "Map 2", the Map 1
> builder will block the other task from reading from Map 2, even though Map 2
> might have started after, but finished ahead of Map 1.
> So this specific optimization can always be considered for cases where the
> shared work unifies the operator tree and the parents of all the RS entries
> involved are same (& the RS layout is the same).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20623) Shared work: Extend sharing of map-join cache entries in LLAP
[
https://issues.apache.org/jira/browse/HIVE-20623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625130#comment-16625130
]
Hive QA commented on HIVE-20623:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
39s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
56s{color} | {color:blue} ql in master has 2326 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
38s{color} | {color:red} ql: The patch generated 1 new + 42 unchanged - 5 fixed
= 43 total (was 47) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 37s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14001/dev-support/hive-personality.sh
|
| git revision | master / c44f2b5 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14001/yetus/diff-checkstyle-ql.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14001/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Shared work: Extend sharing of map-join cache entries in LLAP
> -
>
> Key: HIVE-20623
> URL: https://issues.apache.org/jira/browse/HIVE-20623
> Project: Hive
> Issue Type: Improvement
> Components: llap, Logical Optimizer
>Reporter: Gopal V
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20623.patch, hash-shared-work.json.txt,
> hash-shared-work.svg
>
>
> For a query like this
> {code}
> with all_sales as (
> select ss_customer_sk as customer_sk, ss_ext_list_price-ss_ext_discount_amt
> as ext_price from store_sales
> UNION ALL
> select ws_bill_customer_sk as customer_sk,
> ws_ext_list_price-ws_ext_discount_amt as ext_price from web_sales
> UNION ALL
> select cs_bill_customer_sk as customer_sk, cs_ext_sales_price -
> cs_ext_discount_amt as ext_price from catalog_sales)
> select sum(ext_price) total_price, c_customer_id from all_sales, customer
> where customer_sk = c_customer_sk
> group by c_customer_id
[jira] [Commented] (HIVE-20535) Add new configuration to set the size of the global compile lock
[
https://issues.apache.org/jira/browse/HIVE-20535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625120#comment-16625120
]
Hive QA commented on HIVE-20535:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12940935/HIVE-20535.13.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14993 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14000/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14000/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14000/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12940935 - PreCommit-HIVE-Build
> Add new configuration to set the size of the global compile lock
>
>
> Key: HIVE-20535
> URL: https://issues.apache.org/jira/browse/HIVE-20535
> Project: Hive
> Issue Type: Task
> Components: HiveServer2
>Reporter: denys kuzmenko
>Assignee: denys kuzmenko
>Priority: Major
> Attachments: HIVE-20535.1.patch, HIVE-20535.10.patch,
> HIVE-20535.11.patch, HIVE-20535.12.patch, HIVE-20535.13.patch,
> HIVE-20535.2.patch, HIVE-20535.3.patch, HIVE-20535.4.patch,
> HIVE-20535.5.patch, HIVE-20535.6.patch, HIVE-20535.8.patch, HIVE-20535.9.patch
>
>
> When removing the compile lock, it is quite risky to remove it entirely.
> It would be good to provide a pool size for the concurrent compilation, so
> the administrator can limit the load
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20535) Add new configuration to set the size of the global compile lock
[
https://issues.apache.org/jira/browse/HIVE-20535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625108#comment-16625108
]
Hive QA commented on HIVE-20535:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
36s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
32s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
52s{color} | {color:blue} ql in master has 2326 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
40s{color} | {color:red} ql: The patch generated 1 new + 142 unchanged - 6
fixed = 143 total (was 148) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14000/dev-support/hive-personality.sh
|
| git revision | master / c44f2b5 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14000/yetus/diff-checkstyle-ql.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14000/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add new configuration to set the size of the global compile lock
>
>
> Key: HIVE-20535
> URL: https://issues.apache.org/jira/browse/HIVE-20535
> Project: Hive
> Issue Type: Task
> Components: HiveServer2
>Reporter: denys kuzmenko
>Assignee: denys kuzmenko
>Priority: Major
> Attachments: HIVE-20535.1.patch, HIVE-20535.10.patch,
> HIVE-20535.11.patch, HIVE-20535.12.patch, HIVE-20535.13.patch,
> HIVE-20535.2.patch, HIVE-20535.3.patch, HIVE-20535.4.patch,
> HIVE-20535.5.patch, HIVE-20535.6.patch, HIVE-20535.8.patch, HIVE-20535.9.patch
>
>
> When removing the compile lock, it is quite risky to remove it entirely.
> It would be good to provide a pool size for the concurrent compilation, so
> the administrator can limit the load
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20095) Fix jdbc external table feature
[
https://issues.apache.org/jira/browse/HIVE-20095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625101#comment-16625101
]
Hive QA commented on HIVE-20095:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12940930/HIVE-20095.10.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14994 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/13999/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13999/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13999/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12940930 - PreCommit-HIVE-Build
> Fix jdbc external table feature
> ---
>
> Key: HIVE-20095
> URL: https://issues.apache.org/jira/browse/HIVE-20095
> Project: Hive
> Issue Type: Bug
>Reporter: Jonathan Doron
>Assignee: Jonathan Doron
>Priority: Major
> Attachments: HIVE-20095.1.patch, HIVE-20095.10.patch,
> HIVE-20095.2.patch, HIVE-20095.3.patch, HIVE-20095.4.patch,
> HIVE-20095.5.patch, HIVE-20095.6.patch, HIVE-20095.7.patch,
> HIVE-20095.7.patch, HIVE-20095.8.patch, HIVE-20095.8.patch, HIVE-20095.9.patch
>
>
> It seems like the committed code for HIVE-19161
> (7584b3276bebf64aa006eaa162c0a6264d8fcb56) reverted some of HIVE-18423
> updates, and therefore some of the external table queries are not working
> correctly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20095) Fix jdbc external table feature
[
https://issues.apache.org/jira/browse/HIVE-20095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625091#comment-16625091
]
Hive QA commented on HIVE-20095:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
23s{color} | {color:blue} jdbc-handler in master has 8 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
54s{color} | {color:blue} ql in master has 2326 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
10s{color} | {color:red} jdbc-handler: The patch generated 2 new + 24 unchanged
- 1 fixed = 26 total (was 25) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 4 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
29s{color} | {color:red} jdbc-handler generated 3 new + 8 unchanged - 0 fixed =
11 total (was 8) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 42s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:jdbc-handler |
| | Exception is caught when Exception is not thrown in
org.apache.hive.storage.jdbc.JdbcSerDe.initialize(Configuration, Properties)
At JdbcSerDe.java:is not thrown in
org.apache.hive.storage.jdbc.JdbcSerDe.initialize(Configuration, Properties)
At JdbcSerDe.java:[line 114] |
| |
org.apache.hive.storage.jdbc.dao.GenericJdbcDatabaseAccessor.getColumnTypes(Configuration)
may fail to clean up java.sql.ResultSet Obligation to clean up resource
created at GenericJdbcDatabaseAccessor.java:up java.sql.ResultSet Obligation
to clean up resource created at GenericJdbcDatabaseAccessor.java:[line 115] is
not discharged |
| |
org.apache.hive.storage.jdbc.dao.GenericJdbcDatabaseAccessor.getColumnTypes(Configuration)
may fail to clean up java.sql.Statement Obligation to clean up resource
created at GenericJdbcDatabaseAccessor.java:up java.sql.Statement Obligation
to clean up resource created at GenericJdbcDatabaseAccessor.java:[line 114] is
not discharged |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-13999/dev-support/hive-personality.sh
|
| git revision | master / c44f2b5 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13999/yetus/diff-checkstyle-jdbc-handler.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13999/yetus/whitespace-eol.txt
|
| findbugs |
[jira] [Commented] (HIVE-20612) Create new join multi-key correlation flag for CBO
[
https://issues.apache.org/jira/browse/HIVE-20612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625079#comment-16625079
]
Hive QA commented on HIVE-20612:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12940925/HIVE-20612.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14993 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/13998/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13998/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13998/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12940925 - PreCommit-HIVE-Build
> Create new join multi-key correlation flag for CBO
> --
>
> Key: HIVE-20612
> URL: https://issues.apache.org/jira/browse/HIVE-20612
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20612.01.patch, HIVE-20612.patch
>
>
> Currently we reuse the flag in Hive side. It would be good to have the flag
> separated for debugging purposes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20612) Create new join multi-key correlation flag for CBO
[
https://issues.apache.org/jira/browse/HIVE-20612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625068#comment-16625068
]
Hive QA commented on HIVE-20612:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
32s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
48s{color} | {color:blue} ql in master has 2326 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
39s{color} | {color:red} ql: The patch generated 1 new + 169 unchanged - 0
fixed = 170 total (was 169) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 45s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-13998/dev-support/hive-personality.sh
|
| git revision | master / c44f2b5 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13998/yetus/diff-checkstyle-ql.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13998/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Create new join multi-key correlation flag for CBO
> --
>
> Key: HIVE-20612
> URL: https://issues.apache.org/jira/browse/HIVE-20612
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20612.01.patch, HIVE-20612.patch
>
>
> Currently we reuse the flag in Hive side. It would be good to have the flag
> separated for debugging purposes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20523) Improve table statistics for Parquet format
[
https://issues.apache.org/jira/browse/HIVE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625059#comment-16625059
]
Hive QA commented on HIVE-20523:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12940906/HIVE-20523.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 96 failed/errored test(s), 14994 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nested_column_pruning]
(batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_analyze]
(batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_array_map_emptynullvals]
(batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_array_null_element]
(batchId=79)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_complex_types_vectorization]
(batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_create]
(batchId=94)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_decimal1]
(batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_join]
(batchId=21)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_map_null]
(batchId=90)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_map_of_arrays_of_ints]
(batchId=10)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_map_of_maps]
(batchId=70)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_map_type_vectorization]
(batchId=89)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_nested_complex]
(batchId=94)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_no_row_serde]
(batchId=74)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_read_backward_compatible_files]
(batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_schema_evolution]
(batchId=80)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_struct_type_vectorization]
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_type_promotion]
(batchId=90)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_types_non_dictionary_encoding_vectorization]
(batchId=91)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_types_vectorization]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_0]
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_10]
(batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_11]
(batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_12]
(batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_13]
(batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_14]
(batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_15]
(batchId=92)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_16]
(batchId=87)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_17]
(batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_1]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_2]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_3]
(batchId=82)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_4]
(batchId=46)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_5]
(batchId=75)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_6]
(batchId=44)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_7]
(batchId=90)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_8]
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_9]
(batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_decimal_date]
(batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_div0]
(batchId=82)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_limit]
(batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_nested_udf]
(batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_not]
(batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_offset_limit]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_part]
(batchId=76)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_part_project]
(batchId=37)
[jira] [Commented] (HIVE-20523) Improve table statistics for Parquet format
[
https://issues.apache.org/jira/browse/HIVE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625048#comment-16625048
]
Hive QA commented on HIVE-20523:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
53s{color} | {color:blue} ql in master has 2326 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
36s{color} | {color:red} ql: The patch generated 51 new + 6 unchanged - 0 fixed
= 57 total (was 6) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 14s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-13997/dev-support/hive-personality.sh
|
| git revision | master / c44f2b5 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13997/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13997/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Improve table statistics for Parquet format
> ---
>
> Key: HIVE-20523
> URL: https://issues.apache.org/jira/browse/HIVE-20523
> Project: Hive
> Issue Type: Improvement
> Components: Physical Optimizer
>Reporter: George Pachitariu
>Assignee: George Pachitariu
>Priority: Minor
> Attachments: HIVE-20523.1.patch, HIVE-20523.patch
>
>
> Right now, in the table basic statistics, the *raw data size* for a row with
> any data type in the Parquet format is 1. This is an underestimated value
> when columns are complex data structures, like arrays.
> Having tables with underestimated raw data size makes Hive assign less
> containers (mappers/reducers) to it, making the overall query slower.
> Heavy underestimation also makes Hive choose MapJoin instead of the
> ShuffleJoin that can fail with OOM errors.
> In this patch, I compute the columns data size better, taking into account
> complex structures. I followed the Writer implementation for the ORC format.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20607) TxnHandler should use PreparedStatement to execute direct SQL queries.
[
https://issues.apache.org/jira/browse/HIVE-20607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625043#comment-16625043
]
Hive QA commented on HIVE-20607:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12940890/HIVE-20607.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 14993 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestCatalogNonDefaultClient.listPartitions
(batchId=219)
org.apache.hive.jdbc.TestJdbcWithMiniLlapArrow.testKillQuery (batchId=251)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/13996/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13996/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13996/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12940890 - PreCommit-HIVE-Build
> TxnHandler should use PreparedStatement to execute direct SQL queries.
> --
>
> Key: HIVE-20607
> URL: https://issues.apache.org/jira/browse/HIVE-20607
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore, Transactions
>Affects Versions: 4.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
> Labels: ACID, pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-20607.01.patch
>
>
> TxnHandler uses direct SQL queries to operate on Txn related databases/tables
> in Hive metastore RDBMS.
> Most of the methods are direct calls from Metastore api which should be
> directly append input string arguments to the SQL string.
> Need to use parameterised PreparedStatement object to set these arguments.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20607) TxnHandler should use PreparedStatement to execute direct SQL queries.
[
https://issues.apache.org/jira/browse/HIVE-20607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625029#comment-16625029
]
Hive QA commented on HIVE-20607:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
36s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
25s{color} | {color:blue} hcatalog/server-extensions in master has 2 extant
Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
50s{color} | {color:blue} ql in master has 2326 extant Findbugs warnings.
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
13s{color} | {color:red} metastore-server in master failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
26s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
17s{color} | {color:red} server-extensions in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
17s{color} | {color:red} server-extensions in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 17s{color}
| {color:red} server-extensions in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
16s{color} | {color:red} server-extensions in the patch failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
13s{color} | {color:red} metastore-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 19s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-13996/dev-support/hive-personality.sh
|
| git revision | master / cdba00c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13996/yetus/branch-findbugs-standalone-metastore_metastore-server.txt
|
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13996/yetus/patch-mvninstall-hcatalog_server-extensions.txt
|
| compile |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13996/yetus/patch-compile-hcatalog_server-extensions.txt
|
| javac |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13996/yetus/patch-compile-hcatalog_server-extensions.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13996/yetus/patch-findbugs-hcatalog_server-extensions.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13996/yetus/patch-findbugs-standalone-metastore_metastore-server.txt
|
| modules | C: hcatalog/server-extensions ql
standalone-metastore/metastore-server U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13996/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> TxnHandler should use PreparedStatement to execute direct SQL queries.
[jira] [Assigned] (HIVE-20599) CAST(INTERVAL_DAY_TIME AS STRING) is throwing SemanticException
[
https://issues.apache.org/jira/browse/HIVE-20599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran reassigned HIVE-20599:
Assignee: Naresh P R (was: Prasanth Jayachandran)
> CAST(INTERVAL_DAY_TIME AS STRING) is throwing SemanticException
> ---
>
> Key: HIVE-20599
> URL: https://issues.apache.org/jira/browse/HIVE-20599
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 3.1.0
>Reporter: Naresh P R
>Assignee: Naresh P R
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-20599-branch-3.patch,
> HIVE-20599.1-branch-3.1.patch, HIVE-20599.1-branch-3.patch,
> HIVE-20599.1.patch, HIVE-20599.2.patch
>
>
> SELECT CAST(from_utc_timestamp(timestamp '2018-05-02 15:30:30', 'PST') -
> from_utc_timestamp(timestamp '1970-01-30 16:00:00', 'PST') AS STRING);
> throws below Exception
> {code:java}
> Error: Error while compiling statement: FAILED: SemanticException Line 0:-1
> Wrong arguments ''PST'': No matching method for class
> org.apache.hadoop.hive.ql.udf.UDFToString with (interval_day_time). Possible
> choices: _FUNC_(bigint) _FUNC_(binary) _FUNC_(boolean) _FUNC_(date)
> _FUNC_(decimal(38,18)) _FUNC_(double) _FUNC_(float) _FUNC_(int)
> _FUNC_(smallint) _FUNC_(string) _FUNC_(timestamp) _FUNC_(tinyint)
> _FUNC_(void) (state=42000,code=4){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20599) CAST(INTERVAL_DAY_TIME AS STRING) is throwing SemanticException
[
https://issues.apache.org/jira/browse/HIVE-20599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625027#comment-16625027
]
Prasanth Jayachandran commented on HIVE-20599:
--
Uploaded .1 as .2 master patch which is just test change. Hopefully it should
get green run for commit.
> CAST(INTERVAL_DAY_TIME AS STRING) is throwing SemanticException
> ---
>
> Key: HIVE-20599
> URL: https://issues.apache.org/jira/browse/HIVE-20599
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 3.1.0
>Reporter: Naresh P R
>Assignee: Prasanth Jayachandran
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-20599-branch-3.patch,
> HIVE-20599.1-branch-3.1.patch, HIVE-20599.1-branch-3.patch,
> HIVE-20599.1.patch, HIVE-20599.2.patch
>
>
> SELECT CAST(from_utc_timestamp(timestamp '2018-05-02 15:30:30', 'PST') -
> from_utc_timestamp(timestamp '1970-01-30 16:00:00', 'PST') AS STRING);
> throws below Exception
> {code:java}
> Error: Error while compiling statement: FAILED: SemanticException Line 0:-1
> Wrong arguments ''PST'': No matching method for class
> org.apache.hadoop.hive.ql.udf.UDFToString with (interval_day_time). Possible
> choices: _FUNC_(bigint) _FUNC_(binary) _FUNC_(boolean) _FUNC_(date)
> _FUNC_(decimal(38,18)) _FUNC_(double) _FUNC_(float) _FUNC_(int)
> _FUNC_(smallint) _FUNC_(string) _FUNC_(timestamp) _FUNC_(tinyint)
> _FUNC_(void) (state=42000,code=4){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-20599) CAST(INTERVAL_DAY_TIME AS STRING) is throwing SemanticException
[
https://issues.apache.org/jira/browse/HIVE-20599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran reassigned HIVE-20599:
Assignee: Prasanth Jayachandran (was: Naresh P R)
> CAST(INTERVAL_DAY_TIME AS STRING) is throwing SemanticException
> ---
>
> Key: HIVE-20599
> URL: https://issues.apache.org/jira/browse/HIVE-20599
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 3.1.0
>Reporter: Naresh P R
>Assignee: Prasanth Jayachandran
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-20599-branch-3.patch,
> HIVE-20599.1-branch-3.1.patch, HIVE-20599.1-branch-3.patch,
> HIVE-20599.1.patch, HIVE-20599.2.patch
>
>
> SELECT CAST(from_utc_timestamp(timestamp '2018-05-02 15:30:30', 'PST') -
> from_utc_timestamp(timestamp '1970-01-30 16:00:00', 'PST') AS STRING);
> throws below Exception
> {code:java}
> Error: Error while compiling statement: FAILED: SemanticException Line 0:-1
> Wrong arguments ''PST'': No matching method for class
> org.apache.hadoop.hive.ql.udf.UDFToString with (interval_day_time). Possible
> choices: _FUNC_(bigint) _FUNC_(binary) _FUNC_(boolean) _FUNC_(date)
> _FUNC_(decimal(38,18)) _FUNC_(double) _FUNC_(float) _FUNC_(int)
> _FUNC_(smallint) _FUNC_(string) _FUNC_(timestamp) _FUNC_(tinyint)
> _FUNC_(void) (state=42000,code=4){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20599) CAST(INTERVAL_DAY_TIME AS STRING) is throwing SemanticException
[
https://issues.apache.org/jira/browse/HIVE-20599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625026#comment-16625026
]
Prasanth Jayachandran commented on HIVE-20599:
--
[~nareshpr] branch-3 failures seems to be already existing failures
[https://builds.apache.org/view/H-L/view/Hive/job/PreCommit-HIVE-Build/13969/#showFailuresLink]
[https://builds.apache.org/view/H-L/view/Hive/job/PreCommit-HIVE-Build/13968/#showFailuresLink]
The above test report from some other patches are also showing similar
failures.
> CAST(INTERVAL_DAY_TIME AS STRING) is throwing SemanticException
> ---
>
> Key: HIVE-20599
> URL: https://issues.apache.org/jira/browse/HIVE-20599
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 3.1.0
>Reporter: Naresh P R
>Assignee: Naresh P R
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-20599-branch-3.patch,
> HIVE-20599.1-branch-3.1.patch, HIVE-20599.1-branch-3.patch,
> HIVE-20599.1.patch, HIVE-20599.2.patch
>
>
> SELECT CAST(from_utc_timestamp(timestamp '2018-05-02 15:30:30', 'PST') -
> from_utc_timestamp(timestamp '1970-01-30 16:00:00', 'PST') AS STRING);
> throws below Exception
> {code:java}
> Error: Error while compiling statement: FAILED: SemanticException Line 0:-1
> Wrong arguments ''PST'': No matching method for class
> org.apache.hadoop.hive.ql.udf.UDFToString with (interval_day_time). Possible
> choices: _FUNC_(bigint) _FUNC_(binary) _FUNC_(boolean) _FUNC_(date)
> _FUNC_(decimal(38,18)) _FUNC_(double) _FUNC_(float) _FUNC_(int)
> _FUNC_(smallint) _FUNC_(string) _FUNC_(timestamp) _FUNC_(tinyint)
> _FUNC_(void) (state=42000,code=4){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20599) CAST(INTERVAL_DAY_TIME AS STRING) is throwing SemanticException
[
https://issues.apache.org/jira/browse/HIVE-20599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran updated HIVE-20599:
-
Attachment: HIVE-20599.2.patch
> CAST(INTERVAL_DAY_TIME AS STRING) is throwing SemanticException
> ---
>
> Key: HIVE-20599
> URL: https://issues.apache.org/jira/browse/HIVE-20599
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 3.1.0
>Reporter: Naresh P R
>Assignee: Prasanth Jayachandran
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-20599-branch-3.patch,
> HIVE-20599.1-branch-3.1.patch, HIVE-20599.1-branch-3.patch,
> HIVE-20599.1.patch, HIVE-20599.2.patch
>
>
> SELECT CAST(from_utc_timestamp(timestamp '2018-05-02 15:30:30', 'PST') -
> from_utc_timestamp(timestamp '1970-01-30 16:00:00', 'PST') AS STRING);
> throws below Exception
> {code:java}
> Error: Error while compiling statement: FAILED: SemanticException Line 0:-1
> Wrong arguments ''PST'': No matching method for class
> org.apache.hadoop.hive.ql.udf.UDFToString with (interval_day_time). Possible
> choices: _FUNC_(bigint) _FUNC_(binary) _FUNC_(boolean) _FUNC_(date)
> _FUNC_(decimal(38,18)) _FUNC_(double) _FUNC_(float) _FUNC_(int)
> _FUNC_(smallint) _FUNC_(string) _FUNC_(timestamp) _FUNC_(tinyint)
> _FUNC_(void) (state=42000,code=4){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20621) GetOperationStatus called in resultset.next causing incremental slowness
[ https://issues.apache.org/jira/browse/HIVE-20621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625024#comment-16625024 ] Prasanth Jayachandran commented on HIVE-20621: -- Committed to master and branch-3. Thanks for the review! > GetOperationStatus called in resultset.next causing incremental slowness > > > Key: HIVE-20621 > URL: https://issues.apache.org/jira/browse/HIVE-20621 > Project: Hive > Issue Type: Bug > Components: JDBC >Affects Versions: 4.0.0, 3.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-20621.1.patch, HIVE-20621.2.patch > > > Fetching result set for a result cache hit query gets slower as more rows are > fetched. For fetching 10 row result set it took about 900ms but fetching 200 > row result set took 8 seconds. > Reason for this slowness is GetOperationStatus is invoked inside > resultset.next() and it happens for every row even after operation has > completed. This is one RPC call per row fetched (there is also connection > overhead without keepalive). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20621) GetOperationStatus called in resultset.next causing incremental slowness
[ https://issues.apache.org/jira/browse/HIVE-20621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-20621: - Resolution: Fixed Fix Version/s: 3.2.0 4.0.0 Status: Resolved (was: Patch Available) > GetOperationStatus called in resultset.next causing incremental slowness > > > Key: HIVE-20621 > URL: https://issues.apache.org/jira/browse/HIVE-20621 > Project: Hive > Issue Type: Bug > Components: JDBC >Affects Versions: 4.0.0, 3.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-20621.1.patch, HIVE-20621.2.patch > > > Fetching result set for a result cache hit query gets slower as more rows are > fetched. For fetching 10 row result set it took about 900ms but fetching 200 > row result set took 8 seconds. > Reason for this slowness is GetOperationStatus is invoked inside > resultset.next() and it happens for every row even after operation has > completed. This is one RPC call per row fetched (there is also connection > overhead without keepalive). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20621) GetOperationStatus called in resultset.next causing incremental slowness
[
https://issues.apache.org/jira/browse/HIVE-20621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625017#comment-16625017
]
Hive QA commented on HIVE-20621:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12940874/HIVE-20621.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14993 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/13994/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13994/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13994/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12940874 - PreCommit-HIVE-Build
> GetOperationStatus called in resultset.next causing incremental slowness
>
>
> Key: HIVE-20621
> URL: https://issues.apache.org/jira/browse/HIVE-20621
> Project: Hive
> Issue Type: Bug
> Components: JDBC
>Affects Versions: 4.0.0, 3.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-20621.1.patch, HIVE-20621.2.patch
>
>
> Fetching result set for a result cache hit query gets slower as more rows are
> fetched. For fetching 10 row result set it took about 900ms but fetching 200
> row result set took 8 seconds.
> Reason for this slowness is GetOperationStatus is invoked inside
> resultset.next() and it happens for every row even after operation has
> completed. This is one RPC call per row fetched (there is also connection
> overhead without keepalive).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20621) GetOperationStatus called in resultset.next causing incremental slowness
[
https://issues.apache.org/jira/browse/HIVE-20621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625018#comment-16625018
]
Hive QA commented on HIVE-20621:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12940874/HIVE-20621.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/13995/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13995/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13995/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12940874/HIVE-20621.2.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12940874 - PreCommit-HIVE-Build
> GetOperationStatus called in resultset.next causing incremental slowness
>
>
> Key: HIVE-20621
> URL: https://issues.apache.org/jira/browse/HIVE-20621
> Project: Hive
> Issue Type: Bug
> Components: JDBC
>Affects Versions: 4.0.0, 3.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-20621.1.patch, HIVE-20621.2.patch
>
>
> Fetching result set for a result cache hit query gets slower as more rows are
> fetched. For fetching 10 row result set it took about 900ms but fetching 200
> row result set took 8 seconds.
> Reason for this slowness is GetOperationStatus is invoked inside
> resultset.next() and it happens for every row even after operation has
> completed. This is one RPC call per row fetched (there is also connection
> overhead without keepalive).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18628) Make tez dag status check interval configurable
[ https://issues.apache.org/jira/browse/HIVE-18628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625015#comment-16625015 ] Prasanth Jayachandran commented on HIVE-18628: -- 2 polling interval (200ms and 1000ms) is changed to 1 configurable fixed interval (500ms). I think with HIVE-16635 the polling interval is 200ms until dag start. When dag goes to running state, the polling interval is changed to 1 second. Also when there are no WM triggers, there should be no counter aggregation in AM. [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/monitoring/TezJobMonitor.java#L174] The making it configurable is intentional change, but making it slow isn't :) We can certainly change the default to 1000ms to get back the old behaviour (also not sure if the initial 200ms polling will be useful) > Make tez dag status check interval configurable > --- > > Key: HIVE-18628 > URL: https://issues.apache.org/jira/browse/HIVE-18628 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18628.1.patch > > > DAG status check interval in TezJobMonitor is not configurable. This can > cause cached data to be returned and fail some trigger validation tests. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18778) Needs to capture input/output entities in explain
[
https://issues.apache.org/jira/browse/HIVE-18778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated HIVE-18778:
--
Attachment: HIVE-18778.7.patch
> Needs to capture input/output entities in explain
> -
>
> Key: HIVE-18778
> URL: https://issues.apache.org/jira/browse/HIVE-18778
> Project: Hive
> Issue Type: Bug
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-18778-SparkPositive.patch, HIVE-18778.1.patch,
> HIVE-18778.2.patch, HIVE-18778.3.patch, HIVE-18778.4.patch,
> HIVE-18778.5.patch, HIVE-18778.6.patch, HIVE-18778.7.patch,
> HIVE-18778_TestCliDriver.patch, HIVE-18788_SparkNegative.patch,
> HIVE-18788_SparkPerf.patch
>
>
> With Sentry enabled, commands like explain drop table foo fail with {{explain
> drop table foo;}}
> {code}
> Error: Error while compiling statement: FAILED: SemanticException No valid
> privileges
> Required privilege( Table) not available in input privileges
> The required privileges: (state=42000,code=4)
> {code}
> Sentry fails to authorize because the ExplainSemanticAnalyzer uses an
> instance of DDLSemanticAnalyzer to analyze the explain query.
> {code}
> BaseSemanticAnalyzer sem = SemanticAnalyzerFactory.get(conf, input);
> sem.analyze(input, ctx);
> sem.validate()
> {code}
> The inputs/outputs entities for this query are set in the above code.
> However, these are never set on the instance of ExplainSemanticAnalyzer
> itself and thus is not propagated into the HookContext in the calling Driver
> code.
> {code}
> sem.analyze(tree, ctx); --> this results in calling the above code that uses
> DDLSA
> hookCtx.update(sem); --> sem is an instance of ExplainSemanticAnalyzer, this
> code attempts to update the HookContext with the input/output info from ESA
> which is never set.
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20621) GetOperationStatus called in resultset.next causing incremental slowness
[
https://issues.apache.org/jira/browse/HIVE-20621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625003#comment-16625003
]
Hive QA commented on HIVE-20621:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
27s{color} | {color:blue} jdbc in master has 17 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 11m 36s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-13994/dev-support/hive-personality.sh
|
| git revision | master / cdba00c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: jdbc U: jdbc |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13994/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> GetOperationStatus called in resultset.next causing incremental slowness
>
>
> Key: HIVE-20621
> URL: https://issues.apache.org/jira/browse/HIVE-20621
> Project: Hive
> Issue Type: Bug
> Components: JDBC
>Affects Versions: 4.0.0, 3.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-20621.1.patch, HIVE-20621.2.patch
>
>
> Fetching result set for a result cache hit query gets slower as more rows are
> fetched. For fetching 10 row result set it took about 900ms but fetching 200
> row result set took 8 seconds.
> Reason for this slowness is GetOperationStatus is invoked inside
> resultset.next() and it happens for every row even after operation has
> completed. This is one RPC call per row fetched (there is also connection
> overhead without keepalive).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20385) Date: date + int fails to add days
[
https://issues.apache.org/jira/browse/HIVE-20385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624993#comment-16624993
]
Ashutosh Chauhan commented on HIVE-20385:
-
+1
> Date: date + int fails to add days
> --
>
> Key: HIVE-20385
> URL: https://issues.apache.org/jira/browse/HIVE-20385
> Project: Hive
> Issue Type: Bug
> Components: Types
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Bruno Pusztahazi
>Priority: Minor
> Attachments: HIVE-20385.1.patch, HIVE-20385.2.patch,
> HIVE-20385.3.patch
>
>
> {code}
> select current_date + 5;
> FAILED: SemanticException [Error 10014]: Line 1:7 Wrong arguments '5': No
> matching method for class
> org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPDTIPlus with (date, int)
> {code}
> This works in Postgres 9.6 - http://sqlfiddle.com/#!17/9eecb/19253/0
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20556) Expose an API to retrieve the TBL_ID from TBLS in the metastore tables
[
https://issues.apache.org/jira/browse/HIVE-20556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624988#comment-16624988
]
Hive QA commented on HIVE-20556:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12940864/HIVE-20556.13.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 15003 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[import_exported_table]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[import_exported_table]
(batchId=189)
org.apache.hadoop.hive.metastore.TestEmbeddedHiveMetaStore.testPartition
(batchId=222)
org.apache.hadoop.hive.metastore.TestHiveMetaStorePartitionSpecs.testAddPartitions
(batchId=221)
org.apache.hadoop.hive.metastore.client.TestTablesCreateDropAlterTruncate.testCreateGetDeleteTable[Embedded]
(batchId=219)
org.apache.hadoop.hive.ql.metadata.TestHive.testTable (batchId=295)
org.apache.hadoop.hive.ql.metadata.TestHive.testThriftTable (batchId=295)
org.apache.hadoop.hive.ql.metadata.TestHiveMetaStoreChecker.testPartitionsCheck
(batchId=297)
org.apache.hadoop.hive.ql.metadata.TestHiveMetaStoreChecker.testTableCheck
(batchId=297)
org.apache.hadoop.hive.ql.security.TestAuthorizationPreEventListener.testListener
(batchId=244)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/13993/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13993/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13993/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12940864 - PreCommit-HIVE-Build
> Expose an API to retrieve the TBL_ID from TBLS in the metastore tables
> --
>
> Key: HIVE-20556
> URL: https://issues.apache.org/jira/browse/HIVE-20556
> Project: Hive
> Issue Type: New Feature
> Components: Metastore, Standalone Metastore
>Reporter: Jaume M
>Assignee: Jaume M
>Priority: Major
> Attachments: HIVE-20556.1.patch, HIVE-20556.10.patch,
> HIVE-20556.11.patch, HIVE-20556.12.patch, HIVE-20556.13.patch,
> HIVE-20556.2.patch, HIVE-20556.3.patch, HIVE-20556.4.patch,
> HIVE-20556.5.patch, HIVE-20556.6.patch, HIVE-20556.7.patch,
> HIVE-20556.8.patch, HIVE-20556.9.patch
>
>
> We have two options to do this
> 1) Use the current MTable and add a field for this value
> 2) Add an independent API call to the metastore that would return the TBL_ID.
> Option 1 is preferable.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20556) Expose an API to retrieve the TBL_ID from TBLS in the metastore tables
[
https://issues.apache.org/jira/browse/HIVE-20556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624983#comment-16624983
]
Hive QA commented on HIVE-20556:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
37s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
24s{color} | {color:blue} standalone-metastore/metastore-common in master has
28 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
47s{color} | {color:blue} ql in master has 2326 extant Findbugs warnings.
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
14s{color} | {color:red} metastore-server in master failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
32s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
37s{color} | {color:red} hive-unit in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
13s{color} | {color:red} metastore-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
32s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 39m 25s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-13993/dev-support/hive-personality.sh
|
| git revision | master / cdba00c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13993/yetus/branch-findbugs-standalone-metastore_metastore-server.txt
|
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13993/yetus/patch-mvninstall-itests_hive-unit.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13993/yetus/whitespace-eol.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13993/yetus/patch-findbugs-standalone-metastore_metastore-server.txt
|
| modules | C: standalone-metastore/metastore-common itests/hive-unit ql
standalone-metastore/metastore-server U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13993/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Expose an API to retrieve the TBL_ID from TBLS in the metastore tables
> --
>
> Key: HIVE-20556
> URL:
