[jira] [Commented] (HIVE-20126) OrcInputFormat does not pass conf to orc reader options
[
https://issues.apache.org/jira/browse/HIVE-20126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538179#comment-16538179
]
Hive QA commented on HIVE-20126:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
48s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
55s{color} | {color:blue} ql in master has 2287 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
34s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 1 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m
51s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 59m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12503/dev-support/hive-personality.sh
|
| git revision | master / ddf7e25 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12503/yetus/whitespace-tabs.txt
|
| modules | C: ql . itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12503/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> OrcInputFormat does not pass conf to orc reader options
> ---
>
> Key: HIVE-20126
> URL: https://issues.apache.org/jira/browse/HIVE-20126
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0, 3.2.0
>Reporter: Aswathy Chellammal Sreekumar
>Assignee: Prasanth Jayachandran
>Priority: Critical
> Attachments: HIVE-20126.1.patch
>
>
> VectorizedOrcInputFormat creates Orc reader options without passing in the
> configuration object. Without it setting orc configurations will not have any
> impact.
> Example:
> set orc.force.positional.evolution=true;
> does not work for positional schema evolution (will attach test case).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17751) Separate HMS Client and HMS server into separate sub-modules
[ https://issues.apache.org/jira/browse/HIVE-17751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538180#comment-16538180 ] Peter Vary commented on HIVE-17751: --- +1 for the separate client module > Separate HMS Client and HMS server into separate sub-modules > > > Key: HIVE-17751 > URL: https://issues.apache.org/jira/browse/HIVE-17751 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Vihang Karajgaonkar >Assignee: Alexander Kolbasov >Priority: Major > Attachments: HIVE-17751.01.patch, HIVE-17751.02.patch, > HIVE-17751.03.patch, HIVE-17751.04.patch, HIVE-17751.07.patch, > HIVE-17751.08.patch, HIVE-17751.09.patch, HIVE-17751.10.patch, > HIVE-17751.11.patch, HIVE-17751.13.patch, HIVE-17751.14.patch, > HIVE-17751.15.patch > > > external applications which are interfacing with HMS should ideally only > include HMSClient library instead of one big library containing server as > well. We should ideally have a thin client library so that cross version > support for external applications is easier. We should sub-divide the > standalone module into possibly 3 modules (one for common classes, one for > client classes and one for server) or 2 sub-modules (one for client and one > for server) so that we can generate separate jars for HMS client and server. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20126) OrcInputFormat does not pass conf to orc reader options
[
https://issues.apache.org/jira/browse/HIVE-20126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538192#comment-16538192
]
Hive QA commented on HIVE-20126:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12930910/HIVE-20126.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14622 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12503/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12503/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12503/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12930910 - PreCommit-HIVE-Build
> OrcInputFormat does not pass conf to orc reader options
> ---
>
> Key: HIVE-20126
> URL: https://issues.apache.org/jira/browse/HIVE-20126
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0, 3.2.0
>Reporter: Aswathy Chellammal Sreekumar
>Assignee: Prasanth Jayachandran
>Priority: Critical
> Attachments: HIVE-20126.1.patch
>
>
> VectorizedOrcInputFormat creates Orc reader options without passing in the
> configuration object. Without it setting orc configurations will not have any
> impact.
> Example:
> set orc.force.positional.evolution=true;
> does not work for positional schema evolution (will attach test case).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-19166) TestMiniLlapLocalCliDriver sysdb failure
[ https://issues.apache.org/jira/browse/HIVE-19166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai reassigned HIVE-19166: - Assignee: Daniel Dai (was: Zoltan Haindrich) > TestMiniLlapLocalCliDriver sysdb failure > > > Key: HIVE-19166 > URL: https://issues.apache.org/jira/browse/HIVE-19166 > Project: Hive > Issue Type: Sub-task > Components: Test >Reporter: Vineet Garg >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-19166.04.patch, HIVE-19166.05.patch, > HIVE-19166.06.patch, HIVE-19166.09.patch, HIVE-19166.1.patch, > HIVE-19166.10.patch, HIVE-19166.11.patch, HIVE-19166.2.patch, > HIVE-19166.3.patch > > > Broken by HIVE-18715 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20126) OrcInputFormat does not pass conf to orc reader options
[ https://issues.apache.org/jira/browse/HIVE-20126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-20126: - Resolution: Fixed Fix Version/s: 3.2.0 4.0.0 Status: Resolved (was: Patch Available) Committed to master and branch-3. > OrcInputFormat does not pass conf to orc reader options > --- > > Key: HIVE-20126 > URL: https://issues.apache.org/jira/browse/HIVE-20126 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0, 3.2.0 >Reporter: Aswathy Chellammal Sreekumar >Assignee: Prasanth Jayachandran >Priority: Critical > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-20126.1.patch > > > VectorizedOrcInputFormat creates Orc reader options without passing in the > configuration object. Without it setting orc configurations will not have any > impact. > Example: > set orc.force.positional.evolution=true; > does not work for positional schema evolution (will attach test case). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-20132) External Table: Alter Table Change column is not supported.
[
https://issues.apache.org/jira/browse/HIVE-20132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dileep Kumar Chiguruvada reassigned HIVE-20132:
---
Assignee: Ashutosh Chauhan
> External Table: Alter Table Change column is not supported.
> ---
>
> Key: HIVE-20132
> URL: https://issues.apache.org/jira/browse/HIVE-20132
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Dileep Kumar Chiguruvada
>Assignee: Ashutosh Chauhan
>Priority: Major
> Fix For: 3.0.0
>
>
> External Table: Alter Table Change column is not supported..
> It fails with "ALTER TABLE can only be used for [ADDPROPS, DROPPROPS,
> ADDCOLS] to a non-native table"
> {code}
> 0: jdbc:hive2://ctr-e138-1518143905142-404953> alter table calcs change
> column key string int;
> Error: Error while compiling statement: FAILED: SemanticException [Error
> 10134]: ALTER TABLE can only be used for [ADDPROPS, DROPPROPS, ADDCOLS] to a
> non-native table calcs (state=42000,code=10134)
> {code}
> This is very much required for upgraded clusters where managed tables(in 2.6
> clusters) automatically converted to external tables(in 3.0.0).
> One such use case is Storagehandler Tables, where we might need to alter
> columns.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20070) ptest optimization - Replicate ACID/MM tables write operations.
[ https://issues.apache.org/jira/browse/HIVE-20070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-20070: --- Status: Patch Available (was: Open) > ptest optimization - Replicate ACID/MM tables write operations. > > > Key: HIVE-20070 > URL: https://issues.apache.org/jira/browse/HIVE-20070 > Project: Hive > Issue Type: Sub-task > Components: repl, Transactions >Affects Versions: 3.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: ACID, DR, pull-request-available, replication > Fix For: 4.0.0 > > Attachments: HIVE-20070.01.patch > > > change the test to do incremental replication for each operation , instead of > only one incremental replication at the end -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20070) ptest optimization - Replicate ACID/MM tables write operations.
[ https://issues.apache.org/jira/browse/HIVE-20070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-20070: --- Attachment: HIVE-20070.01.patch > ptest optimization - Replicate ACID/MM tables write operations. > > > Key: HIVE-20070 > URL: https://issues.apache.org/jira/browse/HIVE-20070 > Project: Hive > Issue Type: Sub-task > Components: repl, Transactions >Affects Versions: 3.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: ACID, DR, pull-request-available, replication > Fix For: 4.0.0 > > Attachments: HIVE-20070.01.patch > > > change the test to do incremental replication for each operation , instead of > only one incremental replication at the end -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20070) ptest optimization - Replicate ACID/MM tables write operations.
[
https://issues.apache.org/jira/browse/HIVE-20070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538343#comment-16538343
]
Hive QA commented on HIVE-20070:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
16s{color} | {color:red} itests/hive-unit: The patch generated 31 new + 64
unchanged - 0 fixed = 95 total (was 64) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 47s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12510/dev-support/hive-personality.sh
|
| git revision | master / 18fb1b3 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12510/yetus/diff-checkstyle-itests_hive-unit.txt
|
| modules | C: itests/hive-unit U: itests/hive-unit |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12510/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ptest optimization - Replicate ACID/MM tables write operations.
>
>
> Key: HIVE-20070
> URL: https://issues.apache.org/jira/browse/HIVE-20070
> Project: Hive
> Issue Type: Sub-task
> Components: repl, Transactions
>Affects Versions: 3.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: ACID, DR, pull-request-available, replication
> Fix For: 4.0.0
>
> Attachments: HIVE-20070.01.patch
>
>
> change the test to do incremental replication for each operation , instead of
> only one incremental replication at the end
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20070) ptest optimization - Replicate ACID/MM tables write operations.
[ https://issues.apache.org/jira/browse/HIVE-20070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538348#comment-16538348 ] ASF GitHub Bot commented on HIVE-20070: --- GitHub user maheshk114 opened a pull request: https://github.com/apache/hive/pull/395 HIVE-20070 : ptest optimization - Replicate ACID/MM tables write operations … You can merge this pull request into a Git repository by running: $ git pull https://github.com/maheshk114/hive HIVE-20121 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hive/pull/395.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #395 commit 916ba47cb33c71cfaba55b1e96335e3deadbc424 Author: Mahesh Kumar Behera Date: 2018-07-10T09:25:04Z HIVE-20070 : ptest optimization - Replicate ACID/MM tables write operations > ptest optimization - Replicate ACID/MM tables write operations. > > > Key: HIVE-20070 > URL: https://issues.apache.org/jira/browse/HIVE-20070 > Project: Hive > Issue Type: Sub-task > Components: repl, Transactions >Affects Versions: 3.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: ACID, DR, pull-request-available, replication > Fix For: 4.0.0 > > Attachments: HIVE-20070.01.patch > > > change the test to do incremental replication for each operation , instead of > only one incremental replication at the end -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20094) Update Druid to 0.12.1 version
[ https://issues.apache.org/jira/browse/HIVE-20094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538358#comment-16538358 ] slim bouguerra commented on HIVE-20094: --- [~ashutoshc] can this go in or need to re upload the patch? > Update Druid to 0.12.1 version > -- > > Key: HIVE-20094 > URL: https://issues.apache.org/jira/browse/HIVE-20094 > Project: Hive > Issue Type: Bug >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Minor > Attachments: HIVE-20094.patch > > > As per Jira title. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20070) ptest optimization - Replicate ACID/MM tables write operations.
[
https://issues.apache.org/jira/browse/HIVE-20070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538391#comment-16538391
]
Hive QA commented on HIVE-20070:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12930972/HIVE-20070.01.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14628 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=190)
[druidmini_dynamic_partition.q,druidmini_expressions.q,druidmini_test_alter.q,druidmini_test1.q,druidmini_test_insert.q]
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12510/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12510/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12510/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12930972 - PreCommit-HIVE-Build
> ptest optimization - Replicate ACID/MM tables write operations.
>
>
> Key: HIVE-20070
> URL: https://issues.apache.org/jira/browse/HIVE-20070
> Project: Hive
> Issue Type: Sub-task
> Components: repl, Transactions
>Affects Versions: 3.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: ACID, DR, pull-request-available, replication
> Fix For: 4.0.0
>
> Attachments: HIVE-20070.01.patch
>
>
> change the test to do incremental replication for each operation , instead of
> only one incremental replication at the end
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20095) Fix jdbc external table feature
[ https://issues.apache.org/jira/browse/HIVE-20095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538440#comment-16538440 ] Jonathan Doron commented on HIVE-20095: --- [~daijy] Can you also please explain your change in JdbcRecordIterator.java? I think that this change can cause many troubles when the columnTypes member is set. what was missing it the old code? [~jcamachorodriguez] Please take a look. > Fix jdbc external table feature > --- > > Key: HIVE-20095 > URL: https://issues.apache.org/jira/browse/HIVE-20095 > Project: Hive > Issue Type: Bug >Reporter: Jonathan Doron >Assignee: Jonathan Doron >Priority: Major > Attachments: HIVE-20095.1.patch > > > It seems like the committed code for HIVE-19161 > (7584b3276bebf64aa006eaa162c0a6264d8fcb56) reverted some of HIVE-18423 > updates, and therefore some of the external table queries are not working > correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20133) Cannot set fs.s3a.access.key and fs.s3a.secret.key from HiveMetastoreClient
[ https://issues.apache.org/jira/browse/HIVE-20133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandhya Agarwal updated HIVE-20133: --- Summary: Cannot set fs.s3a.access.key and fs.s3a.secret.key from HiveMetastoreClient (was: Cannot set fs.s3a.access.key and fs.s3a.secret.key for HiveMetastoreClient) > Cannot set fs.s3a.access.key and fs.s3a.secret.key from HiveMetastoreClient > --- > > Key: HIVE-20133 > URL: https://issues.apache.org/jira/browse/HIVE-20133 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Affects Versions: 3.0.0 >Reporter: Sandhya Agarwal >Priority: Major > > Hello, > We want to leverage standalone metastore for our project, for which I want to > enable access to multiple S3 buckets, each with its own access key and secret > key. I am trying to access the metastore operations from a Java client using > HiveMetastoreClient and using the thrift metastore URI to connect to the > metastore service. For the database location and table location, I am using > the S3A file system scheme. I am trying to set the access key and secret key > for the S3 bucket location in the HiveMetastoreClient, but the client side > settings are never honoured and I cannot figure out a way to do this. One way > is to provide the per-bucket access key and secret key on the server side in > metastore-site.xml. However, I want this to be dynamic as I want the > metastore to be multi tenant enabled and based on the current tenant user, I > wish to provide these keys through my client. I tried looking through the > metastore source code, but did not find a way to do this. Can this be done ? > Thank you, > Sandhya > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20134) Improve logging when HoS Driver is killed due to exceeding memory limits
[
https://issues.apache.org/jira/browse/HIVE-20134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538504#comment-16538504
]
Sahil Takiar commented on HIVE-20134:
-
This probably requires fixing HIVE-18921 too.
> Improve logging when HoS Driver is killed due to exceeding memory limits
>
>
> Key: HIVE-20134
> URL: https://issues.apache.org/jira/browse/HIVE-20134
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Priority: Major
>
> This was improved in HIVE-18093, but more can be done. If a HoS Driver gets
> killed because it exceeds its memory limits, YARN will issue a SIGTERM on the
> process. The SIGTERM will cause the shutdown hook in the HoS Driver to be
> triggered. This causes the Driver to kill all running jobs, even if they are
> running. The user ends up seeing an error like the one below. Which isn't
> very informative. We should propagate the error from the Driver shutdown hook
> to the user.
> {code:java}
> INFO : 2018-07-09 17:48:42,580 Stage-64_0: 526/526 Finished Stage-65_0:
> 1405/1405 Finished Stage-66_0: 0(+759)/1102 Stage-67_0: 0/1099 Stage-68_0:
> 0/1099 Stage-69_0: 0/1
> INFO : 2018-07-09 17:48:44,589 Stage-64_0: 526/526 Finished Stage-65_0:
> 1405/1405 Finished Stage-66_0: 1(+759)/1102 Stage-67_0: 0/1099 Stage-68_0:
> 0/1099 Stage-69_0: 0/1
> INFO : 2018-07-09 17:48:45,591 Stage-64_0: 526/526 Finished Stage-65_0:
> 1405/1405 Finished Stage-66_0: 2(+759)/1102 Stage-67_0: 0/1099 Stage-68_0:
> 0/1099 Stage-69_0: 0/1
> INFO : 2018-07-09 17:48:48,596 Stage-64_0: 526/526 Finished Stage-65_0:
> 1405/1405 Finished Stage-66_0: 2(+759)/1102 Stage-67_0: 0/1099 Stage-68_0:
> 0/1099 Stage-69_0: 0/1
> ERROR : Spark job[23] failed
> java.lang.InterruptedException: null
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:998)
> ~[?:1.8.0_141]
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
> ~[?:1.8.0_141]
> at scala.concurrent.impl.Promise$DefaultPromise.tryAwait(Promise.scala:202)
> ~[scala-library-2.11.8.jar:?]
> at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:218)
> ~[scala-library-2.11.8.jar:?]
> at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:153)
> ~[scala-library-2.11.8.jar:?]
> at org.apache.spark.SimpleFutureAction.ready(FutureAction.scala:125)
> ~[spark-core_2.11-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> at org.apache.spark.SimpleFutureAction.ready(FutureAction.scala:114)
> ~[spark-core_2.11-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> at org.apache.spark.util.ThreadUtils$.awaitReady(ThreadUtils.scala:222)
> ~[spark-core_2.11-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> at org.apache.spark.JavaFutureActionWrapper.getImpl(FutureAction.scala:264)
> ~[spark-core_2.11-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> at org.apache.spark.JavaFutureActionWrapper.get(FutureAction.scala:277)
> ~[spark-core_2.11-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> at
> org.apache.hive.spark.client.RemoteDriver$JobWrapper.call(RemoteDriver.java:391)
> ~[hive-exec-2.1.1-SNAPSHOT.jar:2.1.1-SNAPSHOT]
> at
> org.apache.hive.spark.client.RemoteDriver$JobWrapper.call(RemoteDriver.java:352)
> ~[hive-exec-2.1.1-SNAPSHOT.jar:2.1.1-SNAPSHOT]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_141]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [?:1.8.0_141]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [?:1.8.0_141]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_141]
> ERROR : FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.spark.SparkTask. null
> INFO : Completed executing
> command(queryId=hive_20180709174140_0f64ee17-f793-441a-9a77-3ee0cd0a9c32);
> Time taken: 249.727 seconds
> Error: Error while processing statement: FAILED: Execution Error, return code
> 1 from org.apache.hadoop.hive.ql.exec.spark.SparkTask. null
> (state=08S01,code=1){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20095) Fix jdbc external table feature
[ https://issues.apache.org/jira/browse/HIVE-20095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Doron updated HIVE-20095: -- Status: Patch Available (was: Open) > Fix jdbc external table feature > --- > > Key: HIVE-20095 > URL: https://issues.apache.org/jira/browse/HIVE-20095 > Project: Hive > Issue Type: Bug >Reporter: Jonathan Doron >Assignee: Jonathan Doron >Priority: Major > Attachments: HIVE-20095.1.patch, HIVE-20095.2.patch > > > It seems like the committed code for HIVE-19161 > (7584b3276bebf64aa006eaa162c0a6264d8fcb56) reverted some of HIVE-18423 > updates, and therefore some of the external table queries are not working > correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20095) Fix jdbc external table feature
[ https://issues.apache.org/jira/browse/HIVE-20095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Doron updated HIVE-20095: -- Status: Open (was: Patch Available) > Fix jdbc external table feature > --- > > Key: HIVE-20095 > URL: https://issues.apache.org/jira/browse/HIVE-20095 > Project: Hive > Issue Type: Bug >Reporter: Jonathan Doron >Assignee: Jonathan Doron >Priority: Major > Attachments: HIVE-20095.1.patch, HIVE-20095.2.patch > > > It seems like the committed code for HIVE-19161 > (7584b3276bebf64aa006eaa162c0a6264d8fcb56) reverted some of HIVE-18423 > updates, and therefore some of the external table queries are not working > correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20095) Fix jdbc external table feature
[ https://issues.apache.org/jira/browse/HIVE-20095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Doron updated HIVE-20095: -- Attachment: HIVE-20095.2.patch > Fix jdbc external table feature > --- > > Key: HIVE-20095 > URL: https://issues.apache.org/jira/browse/HIVE-20095 > Project: Hive > Issue Type: Bug >Reporter: Jonathan Doron >Assignee: Jonathan Doron >Priority: Major > Attachments: HIVE-20095.1.patch, HIVE-20095.2.patch > > > It seems like the committed code for HIVE-19161 > (7584b3276bebf64aa006eaa162c0a6264d8fcb56) reverted some of HIVE-18423 > updates, and therefore some of the external table queries are not working > correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20044) Arrow Serde should pad char values and handle empty strings correctly
[ https://issues.apache.org/jira/browse/HIVE-20044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538521#comment-16538521 ] ASF GitHub Bot commented on HIVE-20044: --- GitHub user pudidic opened a pull request: https://github.com/apache/hive/pull/396 HIVE-20044: Arrow Serde should pad char values and handle empty strin… …gs correctly (Teddy Choi) Change-Id: Iaa4417056a6e2a88335ed0cd5436142e128009c6 You can merge this pull request into a Git repository by running: $ git pull https://github.com/pudidic/hive HIVE-20044-branch-3 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hive/pull/396.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #396 commit edc90d42a4260272ae5abfeaef8f7966357518a1 Author: Teddy Choi Date: 2018-07-10T12:50:49Z HIVE-20044: Arrow Serde should pad char values and handle empty strings correctly (Teddy Choi) Change-Id: Iaa4417056a6e2a88335ed0cd5436142e128009c6 > Arrow Serde should pad char values and handle empty strings correctly > - > > Key: HIVE-20044 > URL: https://issues.apache.org/jira/browse/HIVE-20044 > Project: Hive > Issue Type: Bug > Components: Serializers/Deserializers >Reporter: Teddy Choi >Assignee: Teddy Choi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20044.patch > > > When Arrow Serde serializes char values, it loses padding. Also when it > counts empty strings, sometimes it makes a smaller number. It should pad char > values and handle empty strings correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20044) Arrow Serde should pad char values and handle empty strings correctly
[ https://issues.apache.org/jira/browse/HIVE-20044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-20044: -- Labels: pull-request-available (was: ) > Arrow Serde should pad char values and handle empty strings correctly > - > > Key: HIVE-20044 > URL: https://issues.apache.org/jira/browse/HIVE-20044 > Project: Hive > Issue Type: Bug > Components: Serializers/Deserializers >Reporter: Teddy Choi >Assignee: Teddy Choi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20044.patch > > > When Arrow Serde serializes char values, it loses padding. Also when it > counts empty strings, sometimes it makes a smaller number. It should pad char > values and handle empty strings correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20044) Arrow Serde should pad char values and handle empty strings correctly
[ https://issues.apache.org/jira/browse/HIVE-20044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538560#comment-16538560 ] ASF GitHub Bot commented on HIVE-20044: --- GitHub user pudidic opened a pull request: https://github.com/apache/hive/pull/397 HIVE-20044: Arrow Serde should pad char values and handle empty strin… …gs correctly (Teddy Choi) Change-Id: Ia87503aabf38c9599b887795d2b62566d8965f9b You can merge this pull request into a Git repository by running: $ git pull https://github.com/pudidic/hive HIVE-20044 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hive/pull/397.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #397 commit e7f463c1865d8518a5eb1756b9a548442fb776c1 Author: Teddy Choi Date: 2018-07-10T13:07:13Z HIVE-20044: Arrow Serde should pad char values and handle empty strings correctly (Teddy Choi) Change-Id: Ia87503aabf38c9599b887795d2b62566d8965f9b > Arrow Serde should pad char values and handle empty strings correctly > - > > Key: HIVE-20044 > URL: https://issues.apache.org/jira/browse/HIVE-20044 > Project: Hive > Issue Type: Bug > Components: Serializers/Deserializers >Reporter: Teddy Choi >Assignee: Teddy Choi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20044.patch > > > When Arrow Serde serializes char values, it loses padding. Also when it > counts empty strings, sometimes it makes a smaller number. It should pad char > values and handle empty strings correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19765) Add Parquet specific tests to BlobstoreCliDriver
[ https://issues.apache.org/jira/browse/HIVE-19765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538562#comment-16538562 ] Sahil Takiar commented on HIVE-19765: - I'm going to merge this if there are no objections. The tests passed and I don't see a point in waiting for another clean Hive QA run given that this patch only modifies q-files, and re-submitting this several times is just a waste of resources. > Add Parquet specific tests to BlobstoreCliDriver > > > Key: HIVE-19765 > URL: https://issues.apache.org/jira/browse/HIVE-19765 > Project: Hive > Issue Type: Sub-task >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-19765.1.patch, HIVE-19765.2.patch, > HIVE-19765.3.patch, HIVE-19765.4.patch, HIVE-19765.5.patch > > > Similar to what was done for RC and ORC files. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20044) Arrow Serde should pad char values and handle empty strings correctly
[ https://issues.apache.org/jira/browse/HIVE-20044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teddy Choi updated HIVE-20044: -- Attachment: HIVE-20044.1.patch HIVE-20044.1.branch-3.patch > Arrow Serde should pad char values and handle empty strings correctly > - > > Key: HIVE-20044 > URL: https://issues.apache.org/jira/browse/HIVE-20044 > Project: Hive > Issue Type: Bug > Components: Serializers/Deserializers >Reporter: Teddy Choi >Assignee: Teddy Choi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20044.1.branch-3.patch, HIVE-20044.1.patch, > HIVE-20044.patch > > > When Arrow Serde serializes char values, it loses padding. Also when it > counts empty strings, sometimes it makes a smaller number. It should pad char > values and handle empty strings correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20044) Arrow Serde should pad char values and handle empty strings correctly
[
https://issues.apache.org/jira/browse/HIVE-20044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538635#comment-16538635
]
Hive QA commented on HIVE-20044:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
58s{color} | {color:blue} ql in master has 2287 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
40s{color} | {color:red} ql: The patch generated 28 new + 361 unchanged - 1
fixed = 389 total (was 362) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 25s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12511/dev-support/hive-personality.sh
|
| git revision | master / 18fb1b3 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12511/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12511/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Arrow Serde should pad char values and handle empty strings correctly
> -
>
> Key: HIVE-20044
> URL: https://issues.apache.org/jira/browse/HIVE-20044
> Project: Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
>Reporter: Teddy Choi
>Assignee: Teddy Choi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20044.1.branch-3.patch, HIVE-20044.1.patch,
> HIVE-20044.patch
>
>
> When Arrow Serde serializes char values, it loses padding. Also when it
> counts empty strings, sometimes it makes a smaller number. It should pad char
> values and handle empty strings correctly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19846) Removed Deprecated Calls From FileUtils-getJarFilesByPath
[ https://issues.apache.org/jira/browse/HIVE-19846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HIVE-19846: --- Attachment: HIVE-19846.3.patch > Removed Deprecated Calls From FileUtils-getJarFilesByPath > - > > Key: HIVE-19846 > URL: https://issues.apache.org/jira/browse/HIVE-19846 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 3.0.0, 4.0.0 >Reporter: BELUGA BEHR >Assignee: BELUGA BEHR >Priority: Minor > Attachments: HIVE-19846.1.patch, HIVE-19846.2.patch, > HIVE-19846.3.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19846) Removed Deprecated Calls From FileUtils-getJarFilesByPath
[ https://issues.apache.org/jira/browse/HIVE-19846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HIVE-19846: --- Status: Open (was: Patch Available) > Removed Deprecated Calls From FileUtils-getJarFilesByPath > - > > Key: HIVE-19846 > URL: https://issues.apache.org/jira/browse/HIVE-19846 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 3.0.0, 4.0.0 >Reporter: BELUGA BEHR >Assignee: BELUGA BEHR >Priority: Minor > Attachments: HIVE-19846.1.patch, HIVE-19846.2.patch, > HIVE-19846.3.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19846) Removed Deprecated Calls From FileUtils-getJarFilesByPath
[ https://issues.apache.org/jira/browse/HIVE-19846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HIVE-19846: --- Status: Patch Available (was: Open) Submitted new patch to kick off tests again and to remove the variable name change > Removed Deprecated Calls From FileUtils-getJarFilesByPath > - > > Key: HIVE-19846 > URL: https://issues.apache.org/jira/browse/HIVE-19846 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 3.0.0, 4.0.0 >Reporter: BELUGA BEHR >Assignee: BELUGA BEHR >Priority: Minor > Attachments: HIVE-19846.1.patch, HIVE-19846.2.patch, > HIVE-19846.3.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19809) Remove Deprecated Code From Utilities Class
[
https://issues.apache.org/jira/browse/HIVE-19809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538677#comment-16538677
]
BELUGA BEHR commented on HIVE-19809:
[~aihuaxu] Please consider including this patch into the project.
> Remove Deprecated Code From Utilities Class
> ---
>
> Key: HIVE-19809
> URL: https://issues.apache.org/jira/browse/HIVE-19809
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-19809.1.patch
>
>
> {quote}
> This can go away once hive moves to support only JDK 7 and can use
> Files.createTempDirectory
> {quote}
> Remove the {{createTempDir}} method from the {{Utilities}} class.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19190) Improve Logging for SemanticException Handling
[
https://issues.apache.org/jira/browse/HIVE-19190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HIVE-19190:
---
Description:
Please improve the logging for queries that fail with SemanticException. For
example, when performing an action on a table that does not exist. The most
common reason why this happens is that someone fat fingers the table/column
name. It is not a system error pe se, but a user validation error. This is
not something the cluster administrator should have to worry about. Yet, Hive
performs some pretty extreme logging on the matter. I have attached to this
JIRA the logging produced by a single submission of the following query:
{code:sql}
select * from madeup;
{code}
For SemanticException exceptions, please print the Exception {{getMessage()}}
to the server INFO logging so that the query's life-cycle can be traced, but do
not blast ERRORs and stack traces to the log file.
was:
Please improve the logging for queries that fail with SemanticException. For
example, when performing an action on a table that does not exist. The most
common reason why this happens is that someone fat fingers the table/column
name. It is not a system error pe se, but a user validation error. This is
not something the cluster administrator should have to worry about. Yet, Hive
performs some pretty extreme logging on the matter. I have attached to this
JIRA the logging produced by a single submission of the following query:
{code:sql}
select * from madeup';
{code}
For SemanticException exceptions, please print the Exception {{getMessage()}}
to the server INFO logging so that the query's life-cycle can be traced, but do
not blast ERRORs and stack traces to the log file.
> Improve Logging for SemanticException Handling
> --
>
> Key: HIVE-19190
> URL: https://issues.apache.org/jira/browse/HIVE-19190
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 2.3.2
>Reporter: BELUGA BEHR
>Priority: Major
> Attachments: table_not_found_example.txt
>
>
> Please improve the logging for queries that fail with SemanticException. For
> example, when performing an action on a table that does not exist. The most
> common reason why this happens is that someone fat fingers the table/column
> name. It is not a system error pe se, but a user validation error. This is
> not something the cluster administrator should have to worry about. Yet,
> Hive performs some pretty extreme logging on the matter. I have attached to
> this JIRA the logging produced by a single submission of the following query:
> {code:sql}
> select * from madeup;
> {code}
>
> For SemanticException exceptions, please print the Exception {{getMessage()}}
> to the server INFO logging so that the query's life-cycle can be traced, but
> do not blast ERRORs and stack traces to the log file.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19190) Improve Logging for SemanticException Handling
[
https://issues.apache.org/jira/browse/HIVE-19190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538681#comment-16538681
]
BELUGA BEHR commented on HIVE-19190:
[~aihuaxu] [~afan]
> Improve Logging for SemanticException Handling
> --
>
> Key: HIVE-19190
> URL: https://issues.apache.org/jira/browse/HIVE-19190
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 2.3.2
>Reporter: BELUGA BEHR
>Priority: Major
> Attachments: table_not_found_example.txt
>
>
> Please improve the logging for queries that fail with SemanticException. For
> example, when performing an action on a table that does not exist. The most
> common reason why this happens is that someone fat fingers the table/column
> name. It is not a system error pe se, but a user validation error. This is
> not something the cluster administrator should have to worry about. Yet,
> Hive performs some pretty extreme logging on the matter. I have attached to
> this JIRA the logging produced by a single submission of the following query:
> {code:sql}
> select * from madeup;
> {code}
>
> For SemanticException exceptions, please print the Exception {{getMessage()}}
> to the server INFO logging so that the query's life-cycle can be traced, but
> do not blast ERRORs and stack traces to the log file.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19403) Demote 'Pattern' Logging
[
https://issues.apache.org/jira/browse/HIVE-19403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538684#comment-16538684
]
BELUGA BEHR commented on HIVE-19403:
[~aihuaxu] [~afan] Can you please push to master if acceptable?
> Demote 'Pattern' Logging
>
>
> Key: HIVE-19403
> URL: https://issues.apache.org/jira/browse/HIVE-19403
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 2.4.0
>Reporter: BELUGA BEHR
>Assignee: gonglinglei
>Priority: Trivial
> Labels: noob
> Attachments: HIVE-19403.1.patch
>
>
> In the {{DDLTask}} class, there is some logging that is not helpful to a
> cluster admin and should be demoted to _debug_ level logging. In fact, in
> one place in the code, it already is.
> {code}
> LOG.info("pattern: {}", showDatabasesDesc.getPattern());
> LOG.debug("pattern: {}", pattern);
> LOG.info("pattern: {}", showFuncs.getPattern());
> LOG.info("pattern: {}", showTblStatus.getPattern());
> {code}
> Here is an example... as an admin, I can already see what the pattern is, I
> do not need this extra logging. It provides no additional context.
> {code:java|title=Example}
> 2018-05-03 03:08:26,354 INFO org.apache.hadoop.hive.ql.Driver:
> [HiveServer2-Background-Pool: Thread-101980]: Executing
> command(queryId=hive_20180503030808_e53c26ef-2280-4eca-929b-668503105e2e):
> SHOW TABLE EXTENDED FROM my_db LIKE '*'
> 2018-05-03 03:08:26,355 INFO hive.ql.exec.DDLTask:
> [HiveServer2-Background-Pool: Thread-101980]: pattern: *
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20044) Arrow Serde should pad char values and handle empty strings correctly
[
https://issues.apache.org/jira/browse/HIVE-20044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538705#comment-16538705
]
Hive QA commented on HIVE-20044:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12931002/HIVE-20044.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14618 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=190)
[druidmini_dynamic_partition.q,druidmini_expressions.q,druidmini_test_alter.q,druidmini_test1.q,druidmini_test_insert.q]
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12511/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12511/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12511/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12931002 - PreCommit-HIVE-Build
> Arrow Serde should pad char values and handle empty strings correctly
> -
>
> Key: HIVE-20044
> URL: https://issues.apache.org/jira/browse/HIVE-20044
> Project: Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
>Reporter: Teddy Choi
>Assignee: Teddy Choi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20044.1.branch-3.patch, HIVE-20044.1.patch,
> HIVE-20044.patch
>
>
> When Arrow Serde serializes char values, it loses padding. Also when it
> counts empty strings, sometimes it makes a smaller number. It should pad char
> values and handle empty strings correctly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18038) org.apache.hadoop.hive.ql.session.OperationLog - Review
[ https://issues.apache.org/jira/browse/HIVE-18038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HIVE-18038: --- Status: Open (was: Patch Available) > org.apache.hadoop.hive.ql.session.OperationLog - Review > --- > > Key: HIVE-18038 > URL: https://issues.apache.org/jira/browse/HIVE-18038 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 3.0.0 >Reporter: BELUGA BEHR >Assignee: BELUGA BEHR >Priority: Trivial > Attachments: HIVE-18038.1.patch, HIVE-18038.2.patch, > HIVE-18038.3.patch, HIVE-18038.4.patch, HIVE-18038.5.patch > > > Simplifications, improve readability -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18038) org.apache.hadoop.hive.ql.session.OperationLog - Review
[ https://issues.apache.org/jira/browse/HIVE-18038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HIVE-18038: --- Attachment: HIVE-18038.5.patch > org.apache.hadoop.hive.ql.session.OperationLog - Review > --- > > Key: HIVE-18038 > URL: https://issues.apache.org/jira/browse/HIVE-18038 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 3.0.0 >Reporter: BELUGA BEHR >Assignee: BELUGA BEHR >Priority: Trivial > Attachments: HIVE-18038.1.patch, HIVE-18038.2.patch, > HIVE-18038.3.patch, HIVE-18038.4.patch, HIVE-18038.5.patch > > > Simplifications, improve readability -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18038) org.apache.hadoop.hive.ql.session.OperationLog - Review
[ https://issues.apache.org/jira/browse/HIVE-18038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HIVE-18038: --- Status: Patch Available (was: Open) Adding new patch to re-launch unit tests for inspection > org.apache.hadoop.hive.ql.session.OperationLog - Review > --- > > Key: HIVE-18038 > URL: https://issues.apache.org/jira/browse/HIVE-18038 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 3.0.0 >Reporter: BELUGA BEHR >Assignee: BELUGA BEHR >Priority: Trivial > Attachments: HIVE-18038.1.patch, HIVE-18038.2.patch, > HIVE-18038.3.patch, HIVE-18038.4.patch, HIVE-18038.5.patch > > > Simplifications, improve readability -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18545) Add UDF to parse complex types from json
[ https://issues.apache.org/jira/browse/HIVE-18545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-18545: Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) pushed to master. Thank you Ashutosh for reviewing the changes! > Add UDF to parse complex types from json > > > Key: HIVE-18545 > URL: https://issues.apache.org/jira/browse/HIVE-18545 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-18545.02.patch, HIVE-18545.03.patch, > HIVE-18545.04.patch, HIVE-18545.05.patch, HIVE-18545.06.patch, > HIVE-18545.06.patch, HIVE-18545.06.patch, HIVE-18545.07.patch, > HIVE-18545.07.patch, HIVE-18545.07.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20069) Fix reoptimization in case of DPP and Semijoin optimization
[ https://issues.apache.org/jira/browse/HIVE-20069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-20069: Attachment: HIVE-20069.02.patch > Fix reoptimization in case of DPP and Semijoin optimization > --- > > Key: HIVE-20069 > URL: https://issues.apache.org/jira/browse/HIVE-20069 > Project: Hive > Issue Type: Bug > Components: Query Planning >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20069.01.patch, HIVE-20069.02.patch, > HIVE-20069.02.patch, HIVE-20069.02.patch, HIVE-20069.02.patch > > > reported by [~t3rmin4t0r] > In case dynamic partition pruning; the operator statistics became partial; to > only reflect the actually scanned partitions; but they are being used as an > information about the "full" table...which leads to the exchange of the 2 > tables being joined...which is really unfortunate... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20094) Update Druid to 0.12.1 version
[ https://issues.apache.org/jira/browse/HIVE-20094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538778#comment-16538778 ] Ashutosh Chauhan commented on HIVE-20094: - Unfortunately a clean run is needed for every commit. Can you please reupload your patch so that it gets a new QA run. > Update Druid to 0.12.1 version > -- > > Key: HIVE-20094 > URL: https://issues.apache.org/jira/browse/HIVE-20094 > Project: Hive > Issue Type: Bug >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Minor > Attachments: HIVE-20094.patch > > > As per Jira title. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20041) ResultsCache: Improve logging for concurrent queries
[
https://issues.apache.org/jira/browse/HIVE-20041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Bodor updated HIVE-20041:
Attachment: HIVE-20041.05.patch
> ResultsCache: Improve logging for concurrent queries
>
>
> Key: HIVE-20041
> URL: https://issues.apache.org/jira/browse/HIVE-20041
> Project: Hive
> Issue Type: Improvement
> Components: Diagnosability
>Reporter: Gopal V
>Assignee: Laszlo Bodor
>Priority: Minor
> Labels: Branch3Candidate
> Attachments: HIVE-20041.01.patch, HIVE-20041.02.patch,
> HIVE-20041.03.patch, HIVE-20041.04.patch, HIVE-20041.05.patch
>
>
> The logging for QueryResultsCache ends up printing information without
> context, like
> {code}
> 2018-06-30T17:48:45,502 INFO [HiveServer2-Background-Pool: Thread-166]
> results.QueryResultsCache: Waiting on pending cacheEntry
> {code}
> {code}
> 2018-06-30T17:50:17,963 INFO [HiveServer2-Background-Pool: Thread-145]
> ql.Driver: savedToCache: true
> {code}
> The previous lines for this are in DEBUG level, so the logging ends up being
> useless at INFO level to debug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20094) Update Druid to 0.12.1 version
[ https://issues.apache.org/jira/browse/HIVE-20094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-20094: -- Attachment: HIVE-20094.patch > Update Druid to 0.12.1 version > -- > > Key: HIVE-20094 > URL: https://issues.apache.org/jira/browse/HIVE-20094 > Project: Hive > Issue Type: Bug >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Minor > Attachments: HIVE-20094.patch, HIVE-20094.patch > > > As per Jira title. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20095) Fix jdbc external table feature
[
https://issues.apache.org/jira/browse/HIVE-20095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538794#comment-16538794
]
Hive QA commented on HIVE-20095:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
45s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
56s{color} | {color:blue} ql in master has 2287 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
24s{color} | {color:blue} jdbc-handler in master has 8 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
9s{color} | {color:red} jdbc-handler: The patch generated 31 new + 23 unchanged
- 2 fixed = 54 total (was 25) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
30s{color} | {color:red} jdbc-handler generated 3 new + 8 unchanged - 0 fixed =
11 total (was 8) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 35s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:jdbc-handler |
| | Exception is caught when Exception is not thrown in
org.apache.hive.storage.jdbc.JdbcSerDe.initialize(Configuration, Properties)
At JdbcSerDe.java:is not thrown in
org.apache.hive.storage.jdbc.JdbcSerDe.initialize(Configuration, Properties)
At JdbcSerDe.java:[line 111] |
| |
org.apache.hive.storage.jdbc.dao.GenericJdbcDatabaseAccessor.getColumnTypes(Configuration)
may fail to clean up java.sql.ResultSet Obligation to clean up resource
created at GenericJdbcDatabaseAccessor.java:up java.sql.ResultSet Obligation
to clean up resource created at GenericJdbcDatabaseAccessor.java:[line 115] is
not discharged |
| |
org.apache.hive.storage.jdbc.dao.GenericJdbcDatabaseAccessor.getColumnTypes(Configuration)
may fail to clean up java.sql.Statement Obligation to clean up resource
created at GenericJdbcDatabaseAccessor.java:up java.sql.Statement Obligation
to clean up resource created at GenericJdbcDatabaseAccessor.java:[line 114] is
not discharged |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12512/dev-support/hive-personality.sh
|
| git revision | master / 18fb1b3 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12512/yetus/diff-checkstyle-jdbc-handler.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12512/yetus/new-findbugs-jdbc-handler.html
|
| modules | C: ql jdbc-handler itests U: . |
| Console output |
http://104.198.109.242/log
[jira] [Updated] (HIVE-19441) Add support for float aggregator and use LLAP test Driver
[
https://issues.apache.org/jira/browse/HIVE-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
slim bouguerra updated HIVE-19441:
--
Attachment: HIVE-19441.9.patch
> Add support for float aggregator and use LLAP test Driver
> -
>
> Key: HIVE-19441
> URL: https://issues.apache.org/jira/browse/HIVE-19441
> Project: Hive
> Issue Type: Improvement
> Components: Druid integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-19441.2.patch, HIVE-19441.3.patch,
> HIVE-19441.4.patch, HIVE-19441.5.patch, HIVE-19441.6.patch,
> HIVE-19441.7.patch, HIVE-19441.8.patch, HIVE-19441.8.patch,
> HIVE-19441.9.patch, HIVE-19441.patch
>
>
> Adding support to the float kind aggregator.
> Use LLAP as test Driver to reduce execution time of tests from about 2 hours
> to 15 min:
> Although this patches unveiling an issue with timezone, maybe it is fixed by
> [~jcamachorodriguez] upcoming set of patches.
>
> Before
> {code}
> [INFO] Executed tasks
> [INFO]
> [INFO] --- maven-compiler-plugin:3.6.1:testCompile (default-testCompile) @
> hive-it-qfile ---
> [INFO] Compiling 21 source files to
> /Users/sbouguerra/Hdev/hive/itests/qtest/target/test-classes
> [INFO]
> [INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ hive-it-qfile
> ---
> [INFO]
> [INFO] ---
> [INFO] T E S T S
> [INFO] ---
> [INFO] Running org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
> [INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 6,654.117 s - in org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
> [INFO]
> [INFO] Results:
> [INFO]
> [INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0
> [INFO]
> [INFO]
>
> [INFO] BUILD SUCCESS
> [INFO]
>
> [INFO] Total time: 01:51 h
> [INFO] Finished at: 2018-05-04T12:43:19-07:00
> [INFO]
>
> {code}
> After
> {code}
> INFO] Executed tasks
> [INFO]
> [INFO] --- maven-compiler-plugin:3.6.1:testCompile (default-testCompile) @
> hive-it-qfile ---
> [INFO] Compiling 22 source files to
> /Users/sbouguerra/Hdev/hive/itests/qtest/target/test-classes
> [INFO]
> [INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ hive-it-qfile
> ---
> [INFO]
> [INFO] ---
> [INFO] T E S T S
> [INFO] ---
> [INFO] Running org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
> [INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 907.167 s - in org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
> [INFO]
> [INFO] Results:
> [INFO]
> [INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0
> [INFO]
> [INFO]
>
> [INFO] BUILD SUCCESS
> [INFO]
>
> [INFO] Total time: 15:31 min
> [INFO] Finished at: 2018-05-04T13:15:11-07:00
> [INFO]
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19441) Add support for float aggregator and use LLAP test Driver
[
https://issues.apache.org/jira/browse/HIVE-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538839#comment-16538839
]
slim bouguerra commented on HIVE-19441:
---
updated the patch, for branch master commit
1105ef3974d8a324637d3d35881a739af3aeb382
> Add support for float aggregator and use LLAP test Driver
> -
>
> Key: HIVE-19441
> URL: https://issues.apache.org/jira/browse/HIVE-19441
> Project: Hive
> Issue Type: Improvement
> Components: Druid integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-19441.2.patch, HIVE-19441.3.patch,
> HIVE-19441.4.patch, HIVE-19441.5.patch, HIVE-19441.6.patch,
> HIVE-19441.7.patch, HIVE-19441.8.patch, HIVE-19441.8.patch,
> HIVE-19441.9.patch, HIVE-19441.patch
>
>
> Adding support to the float kind aggregator.
> Use LLAP as test Driver to reduce execution time of tests from about 2 hours
> to 15 min:
> Although this patches unveiling an issue with timezone, maybe it is fixed by
> [~jcamachorodriguez] upcoming set of patches.
>
> Before
> {code}
> [INFO] Executed tasks
> [INFO]
> [INFO] --- maven-compiler-plugin:3.6.1:testCompile (default-testCompile) @
> hive-it-qfile ---
> [INFO] Compiling 21 source files to
> /Users/sbouguerra/Hdev/hive/itests/qtest/target/test-classes
> [INFO]
> [INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ hive-it-qfile
> ---
> [INFO]
> [INFO] ---
> [INFO] T E S T S
> [INFO] ---
> [INFO] Running org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
> [INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 6,654.117 s - in org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
> [INFO]
> [INFO] Results:
> [INFO]
> [INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0
> [INFO]
> [INFO]
>
> [INFO] BUILD SUCCESS
> [INFO]
>
> [INFO] Total time: 01:51 h
> [INFO] Finished at: 2018-05-04T12:43:19-07:00
> [INFO]
>
> {code}
> After
> {code}
> INFO] Executed tasks
> [INFO]
> [INFO] --- maven-compiler-plugin:3.6.1:testCompile (default-testCompile) @
> hive-it-qfile ---
> [INFO] Compiling 22 source files to
> /Users/sbouguerra/Hdev/hive/itests/qtest/target/test-classes
> [INFO]
> [INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ hive-it-qfile
> ---
> [INFO]
> [INFO] ---
> [INFO] T E S T S
> [INFO] ---
> [INFO] Running org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
> [INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 907.167 s - in org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
> [INFO]
> [INFO] Results:
> [INFO]
> [INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0
> [INFO]
> [INFO]
>
> [INFO] BUILD SUCCESS
> [INFO]
>
> [INFO] Total time: 15:31 min
> [INFO] Finished at: 2018-05-04T13:15:11-07:00
> [INFO]
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19441) Add support for float aggregator and use LLAP test Driver
[
https://issues.apache.org/jira/browse/HIVE-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
slim bouguerra updated HIVE-19441:
--
Target Version/s: 4.0.0 (was: 3.1.0)
> Add support for float aggregator and use LLAP test Driver
> -
>
> Key: HIVE-19441
> URL: https://issues.apache.org/jira/browse/HIVE-19441
> Project: Hive
> Issue Type: Improvement
> Components: Druid integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-19441.2.patch, HIVE-19441.3.patch,
> HIVE-19441.4.patch, HIVE-19441.5.patch, HIVE-19441.6.patch,
> HIVE-19441.7.patch, HIVE-19441.8.patch, HIVE-19441.8.patch,
> HIVE-19441.9.patch, HIVE-19441.patch
>
>
> Adding support to the float kind aggregator.
> Use LLAP as test Driver to reduce execution time of tests from about 2 hours
> to 15 min:
> Although this patches unveiling an issue with timezone, maybe it is fixed by
> [~jcamachorodriguez] upcoming set of patches.
>
> Before
> {code}
> [INFO] Executed tasks
> [INFO]
> [INFO] --- maven-compiler-plugin:3.6.1:testCompile (default-testCompile) @
> hive-it-qfile ---
> [INFO] Compiling 21 source files to
> /Users/sbouguerra/Hdev/hive/itests/qtest/target/test-classes
> [INFO]
> [INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ hive-it-qfile
> ---
> [INFO]
> [INFO] ---
> [INFO] T E S T S
> [INFO] ---
> [INFO] Running org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
> [INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 6,654.117 s - in org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
> [INFO]
> [INFO] Results:
> [INFO]
> [INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0
> [INFO]
> [INFO]
>
> [INFO] BUILD SUCCESS
> [INFO]
>
> [INFO] Total time: 01:51 h
> [INFO] Finished at: 2018-05-04T12:43:19-07:00
> [INFO]
>
> {code}
> After
> {code}
> INFO] Executed tasks
> [INFO]
> [INFO] --- maven-compiler-plugin:3.6.1:testCompile (default-testCompile) @
> hive-it-qfile ---
> [INFO] Compiling 22 source files to
> /Users/sbouguerra/Hdev/hive/itests/qtest/target/test-classes
> [INFO]
> [INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ hive-it-qfile
> ---
> [INFO]
> [INFO] ---
> [INFO] T E S T S
> [INFO] ---
> [INFO] Running org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
> [INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 907.167 s - in org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
> [INFO]
> [INFO] Results:
> [INFO]
> [INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0
> [INFO]
> [INFO]
>
> [INFO] BUILD SUCCESS
> [INFO]
>
> [INFO] Total time: 15:31 min
> [INFO] Finished at: 2018-05-04T13:15:11-07:00
> [INFO]
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19441) Use LLAP test Driver for Druid test suites
[
https://issues.apache.org/jira/browse/HIVE-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
slim bouguerra updated HIVE-19441:
--
Summary: Use LLAP test Driver for Druid test suites (was: Add support for
float aggregator and use LLAP test Driver)
> Use LLAP test Driver for Druid test suites
> --
>
> Key: HIVE-19441
> URL: https://issues.apache.org/jira/browse/HIVE-19441
> Project: Hive
> Issue Type: Improvement
> Components: Druid integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-19441.2.patch, HIVE-19441.3.patch,
> HIVE-19441.4.patch, HIVE-19441.5.patch, HIVE-19441.6.patch,
> HIVE-19441.7.patch, HIVE-19441.8.patch, HIVE-19441.8.patch,
> HIVE-19441.9.patch, HIVE-19441.patch
>
>
> Adding support to the float kind aggregator.
> Use LLAP as test Driver to reduce execution time of tests from about 2 hours
> to 15 min:
> Although this patches unveiling an issue with timezone, maybe it is fixed by
> [~jcamachorodriguez] upcoming set of patches.
>
> Before
> {code}
> [INFO] Executed tasks
> [INFO]
> [INFO] --- maven-compiler-plugin:3.6.1:testCompile (default-testCompile) @
> hive-it-qfile ---
> [INFO] Compiling 21 source files to
> /Users/sbouguerra/Hdev/hive/itests/qtest/target/test-classes
> [INFO]
> [INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ hive-it-qfile
> ---
> [INFO]
> [INFO] ---
> [INFO] T E S T S
> [INFO] ---
> [INFO] Running org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
> [INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 6,654.117 s - in org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
> [INFO]
> [INFO] Results:
> [INFO]
> [INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0
> [INFO]
> [INFO]
>
> [INFO] BUILD SUCCESS
> [INFO]
>
> [INFO] Total time: 01:51 h
> [INFO] Finished at: 2018-05-04T12:43:19-07:00
> [INFO]
>
> {code}
> After
> {code}
> INFO] Executed tasks
> [INFO]
> [INFO] --- maven-compiler-plugin:3.6.1:testCompile (default-testCompile) @
> hive-it-qfile ---
> [INFO] Compiling 22 source files to
> /Users/sbouguerra/Hdev/hive/itests/qtest/target/test-classes
> [INFO]
> [INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ hive-it-qfile
> ---
> [INFO]
> [INFO] ---
> [INFO] T E S T S
> [INFO] ---
> [INFO] Running org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
> [INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 907.167 s - in org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
> [INFO]
> [INFO] Results:
> [INFO]
> [INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0
> [INFO]
> [INFO]
>
> [INFO] BUILD SUCCESS
> [INFO]
>
> [INFO] Total time: 15:31 min
> [INFO] Finished at: 2018-05-04T13:15:11-07:00
> [INFO]
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-20135) Fix incompatible change in TimestampColumnVector to default to UTC
[ https://issues.apache.org/jira/browse/HIVE-20135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley reassigned HIVE-20135: > Fix incompatible change in TimestampColumnVector to default to UTC > -- > > Key: HIVE-20135 > URL: https://issues.apache.org/jira/browse/HIVE-20135 > Project: Hive > Issue Type: Improvement >Reporter: Owen O'Malley >Assignee: Jesus Camacho Rodriguez >Priority: Blocker > > HIVE-20007 changed the default for TimestampColumnVector to be to use UTC, > which breaks the API compatibility with storage-api 2.6. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20135) Fix incompatible change in TimestampColumnVector to default to UTC
[ https://issues.apache.org/jira/browse/HIVE-20135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-20135: - Fix Version/s: storage-2.7.0 > Fix incompatible change in TimestampColumnVector to default to UTC > -- > > Key: HIVE-20135 > URL: https://issues.apache.org/jira/browse/HIVE-20135 > Project: Hive > Issue Type: Improvement >Reporter: Owen O'Malley >Assignee: Jesus Camacho Rodriguez >Priority: Blocker > Fix For: storage-2.7.0 > > > HIVE-20007 changed the default for TimestampColumnVector to be to use UTC, > which breaks the API compatibility with storage-api 2.6. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20135) Fix incompatible change in TimestampColumnVector to default to UTC
[ https://issues.apache.org/jira/browse/HIVE-20135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538847#comment-16538847 ] Owen O'Malley commented on HIVE-20135: -- We need to have Hive always set the useUTC, but not change the default behavior. > Fix incompatible change in TimestampColumnVector to default to UTC > -- > > Key: HIVE-20135 > URL: https://issues.apache.org/jira/browse/HIVE-20135 > Project: Hive > Issue Type: Improvement >Reporter: Owen O'Malley >Assignee: Jesus Camacho Rodriguez >Priority: Blocker > Fix For: storage-2.7.0 > > > HIVE-20007 changed the default for TimestampColumnVector to be to use UTC, > which breaks the API compatibility with storage-api 2.6. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20112) Accumulo-Hive (managed) table creation fails with strict managed table checks: Table is marked as a managed table but is not transactional
[ https://issues.apache.org/jira/browse/HIVE-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20112: --- Resolution: Fixed Fix Version/s: (was: 3.0.0) 3.2.0 4.0.0 Status: Resolved (was: Patch Available) > Accumulo-Hive (managed) table creation fails with strict managed table > checks: Table is marked as a managed table but is not transactional > -- > > Key: HIVE-20112 > URL: https://issues.apache.org/jira/browse/HIVE-20112 > Project: Hive > Issue Type: Bug > Components: Hive, StorageHandler >Affects Versions: 3.0.0 >Reporter: Romil Choksi >Assignee: Jesus Camacho Rodriguez >Priority: Major > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-20112.01.patch, HIVE-20112.patch > > > Similar to HIVE-20085 and HIVE-20111. Accumulo-Hive (managed) table creation > fails with strict managed table checks: Table is marked as a managed table > but is not transactional -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20095) Fix jdbc external table feature
[
https://issues.apache.org/jira/browse/HIVE-20095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538871#comment-16538871
]
Hive QA commented on HIVE-20095:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12930997/HIVE-20095.2.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 14623 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[external_jdbc_table]
(batchId=9)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[strict_managed_tables_sysdb]
(batchId=169)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12512/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12512/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12512/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12930997 - PreCommit-HIVE-Build
> Fix jdbc external table feature
> ---
>
> Key: HIVE-20095
> URL: https://issues.apache.org/jira/browse/HIVE-20095
> Project: Hive
> Issue Type: Bug
>Reporter: Jonathan Doron
>Assignee: Jonathan Doron
>Priority: Major
> Attachments: HIVE-20095.1.patch, HIVE-20095.2.patch
>
>
> It seems like the committed code for HIVE-19161
> (7584b3276bebf64aa006eaa162c0a6264d8fcb56) reverted some of HIVE-18423
> updates, and therefore some of the external table queries are not working
> correctly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-20136) Code Review of ArchiveUtils Class
[
https://issues.apache.org/jira/browse/HIVE-20136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR reassigned HIVE-20136:
--
Assignee: BELUGA BEHR
> Code Review of ArchiveUtils Class
> -
>
> Key: HIVE-20136
> URL: https://issues.apache.org/jira/browse/HIVE-20136
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
>
> General code review of {{ArchiveUtil}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20136) Code Review of ArchiveUtils Class
[
https://issues.apache.org/jira/browse/HIVE-20136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HIVE-20136:
---
Attachment: HIVE-20136.1.patch
> Code Review of ArchiveUtils Class
> -
>
> Key: HIVE-20136
> URL: https://issues.apache.org/jira/browse/HIVE-20136
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20136.1.patch
>
>
> General code review of {{ArchiveUtil}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20136) Code Review of ArchiveUtils Class
[
https://issues.apache.org/jira/browse/HIVE-20136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HIVE-20136:
---
Status: Patch Available (was: Open)
> Code Review of ArchiveUtils Class
> -
>
> Key: HIVE-20136
> URL: https://issues.apache.org/jira/browse/HIVE-20136
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20136.1.patch
>
>
> General code review of {{ArchiveUtil}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19846) Removed Deprecated Calls From FileUtils-getJarFilesByPath
[
https://issues.apache.org/jira/browse/HIVE-19846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538900#comment-16538900
]
Hive QA commented on HIVE-19846:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} common in master has 64 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} common: The patch generated 0 new + 27 unchanged - 1
fixed = 27 total (was 28) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 11m 16s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12513/dev-support/hive-personality.sh
|
| git revision | master / c4c55d2 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: common U: common |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12513/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Removed Deprecated Calls From FileUtils-getJarFilesByPath
> -
>
> Key: HIVE-19846
> URL: https://issues.apache.org/jira/browse/HIVE-19846
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-19846.1.patch, HIVE-19846.2.patch,
> HIVE-19846.3.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20129) Revert to position based schema evolution for orc tables
[ https://issues.apache.org/jira/browse/HIVE-20129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-20129: - Attachment: HIVE-20129.2.patch > Revert to position based schema evolution for orc tables > > > Key: HIVE-20129 > URL: https://issues.apache.org/jira/browse/HIVE-20129 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0, 3.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-20129.1.patch, HIVE-20129.2.patch > > > Hive has been doing positional based schema evolution. ORC-54 changed it to > column name based schema evolution causing unexpected results. Queries > returned results earlier are now returning no results. Change the default in > hive to positional schema evolution. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20129) Revert to position based schema evolution for orc tables
[ https://issues.apache.org/jira/browse/HIVE-20129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538927#comment-16538927 ] Prasanth Jayachandran commented on HIVE-20129: -- Previous patch failed because of github outage. Rebased patch. > Revert to position based schema evolution for orc tables > > > Key: HIVE-20129 > URL: https://issues.apache.org/jira/browse/HIVE-20129 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0, 3.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-20129.1.patch, HIVE-20129.2.patch > > > Hive has been doing positional based schema evolution. ORC-54 changed it to > column name based schema evolution causing unexpected results. Queries > returned results earlier are now returning no results. Change the default in > hive to positional schema evolution. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-19902) Provide Metastore micro-benchmarks
[
https://issues.apache.org/jira/browse/HIVE-19902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538975#comment-16538975
]
Alexander Kolbasov edited comment on HIVE-19902 at 7/10/18 5:15 PM:
An example output:
{code}
Operation Mean Med Min Max Err%
addPartition 16.9716.8913.8427.108.849
addPartitions.100 315.9313.7274.2387.06.485
addPartitions.1000 3016 3017 2854 3226 2.861
concurrentPartitionAdd#2.100 1289 1289 1158 1434 4.872
concurrentPartitionAdd#2.1000 1.221e+04 1.226e+04 1.074e+04 1.354e+04 5.077
createTable18.2118.1514.7824.1710.30
dropDatabase 31.1330.8626.4639.098.192
dropDatabase.100 1436 1435 1165 1637 5.929
dropDatabase.1000 1.376e+04 1.371e+04 1.272e+04 1.516e+04 3.864
dropPartition 29.4328.8124.7963.2413.97
dropPartitions.100 686.5680.3575.1819.86.544
dropPartitions.10006247 6166 5616 7535 6.435
dropTable 27.5327.3423.2335.359.241
dropTableWithPartitions36.4136.1931.3350.418.310
dropTableWithPartitions.100793.3792.0687.9987.47.293
dropTableWithPartitions.1000 6981 6964 6336 9179 5.115
getNid 0.6760 0.6512 0.4482 1.53021.93
getPartition 6.2426.2275.1559.79111.27
getPartitionNames 4.8884.6603.84213.1222.53
getPartitionNames.100 5.0314.9573.9957.15610.77
getPartitionNames.1000 8.9988.9158.01612.657.520
getPartitions.100 9.7179.4757.88313.089.835
getPartitions.1000 32.6032.0328.3050.029.036
getPartitionsByNames 6.5066.3844.8109.50315.51
getPartitionsByNames.100 9.3129.0257.95518.4414.46
getPartitionsByNames.1000 38.4737.4934.5762.5110.23
getTable 4.0923.8683.13212.2024.56
listDatabases 0.6919 0.6835 0.5309 1.05312.25
listPartition 5.5565.4654.7377.96910.00
listPartitions.100 9.0878.8747.63012.1310.86
listPartitions.100033.7932.5528.6346.1511.14
listTables 0.9851 0.9761 0.7948 1.37812.07
listTables.100 1.4161.3741.0513.22816.68
listTables.10004.3274.1833.4846.60414.38
renameTable46.6746.0940.1662.467.536
renameTable.100915.8915.9831.01022 3.833
renameTable.1000 9015 8972 8073 1.137e+04 4.228
{code}
was (Author: akolb):
An example output:
{code}
Operation Mean Med Min Max Err%
addPartition 16.9716.8913.8427.108.849
addPartitions.100 315.9313.7274.2387.06.485
addPartitions.1000 3016 3017 2854 3226 2.861
concurrentPartitionAdd#2.100 1289 1289 1158 1434 4.872
concurrentPartitionAdd#2.1000 1.221e+04 1.226e+04 1.074e+04 1.354e+04 5.077
createTable18.2118.1514.7824.1710.30
dropDatabase 31.1330.8626.4639.098.192
dropDatabase.100 1436 1435 1165 1637 5.929
dropDatabase.1000 1.376e+04 1.371e+04 1.272e+04 1.516e+04 3.864
dropPartition 29.4328.8124.7963.2413.97
dropPartitions.100 686.5680.3575.1819.86.544
dropPartitions.10006247 6166 5616 7535 6.435
dropTable 27.5327.3423.2335.359.241
dropTableWithPartitions36.4136.1931.3350.418.310
dropTableWithPartitions.100793.3792.0687.9987.47.293
dropTableWithPartitions.1000 6981 6964 6336 9179 5.115
getNid 0.6760 0.6512 0.4482 1.53021.93
getPartition 6.2426.2275.1559.79111.27
getPartitionNames 4.8884.6603.84213.1222.53
getPartitionNames.100 5.0314.9573.9957.15610.77
getPartitionNames.1000 8.9988.9
[jira] [Commented] (HIVE-19902) Provide Metastore micro-benchmarks
[
https://issues.apache.org/jira/browse/HIVE-19902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538975#comment-16538975
]
Alexander Kolbasov commented on HIVE-19902:
---
An example output:
{code}
Operation Mean Med Min Max Err%
addPartition 16.9716.8913.8427.108.849
addPartitions.100 315.9313.7274.2387.06.485
addPartitions.1000 3016 3017 2854 3226 2.861
concurrentPartitionAdd#2.100 1289 1289 1158 1434 4.872
concurrentPartitionAdd#2.1000 1.221e+04 1.226e+04 1.074e+04 1.354e+04 5.077
createTable18.2118.1514.7824.1710.30
dropDatabase 31.1330.8626.4639.098.192
dropDatabase.100 1436 1435 1165 1637 5.929
dropDatabase.1000 1.376e+04 1.371e+04 1.272e+04 1.516e+04 3.864
dropPartition 29.4328.8124.7963.2413.97
dropPartitions.100 686.5680.3575.1819.86.544
dropPartitions.10006247 6166 5616 7535 6.435
dropTable 27.5327.3423.2335.359.241
dropTableWithPartitions36.4136.1931.3350.418.310
dropTableWithPartitions.100793.3792.0687.9987.47.293
dropTableWithPartitions.1000 6981 6964 6336 9179 5.115
getNid 0.6760 0.6512 0.4482 1.53021.93
getPartition 6.2426.2275.1559.79111.27
getPartitionNames 4.8884.6603.84213.1222.53
getPartitionNames.100 5.0314.9573.9957.15610.77
getPartitionNames.1000 8.9988.9158.01612.657.520
getPartitions.100 9.7179.4757.88313.089.835
getPartitions.1000 32.6032.0328.3050.029.036
getPartitionsByNames 6.5066.3844.8109.50315.51
getPartitionsByNames.100 9.3129.0257.95518.4414.46
getPartitionsByNames.1000 38.4737.4934.5762.5110.23
getTable 4.0923.8683.13212.2024.56
listDatabases 0.6919 0.6835 0.5309 1.05312.25
listPartition 5.5565.4654.7377.96910.00
listPartitions.100 9.0878.8747.63012.1310.86
listPartitions.100033.7932.5528.6346.1511.14
listTables 0.9851 0.9761 0.7948 1.37812.07
listTables.100 1.4161.3741.0513.22816.68
listTables.10004.3274.1833.4846.60414.38
renameTable46.6746.0940.1662.467.536
renameTable.100915.8915.9831.01022 3.833
renameTable.1000 9015 8972 8073 1.137e+04 4.228
{code}
> Provide Metastore micro-benchmarks
> --
>
> Key: HIVE-19902
> URL: https://issues.apache.org/jira/browse/HIVE-19902
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Alexander Kolbasov
>Assignee: Alexander Kolbasov
>Priority: Major
> Attachments: HIVE-19902.01.patch, HIVE-19902.02.patch,
> HIVE-19902.03.patch
>
>
> It would be very useful to have metastore benchmarks to be able to track perf
> issues.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-19190) Improve Logging for SemanticException Handling
[
https://issues.apache.org/jira/browse/HIVE-19190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alice Fan reassigned HIVE-19190:
Assignee: Alice Fan
> Improve Logging for SemanticException Handling
> --
>
> Key: HIVE-19190
> URL: https://issues.apache.org/jira/browse/HIVE-19190
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 2.3.2
>Reporter: BELUGA BEHR
>Assignee: Alice Fan
>Priority: Major
> Attachments: table_not_found_example.txt
>
>
> Please improve the logging for queries that fail with SemanticException. For
> example, when performing an action on a table that does not exist. The most
> common reason why this happens is that someone fat fingers the table/column
> name. It is not a system error pe se, but a user validation error. This is
> not something the cluster administrator should have to worry about. Yet,
> Hive performs some pretty extreme logging on the matter. I have attached to
> this JIRA the logging produced by a single submission of the following query:
> {code:sql}
> select * from madeup;
> {code}
>
> For SemanticException exceptions, please print the Exception {{getMessage()}}
> to the server INFO logging so that the query's life-cycle can be traced, but
> do not blast ERRORs and stack traces to the log file.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20112) Accumulo-Hive (managed) table creation fails with strict managed table checks: Table is marked as a managed table but is not transactional
[ https://issues.apache.org/jira/browse/HIVE-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538989#comment-16538989 ] Alan Gates commented on HIVE-20112: --- The port of this to branch-3 broke branch-3. I've spoken to [~jcamachorodriguez] and am reverting this from branch-3 only. > Accumulo-Hive (managed) table creation fails with strict managed table > checks: Table is marked as a managed table but is not transactional > -- > > Key: HIVE-20112 > URL: https://issues.apache.org/jira/browse/HIVE-20112 > Project: Hive > Issue Type: Bug > Components: Hive, StorageHandler >Affects Versions: 3.0.0 >Reporter: Romil Choksi >Assignee: Jesus Camacho Rodriguez >Priority: Major > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-20112.01.patch, HIVE-20112.patch > > > Similar to HIVE-20085 and HIVE-20111. Accumulo-Hive (managed) table creation > fails with strict managed table checks: Table is marked as a managed table > but is not transactional -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19846) Removed Deprecated Calls From FileUtils-getJarFilesByPath
[
https://issues.apache.org/jira/browse/HIVE-19846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538993#comment-16538993
]
Hive QA commented on HIVE-19846:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12931011/HIVE-19846.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 14637 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitions
(batchId=249)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitionsUnionAll
(batchId=249)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomNonExistent
(batchId=249)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighBytesRead
(batchId=249)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighShuffleBytes
(batchId=249)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerSlowQueryElapsedTime
(batchId=249)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerSlowQueryExecutionTime
(batchId=249)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12513/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12513/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12513/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12931011 - PreCommit-HIVE-Build
> Removed Deprecated Calls From FileUtils-getJarFilesByPath
> -
>
> Key: HIVE-19846
> URL: https://issues.apache.org/jira/browse/HIVE-19846
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-19846.1.patch, HIVE-19846.2.patch,
> HIVE-19846.3.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-17852) remove support for list bucketing "stored as directories" in 3.0
[ https://issues.apache.org/jira/browse/HIVE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-17852: Attachment: HIVE-17852.18.patch > remove support for list bucketing "stored as directories" in 3.0 > > > Key: HIVE-17852 > URL: https://issues.apache.org/jira/browse/HIVE-17852 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Laszlo Bodor >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-17852.01.patch, HIVE-17852.02.patch, > HIVE-17852.03.patch, HIVE-17852.04.patch, HIVE-17852.05.patch, > HIVE-17852.06.patch, HIVE-17852.07.patch, HIVE-17852.08.patch, > HIVE-17852.09.patch, HIVE-17852.10.patch, HIVE-17852.11.patch, > HIVE-17852.12.patch, HIVE-17852.13.patch, HIVE-17852.14.patch, > HIVE-17852.15.patch, HIVE-17852.16.patch, HIVE-17852.17.patch > > > From the email thread: > 1) LB, when stored as directories, adds a lot of low-level complexity to Hive > tables that has to be accounted for in many places in the code where the > files are written or modified - from FSOP to ACID/replication/export. > 2) While working on some FSOP code I noticed that some of that logic is > broken - e.g. the duplicate file removal from tasks, a pretty fundamental > correctness feature in Hive, may be broken. LB also doesn’t appear to be > compatible with e.g. regular bucketing. > 3) The feature hasn’t seen development activity in a while; it also doesn’t > appear to be used a lot. > Keeping with the theme of cleaning up “legacy” code for 3.0, I was proposing > we remove it. > (2) also suggested that, if needed, it might be easier to implement similar > functionality by adding some flexibility to partitions (which LB directories > look like anyway); that would also keep the logic on a higher level of > abstraction (split generation, partition pruning) as opposed to many > low-level places like FSOP, etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17852) remove support for list bucketing "stored as directories" in 3.0
[ https://issues.apache.org/jira/browse/HIVE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-17852: Attachment: (was: HIVE-17852.18.patch) > remove support for list bucketing "stored as directories" in 3.0 > > > Key: HIVE-17852 > URL: https://issues.apache.org/jira/browse/HIVE-17852 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Laszlo Bodor >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-17852.01.patch, HIVE-17852.02.patch, > HIVE-17852.03.patch, HIVE-17852.04.patch, HIVE-17852.05.patch, > HIVE-17852.06.patch, HIVE-17852.07.patch, HIVE-17852.08.patch, > HIVE-17852.09.patch, HIVE-17852.10.patch, HIVE-17852.11.patch, > HIVE-17852.12.patch, HIVE-17852.13.patch, HIVE-17852.14.patch, > HIVE-17852.15.patch, HIVE-17852.16.patch, HIVE-17852.17.patch > > > From the email thread: > 1) LB, when stored as directories, adds a lot of low-level complexity to Hive > tables that has to be accounted for in many places in the code where the > files are written or modified - from FSOP to ACID/replication/export. > 2) While working on some FSOP code I noticed that some of that logic is > broken - e.g. the duplicate file removal from tasks, a pretty fundamental > correctness feature in Hive, may be broken. LB also doesn’t appear to be > compatible with e.g. regular bucketing. > 3) The feature hasn’t seen development activity in a while; it also doesn’t > appear to be used a lot. > Keeping with the theme of cleaning up “legacy” code for 3.0, I was proposing > we remove it. > (2) also suggested that, if needed, it might be easier to implement similar > functionality by adding some flexibility to partitions (which LB directories > look like anyway); that would also keep the logic on a higher level of > abstraction (split generation, partition pruning) as opposed to many > low-level places like FSOP, etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17852) remove support for list bucketing "stored as directories" in 3.0
[ https://issues.apache.org/jira/browse/HIVE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-17852: Attachment: HIVE-17852.18.patch > remove support for list bucketing "stored as directories" in 3.0 > > > Key: HIVE-17852 > URL: https://issues.apache.org/jira/browse/HIVE-17852 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Laszlo Bodor >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-17852.01.patch, HIVE-17852.02.patch, > HIVE-17852.03.patch, HIVE-17852.04.patch, HIVE-17852.05.patch, > HIVE-17852.06.patch, HIVE-17852.07.patch, HIVE-17852.08.patch, > HIVE-17852.09.patch, HIVE-17852.10.patch, HIVE-17852.11.patch, > HIVE-17852.12.patch, HIVE-17852.13.patch, HIVE-17852.14.patch, > HIVE-17852.15.patch, HIVE-17852.16.patch, HIVE-17852.17.patch, > HIVE-17852.18.patch > > > From the email thread: > 1) LB, when stored as directories, adds a lot of low-level complexity to Hive > tables that has to be accounted for in many places in the code where the > files are written or modified - from FSOP to ACID/replication/export. > 2) While working on some FSOP code I noticed that some of that logic is > broken - e.g. the duplicate file removal from tasks, a pretty fundamental > correctness feature in Hive, may be broken. LB also doesn’t appear to be > compatible with e.g. regular bucketing. > 3) The feature hasn’t seen development activity in a while; it also doesn’t > appear to be used a lot. > Keeping with the theme of cleaning up “legacy” code for 3.0, I was proposing > we remove it. > (2) also suggested that, if needed, it might be easier to implement similar > functionality by adding some flexibility to partitions (which LB directories > look like anyway); that would also keep the logic on a higher level of > abstraction (split generation, partition pruning) as opposed to many > low-level places like FSOP, etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19983) Backport HIVE-19769 to branch-3
[ https://issues.apache.org/jira/browse/HIVE-19983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-19983: -- Resolution: Fixed Fix Version/s: 3.2.0 Status: Resolved (was: Patch Available) > Backport HIVE-19769 to branch-3 > --- > > Key: HIVE-19983 > URL: https://issues.apache.org/jira/browse/HIVE-19983 > Project: Hive > Issue Type: Bug > Components: storage-api >Affects Versions: 3.2.0 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-19983-branch-3.patch, HIVE-19983.2-branch-3.patch > > > This patch will be needed for other catalog related work to be backported to > branch-3. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19983) Backport HIVE-19769 to branch-3
[ https://issues.apache.org/jira/browse/HIVE-19983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539018#comment-16539018 ] Alan Gates commented on HIVE-19983: --- Got a clean test run locally. Patch committed to branch-3. > Backport HIVE-19769 to branch-3 > --- > > Key: HIVE-19983 > URL: https://issues.apache.org/jira/browse/HIVE-19983 > Project: Hive > Issue Type: Bug > Components: storage-api >Affects Versions: 3.2.0 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-19983-branch-3.patch, HIVE-19983.2-branch-3.patch > > > This patch will be needed for other catalog related work to be backported to > branch-3. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20058) Backport HIVE-19711 (HiveSchemaTool refactor) to branch-3
[ https://issues.apache.org/jira/browse/HIVE-20058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-20058: -- Resolution: Fixed Fix Version/s: 3.2.0 Status: Resolved (was: Patch Available) Got a clean local run. Patch committed to branch-3. > Backport HIVE-19711 (HiveSchemaTool refactor) to branch-3 > - > > Key: HIVE-20058 > URL: https://issues.apache.org/jira/browse/HIVE-20058 > Project: Hive > Issue Type: Task > Components: Beeline >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-20058-branch-3.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20069) Fix reoptimization in case of DPP and Semijoin optimization
[
https://issues.apache.org/jira/browse/HIVE-20069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539030#comment-16539030
]
Hive QA commented on HIVE-20069:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
52s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
8s{color} | {color:blue} ql in master has 2287 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
39s{color} | {color:red} ql: The patch generated 2 new + 38 unchanged - 0 fixed
= 40 total (was 38) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 6 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 29 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 19s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12514/dev-support/hive-personality.sh
|
| git revision | master / c4c55d2 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12514/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12514/yetus/whitespace-eol.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12514/yetus/whitespace-tabs.txt
|
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12514/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Fix reoptimization in case of DPP and Semijoin optimization
> ---
>
> Key: HIVE-20069
> URL: https://issues.apache.org/jira/browse/HIVE-20069
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-20069.01.patch, HIVE-20069.02.patch,
> HIVE-20069.02.patch, HIVE-20069.02.patch, HIVE-20069.02.patch
>
>
> reported by [~t3rmin4t0r]
> In case dynamic partition pruning; the operator statistics became partial; to
> only reflect the actually scanned partitions; but they are being used as an
> information about the "full" table...which leads to the exchange of the 2
[jira] [Updated] (HIVE-20135) Fix incompatible change in TimestampColumnVector to default to UTC
[ https://issues.apache.org/jira/browse/HIVE-20135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20135: --- Status: Patch Available (was: In Progress) > Fix incompatible change in TimestampColumnVector to default to UTC > -- > > Key: HIVE-20135 > URL: https://issues.apache.org/jira/browse/HIVE-20135 > Project: Hive > Issue Type: Improvement >Reporter: Owen O'Malley >Assignee: Jesus Camacho Rodriguez >Priority: Blocker > Fix For: storage-2.7.0 > > Attachments: HIVE-20135.patch > > > HIVE-20007 changed the default for TimestampColumnVector to be to use UTC, > which breaks the API compatibility with storage-api 2.6. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20135) Fix incompatible change in TimestampColumnVector to default to UTC
[ https://issues.apache.org/jira/browse/HIVE-20135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20135: --- Attachment: HIVE-20135.patch > Fix incompatible change in TimestampColumnVector to default to UTC > -- > > Key: HIVE-20135 > URL: https://issues.apache.org/jira/browse/HIVE-20135 > Project: Hive > Issue Type: Improvement >Reporter: Owen O'Malley >Assignee: Jesus Camacho Rodriguez >Priority: Blocker > Fix For: storage-2.7.0 > > Attachments: HIVE-20135.patch > > > HIVE-20007 changed the default for TimestampColumnVector to be to use UTC, > which breaks the API compatibility with storage-api 2.6. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HIVE-20135) Fix incompatible change in TimestampColumnVector to default to UTC
[ https://issues.apache.org/jira/browse/HIVE-20135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-20135 started by Jesus Camacho Rodriguez. -- > Fix incompatible change in TimestampColumnVector to default to UTC > -- > > Key: HIVE-20135 > URL: https://issues.apache.org/jira/browse/HIVE-20135 > Project: Hive > Issue Type: Improvement >Reporter: Owen O'Malley >Assignee: Jesus Camacho Rodriguez >Priority: Blocker > Fix For: storage-2.7.0 > > Attachments: HIVE-20135.patch > > > HIVE-20007 changed the default for TimestampColumnVector to be to use UTC, > which breaks the API compatibility with storage-api 2.6. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19846) Removed Deprecated Calls From FileUtils-getJarFilesByPath
[ https://issues.apache.org/jira/browse/HIVE-19846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539053#comment-16539053 ] BELUGA BEHR commented on HIVE-19846: Failed tests are unrelated and are different than the set of tests that failed last time. > Removed Deprecated Calls From FileUtils-getJarFilesByPath > - > > Key: HIVE-19846 > URL: https://issues.apache.org/jira/browse/HIVE-19846 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 3.0.0, 4.0.0 >Reporter: BELUGA BEHR >Assignee: BELUGA BEHR >Priority: Minor > Attachments: HIVE-19846.1.patch, HIVE-19846.2.patch, > HIVE-19846.3.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20069) Fix reoptimization in case of DPP and Semijoin optimization
[
https://issues.apache.org/jira/browse/HIVE-20069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539077#comment-16539077
]
Hive QA commented on HIVE-20069:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12931021/HIVE-20069.02.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14639 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12514/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12514/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12514/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12931021 - PreCommit-HIVE-Build
> Fix reoptimization in case of DPP and Semijoin optimization
> ---
>
> Key: HIVE-20069
> URL: https://issues.apache.org/jira/browse/HIVE-20069
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-20069.01.patch, HIVE-20069.02.patch,
> HIVE-20069.02.patch, HIVE-20069.02.patch, HIVE-20069.02.patch
>
>
> reported by [~t3rmin4t0r]
> In case dynamic partition pruning; the operator statistics became partial; to
> only reflect the actually scanned partitions; but they are being used as an
> information about the "full" table...which leads to the exchange of the 2
> tables being joined...which is really unfortunate...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20100) OpTraits : Select Optraits should stop when a mismatch is detected
[ https://issues.apache.org/jira/browse/HIVE-20100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepak Jaiswal updated HIVE-20100: -- Attachment: HIVE-20100.02-branch-3.patch > OpTraits : Select Optraits should stop when a mismatch is detected > -- > > Key: HIVE-20100 > URL: https://issues.apache.org/jira/browse/HIVE-20100 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-20100.01-branch-3.patch, > HIVE-20100.02-branch-3.patch, HIVE-20100.1.patch, HIVE-20100.2.patch, > HIVE-20100.3.patch, HIVE-20100.4.patch, HIVE-20100.5.patch, > HIVE-20100.6.patch, HIVE-20100.7.patch, HIVE-20100.8.patch > > > The select operator's optraits logic as stated in the comment is, > // For bucket columns > // If all the columns match to the parent, put them in the bucket cols > // else, add empty list. > // For sort columns > // Keep the subset of all the columns as long as order is maintained. > > However, this is not happening due to a bug. The bool found is never reset, > so if a single match is found, the value remains true and allows the optraits > get populated with partial list of columns for bucket col which is incorrect. > This may lead to creation of SMB join which should not happen. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20112) Accumulo-Hive (managed) table creation fails with strict managed table checks: Table is marked as a managed table but is not transactional
[ https://issues.apache.org/jira/browse/HIVE-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20112: --- Fix Version/s: (was: 3.2.0) > Accumulo-Hive (managed) table creation fails with strict managed table > checks: Table is marked as a managed table but is not transactional > -- > > Key: HIVE-20112 > URL: https://issues.apache.org/jira/browse/HIVE-20112 > Project: Hive > Issue Type: Bug > Components: Hive, StorageHandler >Affects Versions: 3.0.0 >Reporter: Romil Choksi >Assignee: Jesus Camacho Rodriguez >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-20112.01.patch, HIVE-20112.patch > > > Similar to HIVE-20085 and HIVE-20111. Accumulo-Hive (managed) table creation > fails with strict managed table checks: Table is marked as a managed table > but is not transactional -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19834) Clear Context Map of Paths to ContentSummary
[
https://issues.apache.org/jira/browse/HIVE-19834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HIVE-19834:
---
Status: Open (was: Patch Available)
> Clear Context Map of Paths to ContentSummary
>
>
> Key: HIVE-19834
> URL: https://issues.apache.org/jira/browse/HIVE-19834
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 2.3.2, 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-19834.1.patch
>
>
> The {{Context}} class has a {{clear}} method which is called. During the
> method, various files are deleted and in-memory maps are cleared. I would
> like to propose that we clear out an additional in-memory map structure that
> may contain a lot of data so that it can be GC'ed asap. This map contains
> mapping of "File Path"->"Content Summary". For a query with a large file
> set, this can be quite large.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19834) Clear Context Map of Paths to ContentSummary
[
https://issues.apache.org/jira/browse/HIVE-19834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HIVE-19834:
---
Attachment: HIVE-19834.2.patch
> Clear Context Map of Paths to ContentSummary
>
>
> Key: HIVE-19834
> URL: https://issues.apache.org/jira/browse/HIVE-19834
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 2.3.2, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-19834.1.patch, HIVE-19834.2.patch
>
>
> The {{Context}} class has a {{clear}} method which is called. During the
> method, various files are deleted and in-memory maps are cleared. I would
> like to propose that we clear out an additional in-memory map structure that
> may contain a lot of data so that it can be GC'ed asap. This map contains
> mapping of "File Path"->"Content Summary". For a query with a large file
> set, this can be quite large.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19834) Clear Context Map of Paths to ContentSummary
[
https://issues.apache.org/jira/browse/HIVE-19834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HIVE-19834:
---
Status: Patch Available (was: Open)
OK, since this object is not re-usable, I suggest that we instead use the
{{Closeable}} interface and use a paradigm that people are more familiar with.
I also wipe out the data structures of interest by simply writing 'null' values
to the variable to allow the data structure to be GC'ed ASAP (for example,
while the scratch directories are being removed).
> Clear Context Map of Paths to ContentSummary
>
>
> Key: HIVE-19834
> URL: https://issues.apache.org/jira/browse/HIVE-19834
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 2.3.2, 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-19834.1.patch, HIVE-19834.2.patch
>
>
> The {{Context}} class has a {{clear}} method which is called. During the
> method, various files are deleted and in-memory maps are cleared. I would
> like to propose that we clear out an additional in-memory map structure that
> may contain a lot of data so that it can be GC'ed asap. This map contains
> mapping of "File Path"->"Content Summary". For a query with a large file
> set, this can be quite large.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18038) org.apache.hadoop.hive.ql.session.OperationLog - Review
[
https://issues.apache.org/jira/browse/HIVE-18038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539108#comment-16539108
]
Hive QA commented on HIVE-18038:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
7s{color} | {color:blue} ql in master has 2287 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} ql: The patch generated 0 new + 4 unchanged - 5
fixed = 4 total (was 9) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
27s{color} | {color:red} ql generated 1 new + 2286 unchanged - 1 fixed = 2287
total (was 2287) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 17s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Found reliance on default encoding in
org.apache.hadoop.hive.ql.session.OperationLog$LogFile.readResults(int):in
org.apache.hadoop.hive.ql.session.OperationLog$LogFile.readResults(int): new
java.io.FileReader(File) At OperationLog.java:[line 189] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12515/dev-support/hive-personality.sh
|
| git revision | master / c4c55d2 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12515/yetus/new-findbugs-ql.html
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12515/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> org.apache.hadoop.hive.ql.session.OperationLog - Review
> ---
>
> Key: HIVE-18038
> URL: https://issues.apache.org/jira/browse/HIVE-18038
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Attachments: HIVE-18038.1.patch, HIVE-18038.2.patch,
> HIVE-18038.3.patch, HIVE-18038.4.patch, HIVE-18038.5.patch
>
>
> Simplifications, improve readability
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-19834) Clear Context Map of Paths to ContentSummary
[
https://issues.apache.org/jira/browse/HIVE-19834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539106#comment-16539106
]
BELUGA BEHR edited comment on HIVE-19834 at 7/10/18 7:09 PM:
-
OK, since this object is not re-usable, I suggest that we instead use the
{{Closeable}} interface and use a paradigm that people are more familiar with.
I also wipe out the data structures of interest by simply writing 'null' values
to the variable to allow the data structure to be GC'ed ASAP (for example,
while the scratch directories are being removed).
In this patch, I also removed the Hadoop StringUtil 'strinfigyException' calls.
It's not needed with slf4j.
was (Author: belugabehr):
OK, since this object is not re-usable, I suggest that we instead use the
{{Closeable}} interface and use a paradigm that people are more familiar with.
I also wipe out the data structures of interest by simply writing 'null' values
to the variable to allow the data structure to be GC'ed ASAP (for example,
while the scratch directories are being removed).
> Clear Context Map of Paths to ContentSummary
>
>
> Key: HIVE-19834
> URL: https://issues.apache.org/jira/browse/HIVE-19834
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 2.3.2, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-19834.1.patch, HIVE-19834.2.patch
>
>
> The {{Context}} class has a {{clear}} method which is called. During the
> method, various files are deleted and in-memory maps are cleared. I would
> like to propose that we clear out an additional in-memory map structure that
> may contain a lot of data so that it can be GC'ed asap. This map contains
> mapping of "File Path"->"Content Summary". For a query with a large file
> set, this can be quite large.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19937) Use BeanSerializer for MapWork to carry calls to String.intern
[
https://issues.apache.org/jira/browse/HIVE-19937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-19937:
Attachment: HIVE-19937.3.patch
> Use BeanSerializer for MapWork to carry calls to String.intern
> --
>
> Key: HIVE-19937
> URL: https://issues.apache.org/jira/browse/HIVE-19937
> Project: Hive
> Issue Type: Improvement
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-19937.1.patch, HIVE-19937.2.patch,
> HIVE-19937.3.patch, report.html
>
>
> When fixing HIVE-16395, we decided that each new Spark task should clone the
> {{JobConf}} object to prevent any {{ConcurrentModificationException}} from
> being thrown. However, setting this variable comes at a cost of storing a
> duplicate {{JobConf}} object for each Spark task. These objects can take up a
> significant amount of memory, we should intern them so that Spark tasks
> running in the same JVM don't store duplicate copies.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19940) Push predicates with deterministic UDFs with RBO
[
https://issues.apache.org/jira/browse/HIVE-19940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539118#comment-16539118
]
Naveen Gangam commented on HIVE-19940:
--
Thank you for the query plan [~janulatha]. It looks good to me as well. Sorry I
took a bit of time to look into the annotation code to ensure that we are not
breaking the current UDFs for customers that may not have annotations defined.
So +1 for me.
> Push predicates with deterministic UDFs with RBO
>
>
> Key: HIVE-19940
> URL: https://issues.apache.org/jira/browse/HIVE-19940
> Project: Hive
> Issue Type: Improvement
>Reporter: Janaki Lahorani
>Assignee: Janaki Lahorani
>Priority: Major
> Attachments: HIVE-19940.1.patch, HIVE-19940.2.patch
>
>
> With RBO, predicates with any UDF doesn't get pushed down. It makes sense to
> not pushdown the predicates with non-deterministic function as the meaning of
> the query changes after the predicate is resolved to use the function. But
> pushing a deterministic function is beneficial.
> Test Case:
> {code}
> set hive.cbo.enable=false;
> CREATE TABLE `testb`(
>`cola` string COMMENT '',
>`colb` string COMMENT '',
>`colc` string COMMENT '')
> PARTITIONED BY (
>`part1` string,
>`part2` string,
>`part3` string)
> STORED AS AVRO;
> CREATE TABLE `testa`(
>`col1` string COMMENT '',
>`col2` string COMMENT '',
>`col3` string COMMENT '',
>`col4` string COMMENT '',
>`col5` string COMMENT '')
> PARTITIONED BY (
>`part1` string,
>`part2` string,
>`part3` string)
> STORED AS AVRO;
> insert into testA partition (part1='US', part2='ABC', part3='123')
> values ('12.34', '100', '200', '300', 'abc'),
> ('12.341', '1001', '2001', '3001', 'abcd');
> insert into testA partition (part1='UK', part2='DEF', part3='123')
> values ('12.34', '100', '200', '300', 'abc'),
> ('12.341', '1001', '2001', '3001', 'abcd');
> insert into testA partition (part1='US', part2='DEF', part3='200')
> values ('12.34', '100', '200', '300', 'abc'),
> ('12.341', '1001', '2001', '3001', 'abcd');
> insert into testA partition (part1='CA', part2='ABC', part3='300')
> values ('12.34', '100', '200', '300', 'abc'),
> ('12.341', '1001', '2001', '3001', 'abcd');
> insert into testB partition (part1='CA', part2='ABC', part3='300')
> values ('600', '700', 'abc'), ('601', '701', 'abcd');
> insert into testB partition (part1='CA', part2='ABC', part3='400')
> values ( '600', '700', 'abc'), ( '601', '701', 'abcd');
> insert into testB partition (part1='UK', part2='PQR', part3='500')
> values ('600', '700', 'abc'), ('601', '701', 'abcd');
> insert into testB partition (part1='US', part2='DEF', part3='200')
> values ( '600', '700', 'abc'), ('601', '701', 'abcd');
> insert into testB partition (part1='US', part2='PQR', part3='123')
> values ( '600', '700', 'abc'), ('601', '701', 'abcd');
> -- views with deterministic functions
> create view viewDeterministicUDFA partitioned on (vpart1, vpart2, vpart3) as
> select
> cast(col1 as decimal(38,18)) as vcol1,
> cast(col2 as decimal(38,18)) as vcol2,
> cast(col3 as decimal(38,18)) as vcol3,
> cast(col4 as decimal(38,18)) as vcol4,
> cast(col5 as char(10)) as vcol5,
> cast(part1 as char(2)) as vpart1,
> cast(part2 as char(3)) as vpart2,
> cast(part3 as char(3)) as vpart3
> from testa
> where part1 in ('US', 'CA');
> create view viewDeterministicUDFB partitioned on (vpart1, vpart2, vpart3) as
> select
> cast(cola as decimal(38,18)) as vcolA,
> cast(colb as decimal(38,18)) as vcolB,
> cast(colc as char(10)) as vcolC,
> cast(part1 as char(2)) as vpart1,
> cast(part2 as char(3)) as vpart2,
> cast(part3 as char(3)) as vpart3
> from testb
> where part1 in ('US', 'CA');
> explain
> select vcol1, vcol2, vcol3, vcola, vcolb
> from viewDeterministicUDFA a inner join viewDeterministicUDFB b
> on a.vpart1 = b.vpart1
> and a.vpart2 = b.vpart2
> and a.vpart3 = b.vpart3
> and a.vpart1 = 'US'
> and a.vpart2 = 'DEF'
> and a.vpart3 = '200';
> {code}
> Plan where the CAST is not pushed down.
> {code}
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Map Operator Tree:
> TableScan
> alias: testa
> filterExpr: (part1) IN ('US', 'CA') (type: boolean)
> Statistics: Num rows: 6 Data size: 13740 Basic stats: COMPLETE
> Column stats: NONE
> Select Operator
> expressions: CAST( col1 AS decimal(38,18)) (type:
> decimal(38,18)), CAST( col2 AS decimal(38,18)) (type: decimal(38,18)), CAST(
> col3 AS decimal(38,18)) (type: decimal(38,18)), CAST( part1 AS CHAR(2))
> (type: char(2)), CAST( part2 AS CHAR(3)) (type: char(3)), CAST( part3 AS
> CHAR(3)) (type: char(3))
> outputColumnNames: _col0, _col1, _col2, _col5, _col6, _col7
> Statisti
[jira] [Commented] (HIVE-19937) Use BeanSerializer for MapWork to carry calls to String.intern
[
https://issues.apache.org/jira/browse/HIVE-19937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539149#comment-16539149
]
Misha Dmitriev commented on HIVE-19937:
---
A few small comments about the last patch:
# In the new *Deserializer code, there are many lines looking like
'partitionDesc.setXXX(partitionDesc.getXXX())'. Such code looks quite
non-obvious, so it would be good to add comments there explaining that this is
done to intern possible duplicate strings.
# Looks like there is a typo: a method that's really a getter has the set...
name:
+ public Map> setEventSourceColumnTypeMap() {
+ return eventSourceColumnTypeMap;
+ }
# Still would be good to publish a jxray report for the same test before and
after this change, to make sure that the problems are indeed fixed.
> Use BeanSerializer for MapWork to carry calls to String.intern
> --
>
> Key: HIVE-19937
> URL: https://issues.apache.org/jira/browse/HIVE-19937
> Project: Hive
> Issue Type: Improvement
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-19937.1.patch, HIVE-19937.2.patch,
> HIVE-19937.3.patch, report.html
>
>
> When fixing HIVE-16395, we decided that each new Spark task should clone the
> {{JobConf}} object to prevent any {{ConcurrentModificationException}} from
> being thrown. However, setting this variable comes at a cost of storing a
> duplicate {{JobConf}} object for each Spark task. These objects can take up a
> significant amount of memory, we should intern them so that Spark tasks
> running in the same JVM don't store duplicate copies.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18038) org.apache.hadoop.hive.ql.session.OperationLog - Review
[ https://issues.apache.org/jira/browse/HIVE-18038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HIVE-18038: --- Attachment: HIVE-18038.6.patch > org.apache.hadoop.hive.ql.session.OperationLog - Review > --- > > Key: HIVE-18038 > URL: https://issues.apache.org/jira/browse/HIVE-18038 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 3.0.0 >Reporter: BELUGA BEHR >Assignee: BELUGA BEHR >Priority: Trivial > Attachments: HIVE-18038.1.patch, HIVE-18038.2.patch, > HIVE-18038.3.patch, HIVE-18038.4.patch, HIVE-18038.5.patch, HIVE-18038.6.patch > > > Simplifications, improve readability -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18038) org.apache.hadoop.hive.ql.session.OperationLog - Review
[ https://issues.apache.org/jira/browse/HIVE-18038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HIVE-18038: --- Status: Open (was: Patch Available) > org.apache.hadoop.hive.ql.session.OperationLog - Review > --- > > Key: HIVE-18038 > URL: https://issues.apache.org/jira/browse/HIVE-18038 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 3.0.0 >Reporter: BELUGA BEHR >Assignee: BELUGA BEHR >Priority: Trivial > Attachments: HIVE-18038.1.patch, HIVE-18038.2.patch, > HIVE-18038.3.patch, HIVE-18038.4.patch, HIVE-18038.5.patch, HIVE-18038.6.patch > > > Simplifications, improve readability -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18038) org.apache.hadoop.hive.ql.session.OperationLog - Review
[ https://issues.apache.org/jira/browse/HIVE-18038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HIVE-18038: --- Status: Patch Available (was: Open) > org.apache.hadoop.hive.ql.session.OperationLog - Review > --- > > Key: HIVE-18038 > URL: https://issues.apache.org/jira/browse/HIVE-18038 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 3.0.0 >Reporter: BELUGA BEHR >Assignee: BELUGA BEHR >Priority: Trivial > Attachments: HIVE-18038.1.patch, HIVE-18038.2.patch, > HIVE-18038.3.patch, HIVE-18038.4.patch, HIVE-18038.5.patch, HIVE-18038.6.patch > > > Simplifications, improve readability -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20097) Convert standalone-metastore to a submodule
[ https://issues.apache.org/jira/browse/HIVE-20097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Kolbasov updated HIVE-20097: -- Attachment: HIVE-20097.04.patch > Convert standalone-metastore to a submodule > --- > > Key: HIVE-20097 > URL: https://issues.apache.org/jira/browse/HIVE-20097 > Project: Hive > Issue Type: Sub-task > Components: Hive, Metastore, Standalone Metastore >Affects Versions: 3.1.0, 4.0.0 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > Attachments: HIVE-20097.01.patch, HIVE-20097.02.patch, > HIVE-20097.03.patch, HIVE-20097.04.patch > > > This is a subtask to stage HIVE-17751 changes into several smaller phases. > The first part is moving existing code in hive-standalone-metastore to a > sub-module. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20097) Convert standalone-metastore to a submodule
[
https://issues.apache.org/jira/browse/HIVE-20097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539166#comment-16539166
]
Alexander Kolbasov commented on HIVE-20097:
---
Patch based on
{code}
* commit c4c55d2abf2f21d776a7f45559084a838bac42eb (origin/master, origin/HEAD)
| Author: Jesus Camacho Rodriguez
| Date: Tue Jul 10 09:03:49 2018 -0700
|
| HIVE-20112: Accumulo-Hive (managed) table creation fails with strict
managed table checks: Table is marked as a managed table but is not
transactional (Jesus Camacho Rodriguez, reviewed by Ashutosh Chauhan)
{code}
> Convert standalone-metastore to a submodule
> ---
>
> Key: HIVE-20097
> URL: https://issues.apache.org/jira/browse/HIVE-20097
> Project: Hive
> Issue Type: Sub-task
> Components: Hive, Metastore, Standalone Metastore
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Alexander Kolbasov
>Assignee: Alexander Kolbasov
>Priority: Major
> Attachments: HIVE-20097.01.patch, HIVE-20097.02.patch,
> HIVE-20097.03.patch, HIVE-20097.04.patch
>
>
> This is a subtask to stage HIVE-17751 changes into several smaller phases.
> The first part is moving existing code in hive-standalone-metastore to a
> sub-module.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20135) Fix incompatible change in TimestampColumnVector to default to UTC
[ https://issues.apache.org/jira/browse/HIVE-20135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20135: --- Attachment: HIVE-20135.patch > Fix incompatible change in TimestampColumnVector to default to UTC > -- > > Key: HIVE-20135 > URL: https://issues.apache.org/jira/browse/HIVE-20135 > Project: Hive > Issue Type: Improvement >Reporter: Owen O'Malley >Assignee: Jesus Camacho Rodriguez >Priority: Blocker > Fix For: storage-2.7.0 > > Attachments: HIVE-20135.patch, HIVE-20135.patch > > > HIVE-20007 changed the default for TimestampColumnVector to be to use UTC, > which breaks the API compatibility with storage-api 2.6. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20135) Fix incompatible change in TimestampColumnVector to default to UTC
[ https://issues.apache.org/jira/browse/HIVE-20135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20135: --- Attachment: (was: HIVE-20135.patch) > Fix incompatible change in TimestampColumnVector to default to UTC > -- > > Key: HIVE-20135 > URL: https://issues.apache.org/jira/browse/HIVE-20135 > Project: Hive > Issue Type: Improvement >Reporter: Owen O'Malley >Assignee: Jesus Camacho Rodriguez >Priority: Blocker > Fix For: storage-2.7.0 > > Attachments: HIVE-20135.patch > > > HIVE-20007 changed the default for TimestampColumnVector to be to use UTC, > which breaks the API compatibility with storage-api 2.6. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18038) org.apache.hadoop.hive.ql.session.OperationLog - Review
[
https://issues.apache.org/jira/browse/HIVE-18038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539177#comment-16539177
]
Hive QA commented on HIVE-18038:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12931018/HIVE-18038.5.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 14627 tests
executed
*Failed tests:*
{noformat}
TestHiveRemote - did not produce a TEST-*.xml file (likely timed out)
(batchId=293)
org.apache.hadoop.hive.ql.exec.tez.TestDynamicPartitionPruner.testSingleSourceMultipleFiltersOrdering1
(batchId=303)
org.apache.hive.service.cli.operation.TestOperationLoggingAPIWithMr.testFetchResultsOfLogWithOrientation
(batchId=244)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12515/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12515/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12515/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12931018 - PreCommit-HIVE-Build
> org.apache.hadoop.hive.ql.session.OperationLog - Review
> ---
>
> Key: HIVE-18038
> URL: https://issues.apache.org/jira/browse/HIVE-18038
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Attachments: HIVE-18038.1.patch, HIVE-18038.2.patch,
> HIVE-18038.3.patch, HIVE-18038.4.patch, HIVE-18038.5.patch, HIVE-18038.6.patch
>
>
> Simplifications, improve readability
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20116) TezTask is using parent logger
[ https://issues.apache.org/jira/browse/HIVE-20116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-20116: - Attachment: HIVE-20116.2.patch > TezTask is using parent logger > -- > > Key: HIVE-20116 > URL: https://issues.apache.org/jira/browse/HIVE-20116 > Project: Hive > Issue Type: Bug > Components: Logging >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-20116.1.patch, HIVE-20116.2.patch > > > TezTask is using parent's logger (Task). It should instead use its own class > name. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20090) Extend creation of semijoin reduction filters to be able to discover new opportunities
[
https://issues.apache.org/jira/browse/HIVE-20090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-20090:
---
Attachment: HIVE-20090.05.patch
> Extend creation of semijoin reduction filters to be able to discover new
> opportunities
> --
>
> Key: HIVE-20090
> URL: https://issues.apache.org/jira/browse/HIVE-20090
> Project: Hive
> Issue Type: Improvement
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20090.01.patch, HIVE-20090.02.patch,
> HIVE-20090.04.patch, HIVE-20090.05.patch
>
>
> Assume the following plan:
> {noformat}
> TS[0] - RS[1] - JOIN[4] - RS[5] - JOIN[8] - FS[9]
> TS[2] - RS[3] - JOIN[4]
> TS[6] - RS[7] - JOIN[8]
> {noformat}
> Currently, {{TS\[6\]}} may only be reduced with the output of {{RS\[5\]}},
> i.e., input to join between both subplans.
> However, it may be useful to consider other possibilities too, e.g., reduced
> by the output of {{RS\[1\]}} or {{RS\[3\]}}. For instance, this is important
> when, given a large plan, an edge between {{RS[5]}} and {{TS[0]}} would
> create a cycle, while an edge between {{RS[1]}} and {{TS[6]}} would not.
> This patch comprises two parts. First, it creates additional predicates when
> possible. Secondly, it removes duplicate semijoin reduction
> branches/predicates, e.g., if another semijoin that consumes the output of
> the same expression already reduces a certain table scan operator (heuristic,
> since this may not result in most efficient plan in all cases). Ultimately,
> the decision on whether to use one or another should be cost-driven
> (follow-up).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19902) Provide Metastore micro-benchmarks
[ https://issues.apache.org/jira/browse/HIVE-19902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Kolbasov updated HIVE-19902: -- Attachment: HIVE-19902.04.patch > Provide Metastore micro-benchmarks > -- > > Key: HIVE-19902 > URL: https://issues.apache.org/jira/browse/HIVE-19902 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Affects Versions: 3.1.0, 4.0.0 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > Attachments: HIVE-19902.01.patch, HIVE-19902.02.patch, > HIVE-19902.03.patch, HIVE-19902.04.patch > > > It would be very useful to have metastore benchmarks to be able to track perf > issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20041) ResultsCache: Improve logging for concurrent queries
[
https://issues.apache.org/jira/browse/HIVE-20041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539216#comment-16539216
]
Hive QA commented on HIVE-20041:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
57s{color} | {color:blue} ql in master has 2287 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 24s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12516/dev-support/hive-personality.sh
|
| git revision | master / c4c55d2 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12516/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ResultsCache: Improve logging for concurrent queries
>
>
> Key: HIVE-20041
> URL: https://issues.apache.org/jira/browse/HIVE-20041
> Project: Hive
> Issue Type: Improvement
> Components: Diagnosability
>Reporter: Gopal V
>Assignee: Laszlo Bodor
>Priority: Minor
> Labels: Branch3Candidate
> Attachments: HIVE-20041.01.patch, HIVE-20041.02.patch,
> HIVE-20041.03.patch, HIVE-20041.04.patch, HIVE-20041.05.patch
>
>
> The logging for QueryResultsCache ends up printing information without
> context, like
> {code}
> 2018-06-30T17:48:45,502 INFO [HiveServer2-Background-Pool: Thread-166]
> results.QueryResultsCache: Waiting on pending cacheEntry
> {code}
> {code}
> 2018-06-30T17:50:17,963 INFO [HiveServer2-Background-Pool: Thread-145]
> ql.Driver: savedToCache: true
> {code}
> The previous lines for this are in DEBUG level, so the logging ends up being
> useless at INFO level to debug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18038) org.apache.hadoop.hive.ql.session.OperationLog - Review
[ https://issues.apache.org/jira/browse/HIVE-18038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HIVE-18038: --- Status: Open (was: Patch Available) > org.apache.hadoop.hive.ql.session.OperationLog - Review > --- > > Key: HIVE-18038 > URL: https://issues.apache.org/jira/browse/HIVE-18038 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 3.0.0 >Reporter: BELUGA BEHR >Assignee: BELUGA BEHR >Priority: Trivial > Attachments: HIVE-18038.1.patch, HIVE-18038.2.patch, > HIVE-18038.3.patch, HIVE-18038.4.patch, HIVE-18038.5.patch, > HIVE-18038.6.patch, HIVE-18038.7.patch > > > Simplifications, improve readability -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18038) org.apache.hadoop.hive.ql.session.OperationLog - Review
[ https://issues.apache.org/jira/browse/HIVE-18038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HIVE-18038: --- Attachment: HIVE-18038.7.patch > org.apache.hadoop.hive.ql.session.OperationLog - Review > --- > > Key: HIVE-18038 > URL: https://issues.apache.org/jira/browse/HIVE-18038 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 3.0.0 >Reporter: BELUGA BEHR >Assignee: BELUGA BEHR >Priority: Trivial > Attachments: HIVE-18038.1.patch, HIVE-18038.2.patch, > HIVE-18038.3.patch, HIVE-18038.4.patch, HIVE-18038.5.patch, > HIVE-18038.6.patch, HIVE-18038.7.patch > > > Simplifications, improve readability -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18038) org.apache.hadoop.hive.ql.session.OperationLog - Review
[ https://issues.apache.org/jira/browse/HIVE-18038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HIVE-18038: --- Status: Patch Available (was: Open) Fixes unit test failure > org.apache.hadoop.hive.ql.session.OperationLog - Review > --- > > Key: HIVE-18038 > URL: https://issues.apache.org/jira/browse/HIVE-18038 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 3.0.0 >Reporter: BELUGA BEHR >Assignee: BELUGA BEHR >Priority: Trivial > Attachments: HIVE-18038.1.patch, HIVE-18038.2.patch, > HIVE-18038.3.patch, HIVE-18038.4.patch, HIVE-18038.5.patch, > HIVE-18038.6.patch, HIVE-18038.7.patch > > > Simplifications, improve readability -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19199) ACID: DbTxnManager heartbeat-service needs static sync init
[
https://issues.apache.org/jira/browse/HIVE-19199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V updated HIVE-19199:
---
Labels: Branch3Candidate (was: )
> ACID: DbTxnManager heartbeat-service needs static sync init
>
>
> Key: HIVE-19199
> URL: https://issues.apache.org/jira/browse/HIVE-19199
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.1.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Labels: Branch3Candidate
> Attachments: HIVE-19199.1.patch, HIVE-19199.1.patch
>
>
> {code}
> private synchronized void initHeartbeatExecutorService() {
> if (heartbeatExecutorService != null &&
> !heartbeatExecutorService.isShutdown()
> && !heartbeatExecutorService.isTerminated()) {
> return;
> }
> ...
> {code}
> heartbeatExecutorService is a static field, so this needs class sync, not
> object sync.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-19360) CBO: Add an "optimizedSQL" to QueryPlan object
[
https://issues.apache.org/jira/browse/HIVE-19360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reassigned HIVE-19360:
--
Assignee: Jesus Camacho Rodriguez (was: Gopal V)
> CBO: Add an "optimizedSQL" to QueryPlan object
> ---
>
> Key: HIVE-19360
> URL: https://issues.apache.org/jira/browse/HIVE-19360
> Project: Hive
> Issue Type: Improvement
> Components: CBO, Diagnosability
>Affects Versions: 3.1.0
>Reporter: Gopal V
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-19360.1.patch, HIVE-19360.2.patch,
> HIVE-19360.3.patch, HIVE-19360.4.patch
>
>
> Calcite RelNodes can be converted back into SQL (as the new JDBC storage
> handler does), which allows Hive to print out the post CBO plan as a SQL
> query instead of having to guess the join orders from the subsequent Tez plan.
> The query generated might not be always valid SQL at this point, but is a
> world ahead of DAG plans in readability.
> Eg. tpc-ds Query4 CTEs gets expanded to
> {code}
> SELECT t16.$f3 customer_preferred_cust_flag
> FROM
> (SELECT t0.c_customer_id $f0,
>SUM((t2.ws_ext_list_price -
> t2.ws_ext_wholesale_cost - t2.ws_ext_discount_amt + t2.ws_ext_sales_price) /
> CAST(2 AS DECIMAL(10, 0))) $f8
>FROM
> (SELECT c_customer_sk,
> c_customer_id,
> c_first_name,
> c_last_name,
> c_preferred_cust_flag,
> c_birth_country,
> c_login,
> c_email_address
> FROM default.customer
> WHERE c_customer_sk IS NOT NULL
> AND c_customer_id IS NOT NULL) t0
>INNER JOIN (
> (SELECT ws_sold_date_sk,
> ws_bill_customer_sk,
> ws_ext_discount_amt,
> ws_ext_sales_price,
> ws_ext_wholesale_cost,
> ws_ext_list_price
> FROM default.web_sales
> WHERE ws_bill_customer_sk IS NOT NULL
> AND ws_sold_date_sk IS NOT NULL) t2
>INNER JOIN
> (SELECT d_date_sk,
> CAST(2002 AS INTEGER) d_year
> FROM default.date_dim
> WHERE d_year = 2002
> AND d_date_sk IS NOT NULL) t4 ON t2.ws_sold_date_sk =
> t4.d_date_sk) ON t0.c_customer_sk = t2.ws_bill_customer_sk
>GROUP BY t0.c_customer_id,
> t0.c_first_name,
> t0.c_last_name,
> t0.c_preferred_cust_flag,
> t0.c_birth_country,
> t0.c_login,
> t0.c_email_address) t7
> INNER JOIN (
> (SELECT t9.c_customer_id $f0,
>t9.c_preferred_cust_flag $f3,
>
> SUM((t11.ss_ext_list_price - t11.ss_ext_wholesale_cost -
> t11.ss_ext_discount_amt + t11.ss_ext_sales_price) / CAST(2 AS DECIMAL(10,
> 0))) $f8
>FROM
> (SELECT c_customer_sk,
> c_customer_id,
> c_first_name,
> c_last_name,
> c_preferred_cust_flag,
> c_birth_country,
> c_login,
> c_email_address
> FROM default.customer
> WHERE c_customer_sk IS NOT NULL
> AND c_customer_id IS NOT NULL) t9
>INNER JOIN (
> (SELECT ss_sold_date_sk,
> ss_customer_sk,
> ss_ext_discount_amt,
> ss_ext_sales_price,
> ss_ext_wholesale_cost,
> ss_ext_list_price
> FROM default.store_sales
> WHERE ss_customer_sk IS NOT NULL
> AND ss_sold_date_sk IS NOT NULL) t11
>INNER JOIN
> (SELECT d_date_sk,
> CAST(2002 AS INTEGER) d_year
> FROM default.date_dim
> WHERE d_year = 2002
> AND d_date_sk IS NOT NULL) t13 ON
> t11.ss_sold_date_sk = t13.d_date_sk) ON t9.c_customer_sk = t11.ss_customer_sk
>GROUP BY t9.c_customer_id,
> t9.c_first_name,
> t9.c_last_name,
> t9.c_preferred_cust_flag,
> t9.c_birth_country,
> t9.c_lo
