[jira] [Commented] (HIVE-18540) remove logic for wide terminal to display in-place updates
[
https://issues.apache.org/jira/browse/HIVE-18540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340660#comment-16340660
]
Song Jun commented on HIVE-18540:
-
[~anishek] Can you help to review this? Thanks
> remove logic for wide terminal to display in-place updates
> ---
>
> Key: HIVE-18540
> URL: https://issues.apache.org/jira/browse/HIVE-18540
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 2.3.2
>Reporter: Song Jun
>Assignee: Song Jun
>Priority: Minor
> Attachments: HIVE-18540.patch
>
>
> Hive Shell/beeline run Tez job, when the terminal wide is lower than 95,
> there will be no output(progress bar) to the console, because of the
> following limit logic in class InPlaceUpdate:
> {code:java}
> public static boolean canRenderInPlace(HiveConf conf) { boolean
> inPlaceUpdates = HiveConf.getBoolVar(conf,
> HiveConf.ConfVars.TEZ_EXEC_INPLACE_PROGRESS); // we need at least 80 chars
> wide terminal to display in-place updates properly return inPlaceUpdates &&
> isUnixTerminal() && TerminalFactory.get().getWidth() >= MIN_TERMINAL_WIDTH;
> }{code}
>
> {code:java}
> && isUnixTerminal() && TerminalFactory.get().getWidth() >=
> MIN_TERMINAL_WIDTH;{code}
> If someone do not know this logic ,and the terminal is not enough wide, he
> will think something wrong and do know how to fix it.
>
> and I think it is better to remove the trouble logic.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18529) Vectorization: Add a debug config option to disable scratch column reuse
[
https://issues.apache.org/jira/browse/HIVE-18529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340654#comment-16340654
]
Hive QA commented on HIVE-18529:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12907648/HIVE-18529.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 19 failed/errored test(s), 11267 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=35)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_reuse_scratchcols]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_input_format_excludes]
(batchId=162)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.org.apache.hadoop.hive.cli.TestNegativeCliDriver
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=121)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query64]
(batchId=246)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=254)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=186)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=232)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=232)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=232)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8844/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8844/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8844/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 19 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12907648 - PreCommit-HIVE-Build
> Vectorization: Add a debug config option to disable scratch column reuse
>
>
> Key: HIVE-18529
> URL: https://issues.apache.org/jira/browse/HIVE-18529
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 3.0.0, 2.3.2
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-18529-branch-2.patch, HIVE-18529.1.patch
>
>
> Debugging scratch column reuse is particularly painful and slow, adding a
> config allows for this to be done without rebuilds.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-18378) Explain plan should show if a Map/Reduce Work is being cached
[
https://issues.apache.org/jira/browse/HIVE-18378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340647#comment-16340647
]
liyunzhang edited comment on HIVE-18378 at 1/26/18 7:21 AM:
[~stakiar]: can you assign this jira to me ? I guess this feature is need by
HIVE-17486 and once HIVE-17486 is resolved, it is better to show whether the
table is cached or not in explain.
was (Author: kellyzly):
[~stakiar]: can you assign this jira to me ? I guess this feature is need by
HIVE-17486 and once HIVE-17486 is resolved, it is better to show whether the
work is cached in explain.
> Explain plan should show if a Map/Reduce Work is being cached
> -
>
> Key: HIVE-18378
> URL: https://issues.apache.org/jira/browse/HIVE-18378
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: liyunzhang
>Priority: Major
>
> It would be nice if the explain plan showed what {{MapWork}} / {{ReduceWork}}
> objects are being cached by Spark.
> The {{CombineEquivalentWorkResolver}} is the only code that triggers Spark
> cache-ing, so we should be able to modify it so that it displays if a work
> object will be cached or not.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-18378) Explain plan should show if a Map/Reduce Work is being cached
[
https://issues.apache.org/jira/browse/HIVE-18378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liyunzhang reassigned HIVE-18378:
-
Assignee: liyunzhang
> Explain plan should show if a Map/Reduce Work is being cached
> -
>
> Key: HIVE-18378
> URL: https://issues.apache.org/jira/browse/HIVE-18378
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: liyunzhang
>Priority: Major
>
> It would be nice if the explain plan showed what {{MapWork}} / {{ReduceWork}}
> objects are being cached by Spark.
> The {{CombineEquivalentWorkResolver}} is the only code that triggers Spark
> cache-ing, so we should be able to modify it so that it displays if a work
> object will be cached or not.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18378) Explain plan should show if a Map/Reduce Work is being cached
[
https://issues.apache.org/jira/browse/HIVE-18378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340647#comment-16340647
]
liyunzhang commented on HIVE-18378:
---
[~stakiar]: can you assign this jira to me ? I guess this feature is need by
HIVE-17486 and once HIVE-17486 is resolved, it is better to show whether the
work is cached in explain.
> Explain plan should show if a Map/Reduce Work is being cached
> -
>
> Key: HIVE-18378
> URL: https://issues.apache.org/jira/browse/HIVE-18378
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: liyunzhang
>Priority: Major
>
> It would be nice if the explain plan showed what {{MapWork}} / {{ReduceWork}}
> objects are being cached by Spark.
> The {{CombineEquivalentWorkResolver}} is the only code that triggers Spark
> cache-ing, so we should be able to modify it so that it displays if a work
> object will be cached or not.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18551) Vectorization: VectorMapOperator tries to write too many vector columns for Hybrid Grace
[ https://issues.apache.org/jira/browse/HIVE-18551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340633#comment-16340633 ] Teddy Choi commented on HIVE-18551: --- +1. LGTM. Tests pending. > Vectorization: VectorMapOperator tries to write too many vector columns for > Hybrid Grace > > > Key: HIVE-18551 > URL: https://issues.apache.org/jira/browse/HIVE-18551 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 3.0.0 > > Attachments: HIVE-18551.01.patch > > > Code incorrectly uses projectedColumns.length instead of singleRow.length -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18529) Vectorization: Add a debug config option to disable scratch column reuse
[
https://issues.apache.org/jira/browse/HIVE-18529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340623#comment-16340623
]
Hive QA commented on HIVE-18529:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
38s{color} | {color:red} ql: The patch generated 1 new + 354 unchanged - 0
fixed = 355 total (was 354) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
12s{color} | {color:red} The patch generated 6 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 49s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 26be99d |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8844/yetus/diff-checkstyle-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8844/yetus/patch-asflicense-problems.txt
|
| modules | C: common ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8844/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Vectorization: Add a debug config option to disable scratch column reuse
>
>
> Key: HIVE-18529
> URL: https://issues.apache.org/jira/browse/HIVE-18529
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 3.0.0, 2.3.2
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-18529-branch-2.patch, HIVE-18529.1.patch
>
>
> Debugging scratch column reuse is particularly painful and slow, adding a
> config allows for this to be done without rebuilds.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18531) Vectorization: Vectorized PTF operator should not set the initial type infos
[
https://issues.apache.org/jira/browse/HIVE-18531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340610#comment-16340610
]
Hive QA commented on HIVE-18531:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12907595/HIVE-18531.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 20 failed/errored test(s), 11661 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=35)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_input_format_excludes]
(batchId=162)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_part]
(batchId=94)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_aggregator_error_1]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=121)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query64]
(batchId=246)
org.apache.hadoop.hive.metastore.TestMetastoreExpr.testPartitionExpr
(batchId=257)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=254)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=186)
org.apache.hive.hcatalog.common.TestHiveClientCache.testCloseAllClients
(batchId=198)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=232)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=232)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=232)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8843/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8843/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8843/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 20 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12907595 - PreCommit-HIVE-Build
> Vectorization: Vectorized PTF operator should not set the initial type infos
>
>
> Key: HIVE-18531
> URL: https://issues.apache.org/jira/browse/HIVE-18531
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Matt McCline
>Priority: Critical
> Attachments: HIVE-18531.01.patch
>
>
> The Vectorized PTF operator is mistakenly setting the initial type infos for
> its output VectorizationContext. It should not. It is only creating a
> projection of the initial column names/type infos from ReduceSink (i.e. keys,
> values) plus scratch columns for output columns for which the type infos are
> already known.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18237) missing results for insert_only table after DP insert
[
https://issues.apache.org/jira/browse/HIVE-18237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Yeom updated HIVE-18237:
--
Status: Patch Available (was: Open)
This patch have been used for HIVE 18052 efforts. Now I have detached the fix
for this jira's
test case.
> missing results for insert_only table after DP insert
> -
>
> Key: HIVE-18237
> URL: https://issues.apache.org/jira/browse/HIVE-18237
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Zoltan Haindrich
>Assignee: Steve Yeom
>Priority: Major
> Attachments: HIVE-18237.01.patch, HIVE-18237.02.patch
>
>
> {code}
> set hive.stats.column.autogather=false;
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.exec.max.dynamic.partitions.pernode=200;
> set hive.exec.max.dynamic.partitions=200;
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> create table i0 (p int,v int);
> insert into i0 values
> (0,0),
> (2,2),
> (3,3);
> create table p0 (v int) partitioned by (p int) stored as orc
> tblproperties ("transactional"="true",
> "transactional_properties"="insert_only");
> explain insert overwrite table p0 partition (p) select * from i0 where v < 3;
> insert overwrite table p0 partition (p) select * from i0 where v < 3;
> select count(*) from p0 where v!=1;
> {code}
> The table p0 should contain {{2}} rows at this point; but the result is {{0}}.
> * seems to be specific to insert_only tables
> * the existing data appears if an {{insert into}} is executed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18543) Add print sessionid in console
[ https://issues.apache.org/jira/browse/HIVE-18543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tartarus updated HIVE-18543: Release Note: add print sessionid in console Status: Patch Available (was: In Progress) Print in the console sessionid help find the corresponding log information from the log file > Add print sessionid in console > -- > > Key: HIVE-18543 > URL: https://issues.apache.org/jira/browse/HIVE-18543 > Project: Hive > Issue Type: Improvement > Components: CLI, Clients >Affects Versions: 2.3.2 > Environment: CentOS6.5 > Hive-1.2.1 > Hive-2.3.2 >Reporter: tartarus >Assignee: tartarus >Priority: Major > Labels: pull-request-available > Fix For: 3.0.0 > > Original Estimate: 24h > Remaining Estimate: 24h > > Hive client log file already contains sessionid information, but the console > does not have sessionid information, the user can not be associated with the > log well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18237) missing results for insert_only table after DP insert
[
https://issues.apache.org/jira/browse/HIVE-18237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Yeom updated HIVE-18237:
--
Status: Open (was: Patch Available)
> missing results for insert_only table after DP insert
> -
>
> Key: HIVE-18237
> URL: https://issues.apache.org/jira/browse/HIVE-18237
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Zoltan Haindrich
>Assignee: Steve Yeom
>Priority: Major
> Attachments: HIVE-18237.01.patch, HIVE-18237.02.patch
>
>
> {code}
> set hive.stats.column.autogather=false;
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.exec.max.dynamic.partitions.pernode=200;
> set hive.exec.max.dynamic.partitions=200;
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> create table i0 (p int,v int);
> insert into i0 values
> (0,0),
> (2,2),
> (3,3);
> create table p0 (v int) partitioned by (p int) stored as orc
> tblproperties ("transactional"="true",
> "transactional_properties"="insert_only");
> explain insert overwrite table p0 partition (p) select * from i0 where v < 3;
> insert overwrite table p0 partition (p) select * from i0 where v < 3;
> select count(*) from p0 where v!=1;
> {code}
> The table p0 should contain {{2}} rows at this point; but the result is {{0}}.
> * seems to be specific to insert_only tables
> * the existing data appears if an {{insert into}} is executed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18237) missing results for insert_only table after DP insert
[
https://issues.apache.org/jira/browse/HIVE-18237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Yeom updated HIVE-18237:
--
Attachment: HIVE-18237.02.patch
> missing results for insert_only table after DP insert
> -
>
> Key: HIVE-18237
> URL: https://issues.apache.org/jira/browse/HIVE-18237
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Zoltan Haindrich
>Assignee: Steve Yeom
>Priority: Major
> Attachments: HIVE-18237.01.patch, HIVE-18237.02.patch
>
>
> {code}
> set hive.stats.column.autogather=false;
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.exec.max.dynamic.partitions.pernode=200;
> set hive.exec.max.dynamic.partitions=200;
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> create table i0 (p int,v int);
> insert into i0 values
> (0,0),
> (2,2),
> (3,3);
> create table p0 (v int) partitioned by (p int) stored as orc
> tblproperties ("transactional"="true",
> "transactional_properties"="insert_only");
> explain insert overwrite table p0 partition (p) select * from i0 where v < 3;
> insert overwrite table p0 partition (p) select * from i0 where v < 3;
> select count(*) from p0 where v!=1;
> {code}
> The table p0 should contain {{2}} rows at this point; but the result is {{0}}.
> * seems to be specific to insert_only tables
> * the existing data appears if an {{insert into}} is executed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work started] (HIVE-18543) Add print sessionid in console
[ https://issues.apache.org/jira/browse/HIVE-18543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-18543 started by tartarus. --- > Add print sessionid in console > -- > > Key: HIVE-18543 > URL: https://issues.apache.org/jira/browse/HIVE-18543 > Project: Hive > Issue Type: Improvement > Components: CLI, Clients >Affects Versions: 2.3.2 > Environment: CentOS6.5 > Hive-1.2.1 > Hive-2.3.2 >Reporter: tartarus >Assignee: tartarus >Priority: Major > Labels: pull-request-available > Fix For: 3.0.0 > > Original Estimate: 24h > Remaining Estimate: 24h > > Hive client log file already contains sessionid information, but the console > does not have sessionid information, the user can not be associated with the > log well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-18412) FileSinkOperator thows NullPointerException
[
https://issues.apache.org/jira/browse/HIVE-18412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
tartarus reassigned HIVE-18412:
---
Assignee: (was: tartarus)
> FileSinkOperator thows NullPointerException
>
>
> Key: HIVE-18412
> URL: https://issues.apache.org/jira/browse/HIVE-18412
> Project: Hive
> Issue Type: Bug
> Components: Hive, Transactions
> Environment: HDP2.6.1, Hive 1.2.1
>Reporter: Benjamin BONNET
>Priority: Blocker
>
> Hi,
> while executing a query (DELETE with a join) on an ACID table, I get a
> NullPointerException in reducer.
> See stack trace below.
> According to FileSinkOperator source code, it seems that buckepMap transient
> field is Null.
> In my opinion, the only circumstance in which this field may be null is when
> the involved FileSinkOperator has been serialized and then deserialized.
> Actually, deserialization lets that transient reference uninitialized.
> I checked source code for more recent versions (including Hive 2.x) but
> everywhere that field may remain uninitialized (if FileSinkOperator is
> serialized/deserialized). So I think that issue may concern any version of
> Hive.
> {noformat}
> ERROR : Vertex failed, vertexName=Reducer 3,
> vertexId=vertex_1513704146031_77754_2_05, diagnostics=[Task failed,
> taskId=task_1513704146031_77754_2_05_00, diagnostics=[TaskAttempt 0
> failed, info=[Error: Failure while running task:java
> .lang.RuntimeException: java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row (tag=0)
> {"key":{"reducesinkkey0":{"transactionid":108117,"bucketid":0,"rowid":1114}},"value":{"
> _col0":"2017","_col1":"10"}}
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:347)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:194)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:185)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:185)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:181)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row (tag=0)
> {"key":{"reducesinkkey0":{"transactionid":108117,"bucketid":0,"rowid":1114}},"value":{"_col0":"2017"
> ,"_col1":"10"}}
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:284)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:266)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:150)
> ... 14 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row (tag=0)
> {"key":{"reducesinkkey0":{"transactionid":108117,"bucketid":0,"rowid":1114}},"value":{"_col0":"2017","_col1":"10"}}
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:352)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:274)
> ... 16 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.findWriterOffset(FileSinkOperator.java:830)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:758)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:841)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:88)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:343)
> ... 17 more
> ], TaskAttempt 1 failed, info=[Error: Failure while
[jira] [Assigned] (HIVE-18412) FileSinkOperator thows NullPointerException
[
https://issues.apache.org/jira/browse/HIVE-18412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
tartarus reassigned HIVE-18412:
---
Assignee: tartarus
> FileSinkOperator thows NullPointerException
>
>
> Key: HIVE-18412
> URL: https://issues.apache.org/jira/browse/HIVE-18412
> Project: Hive
> Issue Type: Bug
> Components: Hive, Transactions
> Environment: HDP2.6.1, Hive 1.2.1
>Reporter: Benjamin BONNET
>Assignee: tartarus
>Priority: Blocker
>
> Hi,
> while executing a query (DELETE with a join) on an ACID table, I get a
> NullPointerException in reducer.
> See stack trace below.
> According to FileSinkOperator source code, it seems that buckepMap transient
> field is Null.
> In my opinion, the only circumstance in which this field may be null is when
> the involved FileSinkOperator has been serialized and then deserialized.
> Actually, deserialization lets that transient reference uninitialized.
> I checked source code for more recent versions (including Hive 2.x) but
> everywhere that field may remain uninitialized (if FileSinkOperator is
> serialized/deserialized). So I think that issue may concern any version of
> Hive.
> {noformat}
> ERROR : Vertex failed, vertexName=Reducer 3,
> vertexId=vertex_1513704146031_77754_2_05, diagnostics=[Task failed,
> taskId=task_1513704146031_77754_2_05_00, diagnostics=[TaskAttempt 0
> failed, info=[Error: Failure while running task:java
> .lang.RuntimeException: java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row (tag=0)
> {"key":{"reducesinkkey0":{"transactionid":108117,"bucketid":0,"rowid":1114}},"value":{"
> _col0":"2017","_col1":"10"}}
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:347)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:194)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:185)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:185)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:181)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row (tag=0)
> {"key":{"reducesinkkey0":{"transactionid":108117,"bucketid":0,"rowid":1114}},"value":{"_col0":"2017"
> ,"_col1":"10"}}
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:284)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:266)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:150)
> ... 14 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row (tag=0)
> {"key":{"reducesinkkey0":{"transactionid":108117,"bucketid":0,"rowid":1114}},"value":{"_col0":"2017","_col1":"10"}}
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:352)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:274)
> ... 16 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.findWriterOffset(FileSinkOperator.java:830)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:758)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:841)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:88)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:343)
> ... 17 more
> ], TaskAttempt 1 failed, info=[
[jira] [Commented] (HIVE-18531) Vectorization: Vectorized PTF operator should not set the initial type infos
[
https://issues.apache.org/jira/browse/HIVE-18531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340590#comment-16340590
]
Hive QA commented on HIVE-18531:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 6 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 26s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 26be99d |
| Default Java | 1.8.0_111 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8843/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8843/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Vectorization: Vectorized PTF operator should not set the initial type infos
>
>
> Key: HIVE-18531
> URL: https://issues.apache.org/jira/browse/HIVE-18531
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Matt McCline
>Priority: Critical
> Attachments: HIVE-18531.01.patch
>
>
> The Vectorized PTF operator is mistakenly setting the initial type infos for
> its output VectorizationContext. It should not. It is only creating a
> projection of the initial column names/type infos from ReduceSink (i.e. keys,
> values) plus scratch columns for output columns for which the type infos are
> already known.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18530) Replication should skip MM table (for now)
[
https://issues.apache.org/jira/browse/HIVE-18530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340578#comment-16340578
]
Hive QA commented on HIVE-18530:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12907590/HIVE-18530.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 18 failed/errored test(s), 11661 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=35)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_input_format_excludes]
(batchId=162)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_part]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=121)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=254)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=186)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=232)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=232)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=232)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8842/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8842/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8842/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 18 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12907590 - PreCommit-HIVE-Build
> Replication should skip MM table (for now)
> --
>
> Key: HIVE-18530
> URL: https://issues.apache.org/jira/browse/HIVE-18530
> Project: Hive
> Issue Type: Bug
> Components: repl
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-18530.1.patch
>
>
> Currently replication cannot handle transactional table (including MM table)
> until HIVE-18320. HIVE-17504 skips table with transactional=true explicitly.
> HIVE-18352 changes the logic to use AcidUtils.isAcidTable for the same
> purpose. However, isAcidTable returns false for mm table, thus Hive still
> dump mm table during replication.
> Here is an error message during dump mm table:
> {code}
> ERROR : FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.repl.ReplDumpTask.
> java.io.FileNotFoundException: Path is not a file:
> /apps/hive/warehouse/testrepldb5.db/test1/delta_261_261_
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:89)
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:75)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:154)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1920)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:731)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:424)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
> at org.ap
[jira] [Updated] (HIVE-18478) Drop of temp table creating recycle files at CM path
[ https://issues.apache.org/jira/browse/HIVE-18478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-18478: --- Attachment: HIVE-18478.01.patch > Drop of temp table creating recycle files at CM path > > > Key: HIVE-18478 > URL: https://issues.apache.org/jira/browse/HIVE-18478 > Project: Hive > Issue Type: Sub-task > Components: Hive, HiveServer2 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Minor > Labels: pull-request-available > Fix For: 3.0.0 > > Attachments: HIVE-18478.01.patch > > > Drop TEMP table operation invokes deleteDir which moves the file to $CMROOT > which is not needed as temp tables need not be replicated -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18530) Replication should skip MM table (for now)
[
https://issues.apache.org/jira/browse/HIVE-18530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340547#comment-16340547
]
Hive QA commented on HIVE-18530:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
12s{color} | {color:red} The patch generated 6 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 30s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 26be99d |
| Default Java | 1.8.0_111 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8842/yetus/patch-asflicense-problems.txt
|
| modules | C: ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8842/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Replication should skip MM table (for now)
> --
>
> Key: HIVE-18530
> URL: https://issues.apache.org/jira/browse/HIVE-18530
> Project: Hive
> Issue Type: Bug
> Components: repl
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-18530.1.patch
>
>
> Currently replication cannot handle transactional table (including MM table)
> until HIVE-18320. HIVE-17504 skips table with transactional=true explicitly.
> HIVE-18352 changes the logic to use AcidUtils.isAcidTable for the same
> purpose. However, isAcidTable returns false for mm table, thus Hive still
> dump mm table during replication.
> Here is an error message during dump mm table:
> {code}
> ERROR : FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.repl.ReplDumpTask.
> java.io.FileNotFoundException: Path is not a file:
> /apps/hive/warehouse/testrepldb5.db/test1/delta_261_261_
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:89)
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:75)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:154)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1920)
> at
> org.apache.had
[jira] [Commented] (HIVE-18528) Stats: In the bitvector codepath, when extrapolating column stats for String type columnStringColumnStatsAggregator uses the min value instead of max
[
https://issues.apache.org/jira/browse/HIVE-18528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340523#comment-16340523
]
Hive QA commented on HIVE-18528:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12907582/HIVE-18528.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 23 failed/errored test(s), 11660 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_outer_join4]
(batchId=85)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_input_format_excludes]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[bucketizedhiveinputformat]
(batchId=178)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_part]
(batchId=94)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[mismatch_columns_insertion]
(batchId=94)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[repl_load_requires_admin]
(batchId=94)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_aggregator_error_1]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=121)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=254)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=186)
org.apache.hive.hcatalog.pig.TestHCatLoaderComplexSchema.testMapNullKey[2]
(batchId=191)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=232)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=232)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=232)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8841/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8841/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8841/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 23 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12907582 - PreCommit-HIVE-Build
> Stats: In the bitvector codepath, when extrapolating column stats for String
> type columnStringColumnStatsAggregator uses the min value instead of max
> -
>
> Key: HIVE-18528
> URL: https://issues.apache.org/jira/browse/HIVE-18528
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
>Priority: Major
> Attachments: HIVE-18528.1.patch
>
>
> This line:
> [https://github.com/apache/hive/blob/456a65180dcb84f69f26b4c9b9265165ad16dfe4/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/columnstats/aggr/StringColumnStatsAggregator.java#L181]
> Should be:
> aggregateData.setAvgColLen(Math.max(aggregateData.getAvgColLen(),
> newData.getAvgColLen()));
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18514) add service output for ranger to WM DDL operations
[ https://issues.apache.org/jira/browse/HIVE-18514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18514: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Committed to master. Thanks for the review! The failed auth test passes locally and seems to fail often on HiveQA > add service output for ranger to WM DDL operations > -- > > Key: HIVE-18514 > URL: https://issues.apache.org/jira/browse/HIVE-18514 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18514.01.patch, HIVE-18514.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18528) Stats: In the bitvector codepath, when extrapolating column stats for String type columnStringColumnStatsAggregator uses the min value instead of max
[
https://issues.apache.org/jira/browse/HIVE-18528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340491#comment-16340491
]
Hive QA commented on HIVE-18528:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
12s{color} | {color:red} The patch generated 6 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 11m 13s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 948684a |
| Default Java | 1.8.0_111 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8841/yetus/patch-asflicense-problems.txt
|
| modules | C: standalone-metastore U: standalone-metastore |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8841/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Stats: In the bitvector codepath, when extrapolating column stats for String
> type columnStringColumnStatsAggregator uses the min value instead of max
> -
>
> Key: HIVE-18528
> URL: https://issues.apache.org/jira/browse/HIVE-18528
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
>Priority: Major
> Attachments: HIVE-18528.1.patch
>
>
> This line:
> [https://github.com/apache/hive/blob/456a65180dcb84f69f26b4c9b9265165ad16dfe4/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/columnstats/aggr/StringColumnStatsAggregator.java#L181]
> Should be:
> aggregateData.setAvgColLen(Math.max(aggregateData.getAvgColLen(),
> newData.getAvgColLen()));
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18514) add service output for ranger to WM DDL operations
[
https://issues.apache.org/jira/browse/HIVE-18514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340479#comment-16340479
]
Hive QA commented on HIVE-18514:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12907576/HIVE-18514.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 20 failed/errored test(s), 11661 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=35)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_input_format_excludes]
(batchId=162)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_part]
(batchId=94)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_aggregator_error_1]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=121)
org.apache.hadoop.hive.metastore.TestHiveMetaTool.testListFSRoot (batchId=223)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=254)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=186)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=232)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=232)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=232)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8840/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8840/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8840/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 20 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12907576 - PreCommit-HIVE-Build
> add service output for ranger to WM DDL operations
> --
>
> Key: HIVE-18514
> URL: https://issues.apache.org/jira/browse/HIVE-18514
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-18514.01.patch, HIVE-18514.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently
[
https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340466#comment-16340466
]
Alexander Kolbasov commented on HIVE-16886:
---
[~anishek] [~thejas] What do you think about implementing SELECT FOR UPDATE
using the mechanism from HIVE-18526? It is much simpler and uses documented
DataNucleus semantics.
> HMS log notifications may have duplicated event IDs if multiple HMS are
> running concurrently
>
>
> Key: HIVE-16886
> URL: https://issues.apache.org/jira/browse/HIVE-16886
> Project: Hive
> Issue Type: Bug
> Components: Hive, Metastore
>Affects Versions: 3.0.0, 2.3.2, 2.3.3
>Reporter: Sergio Peña
>Assignee: anishek
>Priority: Major
> Labels: TODOC3.0
> Fix For: 3.0.0
>
> Attachments: HIVE-16886.1.patch, HIVE-16886.2.patch,
> HIVE-16886.3.patch, HIVE-16886.4.patch, HIVE-16886.5.patch,
> HIVE-16886.6.patch, HIVE-16886.7.patch, HIVE-16886.8.patch,
> datastore-identity-holes.diff
>
>
> When running multiple Hive Metastore servers and DB notifications are
> enabled, I could see that notifications can be persisted with a duplicated
> event ID.
> This does not happen when running multiple threads in a single HMS node due
> to the locking acquired on the DbNotificationsLog class, but multiple HMS
> could cause conflicts.
> The issue is in the ObjectStore#addNotificationEvent() method. The event ID
> fetched from the datastore is used for the new notification, incremented in
> the server itself, then persisted or updated back to the datastore. If 2
> servers read the same ID, then these 2 servers write a new notification with
> the same ID.
> The event ID is not unique nor a primary key.
> Here's a test case using the TestObjectStore class that confirms this issue:
> {noformat}
> @Test
> public void testConcurrentAddNotifications() throws ExecutionException,
> InterruptedException {
> final int NUM_THREADS = 2;
> CountDownLatch countIn = new CountDownLatch(NUM_THREADS);
> CountDownLatch countOut = new CountDownLatch(1);
> HiveConf conf = new HiveConf();
> conf.setVar(HiveConf.ConfVars.METASTORE_EXPRESSION_PROXY_CLASS,
> MockPartitionExpressionProxy.class.getName());
> ExecutorService executorService =
> Executors.newFixedThreadPool(NUM_THREADS);
> FutureTask tasks[] = new FutureTask[NUM_THREADS];
> for (int i=0; i final int n = i;
> tasks[i] = new FutureTask(new Callable() {
> @Override
> public Void call() throws Exception {
> ObjectStore store = new ObjectStore();
> store.setConf(conf);
> NotificationEvent dbEvent =
> new NotificationEvent(0, 0,
> EventMessage.EventType.CREATE_DATABASE.toString(), "CREATE DATABASE DB" + n);
> System.out.println("ADDING NOTIFICATION");
> countIn.countDown();
> countOut.await();
> store.addNotificationEvent(dbEvent);
> System.out.println("FINISH NOTIFICATION");
> return null;
> }
> });
> executorService.execute(tasks[i]);
> }
> countIn.await();
> countOut.countDown();
> for (int i = 0; i < NUM_THREADS; ++i) {
> tasks[i].get();
> }
> NotificationEventResponse eventResponse =
> objectStore.getNextNotification(new NotificationEventRequest());
> Assert.assertEquals(2, eventResponse.getEventsSize());
> Assert.assertEquals(1, eventResponse.getEvents().get(0).getEventId());
> // This fails because the next notification has an event ID = 1
> Assert.assertEquals(2, eventResponse.getEvents().get(1).getEventId());
> }
> {noformat}
> The last assertion fails expecting an event ID 1 instead of 2.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18552) Split hive.strict.checks.large.query into two configs, enable partition filter check by default
[
https://issues.apache.org/jira/browse/HIVE-18552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340441#comment-16340441
]
Sergey Shelukhin commented on HIVE-18552:
-
It makes sense to separate them, but requiring a partitioned filter should not
be on by default because it fails many legitimate queries.
IIRC there's a separate setting that controls the max # of partitions to allow
in a scan
> Split hive.strict.checks.large.query into two configs, enable partition
> filter check by default
> ---
>
> Key: HIVE-18552
> URL: https://issues.apache.org/jira/browse/HIVE-18552
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
>
> {{hive.strict.checks.large.query}} controls the strict checks for restricting
> order bys with no limits, and scans of a partitioned table without a filter
> on the partition table.
> While both checks prevent "large" queries from being run, they both control
> very different behavior. It would be better if users could control these
> restrictions separately.
> Furthermore, many users make the mistake of abusing partitioned tables and
> often end up in a situation where they are running queries that are doing
> full-table scans of partitioned tables. This can lead to lots of issues for
> Hive - e.g. OOM issues because so many partitions are loaded in memory. So it
> would be good if we enabled this restriction by default.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18553) VectorizedParquetReader fails after adding a new column to table
[
https://issues.apache.org/jira/browse/HIVE-18553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340435#comment-16340435
]
Vihang Karajgaonkar commented on HIVE-18553:
cc: [~Ferd]
> VectorizedParquetReader fails after adding a new column to table
>
>
> Key: HIVE-18553
> URL: https://issues.apache.org/jira/browse/HIVE-18553
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 3.0.0, 2.4.0, 2.3.2
>Reporter: Vihang Karajgaonkar
>Priority: Major
>
> VectorizedParquetReader throws an exception when trying to reading from a
> parquet table on which new columns are added. Steps to reproduce below:
> {code}
> 0: jdbc:hive2://localhost:1/default> desc test_p;
> +---++--+
> | col_name | data_type | comment |
> +---++--+
> | t1| tinyint| |
> | t2| tinyint| |
> | i1| int| |
> | i2| int| |
> +---++--+
> 0: jdbc:hive2://localhost:1/default> set hive.fetch.task.conversion=none;
> 0: jdbc:hive2://localhost:1/default> set
> hive.vectorized.execution.enabled=true;
> 0: jdbc:hive2://localhost:1/default> alter table test_p add columns (ts
> timestamp);
> 0: jdbc:hive2://localhost:1/default> select * from test_p;
> Error: Error while processing statement: FAILED: Execution Error, return code
> 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask (state=08S01,code=2)
> {code}
> Following exception is seen in the logs
> {code}
> Caused by: java.lang.IllegalArgumentException: [ts] BINARY is not in the
> store: [[i1] INT32, [i2] INT32, [t1] INT32, [t2] INT32] 3
> at
> org.apache.parquet.hadoop.ColumnChunkPageReadStore.getPageReader(ColumnChunkPageReadStore.java:160)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.io.parquet.vector.VectorizedParquetRecordReader.buildVectorizedParquetReader(VectorizedParquetRecordReader.java:479)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.io.parquet.vector.VectorizedParquetRecordReader.checkEndOfRowGroup(VectorizedParquetRecordReader.java:432)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.io.parquet.vector.VectorizedParquetRecordReader.nextBatch(VectorizedParquetRecordReader.java:393)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.io.parquet.vector.VectorizedParquetRecordReader.next(VectorizedParquetRecordReader.java:345)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.io.parquet.vector.VectorizedParquetRecordReader.next(VectorizedParquetRecordReader.java:88)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:360)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.doNext(CombineHiveRecordReader.java:167)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.doNext(CombineHiveRecordReader.java:52)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.next(HiveContextAwareRecordReader.java:116)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.doNextWithExceptionHandler(HadoopShimsSecure.java:229)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.next(HadoopShimsSecure.java:142)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.java:199)
> ~[hadoop-mapreduce-client-core-3.0.0-alpha3-cdh6.x-SNAPSHOT.jar:?]
> at
> org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:185)
> ~[hadoop-mapreduce-client-core-3.0.0-alpha3-cdh6.x-SNAPSHOT.jar:?]
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:52)
> ~[hadoop-mapreduce-client-core-3.0.0-alpha3-cdh6.x-SNAPSHOT.jar:?]
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
> ~[hadoop-mapreduce-client-core-3.0.0-alpha3-cdh6.x-SNAPSHOT.jar:?]
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> ~[hadoop-mapreduce-client-core-3.0.0-alpha3-cdh6.x-SNAPSHOT.jar:?]
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:271)
> ~[hadoop-mapreduce-client-common-3.0.0-alpha3-cdh6.x-SNAPSHOT.jar:?]
[jira] [Commented] (HIVE-18552) Split hive.strict.checks.large.query into two configs, enable partition filter check by default
[
https://issues.apache.org/jira/browse/HIVE-18552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340428#comment-16340428
]
Sahil Takiar commented on HIVE-18552:
-
[~sershe], [~ashutoshc] any thoughts on this?
> Split hive.strict.checks.large.query into two configs, enable partition
> filter check by default
> ---
>
> Key: HIVE-18552
> URL: https://issues.apache.org/jira/browse/HIVE-18552
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
>
> {{hive.strict.checks.large.query}} controls the strict checks for restricting
> order bys with no limits, and scans of a partitioned table without a filter
> on the partition table.
> While both checks prevent "large" queries from being run, they both control
> very different behavior. It would be better if users could control these
> restrictions separately.
> Furthermore, many users make the mistake of abusing partitioned tables and
> often end up in a situation where they are running queries that are doing
> full-table scans of partitioned tables. This can lead to lots of issues for
> Hive - e.g. OOM issues because so many partitions are loaded in memory. So it
> would be good if we enabled this restriction by default.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-17848) Bucket Map Join : Implement an efficient way to minimize loading hash table
[ https://issues.apache.org/jira/browse/HIVE-17848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepak Jaiswal updated HIVE-17848: -- Attachment: HIVE-17848.7.patch > Bucket Map Join : Implement an efficient way to minimize loading hash table > --- > > Key: HIVE-17848 > URL: https://issues.apache.org/jira/browse/HIVE-17848 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-17848.2.patch, HIVE-17848.4.patch, > HIVE-17848.5.patch, HIVE-17848.6.patch, HIVE-17848.7.patch > > > In bucket mapjoin, each task loads its own copy of hash table which is > inefficient as load is IO heavy and due to multiple copies of same hash > table, the tables may get GCed on a busy system. > Implement a subcache with softreference to each hash table corresponding to > its bucketID such that it can be reused by a task. > This needs changes from Tez side to push bucket id to TezProcessor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-18552) Split hive.strict.checks.large.query into two configs, enable partition filter check by default
[
https://issues.apache.org/jira/browse/HIVE-18552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar reassigned HIVE-18552:
---
Assignee: Sahil Takiar
> Split hive.strict.checks.large.query into two configs, enable partition
> filter check by default
> ---
>
> Key: HIVE-18552
> URL: https://issues.apache.org/jira/browse/HIVE-18552
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
>
> {{hive.strict.checks.large.query}} controls the strict checks for restricting
> order bys with no limits, and scans of a partitioned table without a filter
> on the partition table.
> While both checks prevent "large" queries from being run, they both control
> very different behavior. It would be better if users could control these
> restrictions separately.
> Furthermore, many users make the mistake of abusing partitioned tables and
> often end up in a situation where they are running queries that are doing
> full-table scans of partitioned tables. This can lead to lots of issues for
> Hive - e.g. OOM issues because so many partitions are loaded in memory. So it
> would be good if we enabled this restriction by default.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18514) add service output for ranger to WM DDL operations
[
https://issues.apache.org/jira/browse/HIVE-18514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340414#comment-16340414
]
Hive QA commented on HIVE-18514:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} ql: The patch generated 0 new + 244 unchanged - 1
fixed = 244 total (was 245) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
13s{color} | {color:red} The patch generated 6 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 7s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 948684a |
| Default Java | 1.8.0_111 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8840/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8840/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> add service output for ranger to WM DDL operations
> --
>
> Key: HIVE-18514
> URL: https://issues.apache.org/jira/browse/HIVE-18514
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-18514.01.patch, HIVE-18514.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18551) Vectorization: VectorMapOperator tries to write too many vector columns for Hybrid Grace
[ https://issues.apache.org/jira/browse/HIVE-18551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340408#comment-16340408 ] Matt McCline commented on HIVE-18551: - Change confirmed with a large query that triggered the issue. No small test case. > Vectorization: VectorMapOperator tries to write too many vector columns for > Hybrid Grace > > > Key: HIVE-18551 > URL: https://issues.apache.org/jira/browse/HIVE-18551 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 3.0.0 > > Attachments: HIVE-18551.01.patch > > > Code incorrectly uses projectedColumns.length instead of singleRow.length -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18531) Vectorization: Vectorized PTF operator should not set the initial type infos
[ https://issues.apache.org/jira/browse/HIVE-18531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340409#comment-16340409 ] Matt McCline commented on HIVE-18531: - Change confirmed with a large query that triggered the issue. No small test case. > Vectorization: Vectorized PTF operator should not set the initial type infos > > > Key: HIVE-18531 > URL: https://issues.apache.org/jira/browse/HIVE-18531 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Priority: Critical > Attachments: HIVE-18531.01.patch > > > The Vectorized PTF operator is mistakenly setting the initial type infos for > its output VectorizationContext. It should not. It is only creating a > projection of the initial column names/type infos from ReduceSink (i.e. keys, > values) plus scratch columns for output columns for which the type infos are > already known. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18551) Vectorization: VectorMapOperator tries to write too many vector columns for Hybrid Grace
[ https://issues.apache.org/jira/browse/HIVE-18551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18551: Status: Patch Available (was: Open) > Vectorization: VectorMapOperator tries to write too many vector columns for > Hybrid Grace > > > Key: HIVE-18551 > URL: https://issues.apache.org/jira/browse/HIVE-18551 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 3.0.0 > > Attachments: HIVE-18551.01.patch > > > Code incorrectly uses projectedColumns.length instead of singleRow.length -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18551) Vectorization: VectorMapOperator tries to write too many vector columns for Hybrid Grace
[ https://issues.apache.org/jira/browse/HIVE-18551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18551: Attachment: HIVE-18551.01.patch > Vectorization: VectorMapOperator tries to write too many vector columns for > Hybrid Grace > > > Key: HIVE-18551 > URL: https://issues.apache.org/jira/browse/HIVE-18551 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 3.0.0 > > Attachments: HIVE-18551.01.patch > > > Code incorrectly uses projectedColumns.length instead of singleRow.length -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-18551) Vectorization: VectorMapOperator tries to write too many vector columns for Hybrid Grace
[ https://issues.apache.org/jira/browse/HIVE-18551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline reassigned HIVE-18551: --- > Vectorization: VectorMapOperator tries to write too many vector columns for > Hybrid Grace > > > Key: HIVE-18551 > URL: https://issues.apache.org/jira/browse/HIVE-18551 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 3.0.0 > > > Code incorrectly uses projectedColumns.length instead of singleRow.length -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17751) Separate HMS Client and HMS server into separate sub-modules
[
https://issues.apache.org/jira/browse/HIVE-17751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340387#comment-16340387
]
Hive QA commented on HIVE-17751:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12907567/HIVE-17751.06-standalone-metastore.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 17 failed/errored test(s), 11222 tests
executed
*Failed tests:*
{noformat}
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=94)
[udf_invalid.q,authorization_uri_export.q,druid_datasource2.q,view_update.q,default_partition_name.q,authorization_public_create.q,load_wrong_fileformat_rc_seq.q,altern1.q,describe_xpath1.q,drop_view_failure2.q,orc_replace_columns2_acid.q,temp_table_rename.q,invalid_select_column_with_subquery.q,udf_trunc_error1.q,insert_view_failure.q,dbtxnmgr_nodbunlock.q,authorization_show_columns.q,cte_recursion.q,load_part_nospec.q,clusterbyorderby.q,orc_type_promotion2.q,ctas_noperm_loc.q,duplicate_alias_in_transform.q,invalid_create_tbl2.q,part_col_complex_type.q,authorization_drop_db_empty.q,smb_mapjoin_14.q,subquery_scalar_multi_rows.q,alter_partition_coltype_2columns.q,subquery_corr_in_agg.q,authorization_show_grant_otheruser_wtab.q,regex_col_groupby.q,udaf_collect_set_unsupported.q,ptf_negative_DuplicateWindowAlias.q,exim_22_export_authfail.q,udf_likeany_wrong1.q,groupby_key.q,ambiguous_col.q,groupby3_multi_distinct.q,authorization_alter_drop_ptn.q,invalid_cast_from_binary_5.q,show_create_table_does_not_exist.q,exim_20_managed_location_over_existing.q,interval_3.q,authorization_compile.q,join35.q,merge_negative_3.q,udf_concat_ws_wrong3.q,create_or_replace_view8.q,split_sample_out_of_range.q,materialized_view_no_transactional_rewrite.q,alter_concatenate_indexed_table.q,authorization_show_grant_otherrole.q,create_with_constraints_duplicate_name.q,invalid_stddev_samp_syntax.q,authorization_view_disable_cbo_7.q,autolocal1.q,analyze_view.q,exim_14_nonpart_part.q,avro_non_nullable_union.q,load_orc_negative_part.q,drop_view_failure1.q,columnstats_partlvl_invalid_values_autogather.q,exim_13_nonnative_import.q,alter_table_wrong_regex.q,add_partition_with_whitelist.q,udf_next_day_error_2.q,authorization_select.q,udf_trunc_error2.q,authorization_view_7.q,udf_format_number_wrong5.q,touch2.q,exim_03_nonpart_noncompat_colschema.q,orc_type_promotion1.q,lateral_view_alias.q,show_tables_bad_db1.q,unset_table_property.q,alter_non_native.q,nvl_mismatch_type.q,load_orc_negative3.q,authorization_create_role_no_admin.q,invalid_distinct1.q,authorization_grant_server.q,orc_type_promotion3_acid.q,show_tables_bad1.q,macro_unused_parameter.q,drop_invalid_constraint3.q,char_pad_convert_fail3.q,exim_23_import_exist_authfail.q,drop_invalid_constraint4.q,archive1.q,subquery_multiple_cols_in_select.q,drop_index_failure.q,change_hive_hdfs_session_path.q,udf_trunc_error3.q,invalid_variance_syntax.q,authorization_truncate_2.q,invalid_avg_syntax.q,invalid_select_column_with_tablename.q,mm_truncate_cols.q,groupby_grouping_sets1.q,druid_location.q,groupby2_multi_distinct.q,authorization_sba_drop_table.q,dynamic_partitions_with_whitelist.q,delete_non_acid_table.q,udf_greatest_error_2.q,create_with_constraints_validate.q,authorization_view_6.q,show_tablestatus.q,describe_xpath3.q,duplicate_alias_in_transform_schema.q,create_with_fk_uk_same_tab.q,authorization_create_tbl.q,udtf_not_supported3.q,alter_table_constraint_invalid_fk_col2.q,udtf_not_supported1.q,dbtxnmgr_notableunlock.q,ptf_negative_InvalidValueBoundary.q,alter_table_constraint_duplicate_pk.q,udf_size_wrong_type.q,exim_04_nonpart_noncompat_colnumber.q,udf_printf_wrong4.q,create_view_failure9.q,udf_elt_wrong_type.q,selectDistinctStarNeg_1.q,invalid_mapjoin1.q,load_stored_as_dirs.q,input1.q,udf_sort_array_wrong1.q,invalid_distinct2.q,dyn_part4.q,invalid_select_fn.q,authorization_role_grant_otherrole.q,archive4.q,load_nonpart_authfail.q,recursive_view.q,authorization_view_disable_cbo_1.q,create_unknown_genericudf.q,desc_failure4.q,create_not_acid.q,udf_sort_array_wrong3.q,udf_map_values_arg_type.q,alter_partition_change_col_nonexist.q,create_with_constraints_enforced.q,update_non_acid_table.q,authorization_view_disable_cbo_5.q,ct_noperm_loc.q,interval_1.q,authorization_uri_index.q,authorization_show_grant_otheruser_all.q,authorization_view_2.q,show_tables_bad2.q,groupby_rollup2.q,truncate_column_seqfile.q,create_view_failure5.q,authorization_create_view.q,ctasnullcol.q,create_or_replace_view1.q,udf_max.q,exim_01_nonpart_over_loaded.q,msck_repair_1.q,orc_change_fileformat_acid.q,udf_nonexistent_resource.q,exim_19_external_over_existing.q,serde_regex2.q,msck_repair_2.q,exim_06_nonpart_noncompat_storage.q,illegal_partition_type4.q,udf_sort_array_by_wrong1.q,create_or_replace_view5.q,windowing_leadlag_in_udaf.q,authorization_drop
[jira] [Commented] (HIVE-8472) Add ALTER DATABASE SET LOCATION
[ https://issues.apache.org/jira/browse/HIVE-8472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340357#comment-16340357 ] Lefty Leverenz commented on HIVE-8472: -- The documentation is here (thanks [~mithun]): * [DDL -- Alter Database | https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-AlterDatabase] By the way, Fix Version/s should include 2.2.1. > Add ALTER DATABASE SET LOCATION > --- > > Key: HIVE-8472 > URL: https://issues.apache.org/jira/browse/HIVE-8472 > Project: Hive > Issue Type: Improvement > Components: Database/Schema >Affects Versions: 2.2.0, 3.0.0, 2.4.0 >Reporter: Jeremy Beard >Assignee: Mithun Radhakrishnan >Priority: Major > Fix For: 3.0.0, 2.4.0 > > Attachments: HIVE-8472.1-branch-2.patch, HIVE-8472.1.patch, > HIVE-8472.2-branch-2.patch, HIVE-8472.3.patch > > > Similarly to ALTER TABLE tablename SET LOCATION, it would be helpful if there > was an equivalent for databases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-11759) Extend new cost model to correctly reflect limit cost
[ https://issues.apache.org/jira/browse/HIVE-11759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-11759: --- Status: Open (was: Patch Available) > Extend new cost model to correctly reflect limit cost > - > > Key: HIVE-11759 > URL: https://issues.apache.org/jira/browse/HIVE-11759 > Project: Hive > Issue Type: Improvement > Components: CBO >Affects Versions: 2.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-11759.01.patch, HIVE-11759.02.patch, > HIVE-11759.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-11759) Extend new cost model to correctly reflect limit cost
[ https://issues.apache.org/jira/browse/HIVE-11759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez reassigned HIVE-11759: -- Assignee: (was: Jesus Camacho Rodriguez) > Extend new cost model to correctly reflect limit cost > - > > Key: HIVE-11759 > URL: https://issues.apache.org/jira/browse/HIVE-11759 > Project: Hive > Issue Type: Improvement > Components: CBO >Affects Versions: 2.0.0 >Reporter: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-11759.01.patch, HIVE-11759.02.patch, > HIVE-11759.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work stopped] (HIVE-11784) Extend new cost model to reflect HDFS read/write cost when a new execution phase is created
[ https://issues.apache.org/jira/browse/HIVE-11784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-11784 stopped by Jesus Camacho Rodriguez. -- > Extend new cost model to reflect HDFS read/write cost when a new execution > phase is created > --- > > Key: HIVE-11784 > URL: https://issues.apache.org/jira/browse/HIVE-11784 > Project: Hive > Issue Type: Bug > Components: CBO >Affects Versions: 2.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-11784.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-11784) Extend new cost model to reflect HDFS read/write cost when a new execution phase is created
[ https://issues.apache.org/jira/browse/HIVE-11784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez reassigned HIVE-11784: -- Assignee: (was: Jesus Camacho Rodriguez) > Extend new cost model to reflect HDFS read/write cost when a new execution > phase is created > --- > > Key: HIVE-11784 > URL: https://issues.apache.org/jira/browse/HIVE-11784 > Project: Hive > Issue Type: Bug > Components: CBO >Affects Versions: 2.0.0 >Reporter: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-11784.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-16950) Dropping hive database/table which was created explicitly in default database location, deletes all databases data from default database location
[
https://issues.apache.org/jira/browse/HIVE-16950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340314#comment-16340314
]
Hive QA commented on HIVE-16950:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12907564/HIVE-16950.4.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8838/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8838/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8838/
Messages:
{noformat}
This message was trimmed, see log for full details
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar(javax/servlet/Filter.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar(javax/servlet/FilterChain.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar(javax/servlet/FilterConfig.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar(javax/servlet/ServletException.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar(javax/servlet/ServletRequest.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar(javax/servlet/ServletResponse.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar(javax/servlet/annotation/WebFilter.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar(javax/servlet/http/HttpServletRequest.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar(javax/servlet/http/HttpServletResponse.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/classification/target/hive-classification-3.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/common/classification/InterfaceAudience$LimitedPrivate.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/classification/target/hive-classification-3.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/common/classification/InterfaceStability$Unstable.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/io/ByteArrayOutputStream.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/io/OutputStream.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/io/Closeable.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/lang/AutoCloseable.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/io/Flushable.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(javax/xml/bind/annotation/XmlRootElement.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/commons/commons-exec/1.1/commons-exec-1.1.jar(org/apache/commons/exec/ExecuteException.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/security/PrivilegedExceptionAction.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/util/concurrent/ExecutionException.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/util/concurrent/TimeoutException.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-common/3.0.0-beta1/hadoop-common-3.0.0-beta1.jar(org/apache/hadoop/fs/FileSystem.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/shims/common/target/hive-shims-common-3.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/shims/HadoopShimsSecure.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/shims/common/target/hive-shims-common-3.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/shims/ShimLoader.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/shims/common/target/hive-shims-common-3.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/shims/HadoopShims.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/shims/common/target/hive-shims-common-3.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/shims/HadoopShims$WebHCatJTShim.class)]]
[loading
ZipFileIndexFileObject[/data/hivept
[jira] [Assigned] (HIVE-18550) Keep the hbase table name property as hbase.table.name
[ https://issues.apache.org/jira/browse/HIVE-18550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu reassigned HIVE-18550: --- > Keep the hbase table name property as hbase.table.name > -- > > Key: HIVE-18550 > URL: https://issues.apache.org/jira/browse/HIVE-18550 > Project: Hive > Issue Type: Sub-task > Components: HBase Handler >Affects Versions: 3.0.0 >Reporter: Aihua Xu >Assignee: Aihua Xu >Priority: Major > > With hbase 2.0 support, I made some changes to the hbase table name property > change in HIVE-18366 and HIVE-18202. By checking the logic, seems the change > is not necessary since hbase.table.name is internal to hive hbase handler. We > just need to map hbase.table.name to > hbase.mapreduce.hfileoutputformat.table.name for HiveHFileOutputFormat. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18513) Query results caching
[
https://issues.apache.org/jira/browse/HIVE-18513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340305#comment-16340305
]
Hive QA commented on HIVE-18513:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12907641/HIVE-18513.3.patch
{color:green}SUCCESS:{color} +1 due to 7 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 19 failed/errored test(s), 11667 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=35)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mergejoin]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_input_format_excludes]
(batchId=162)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_part]
(batchId=94)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_aggregator_error_1]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=121)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testApplyPlanQpChanges
(batchId=285)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=254)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=186)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=232)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=232)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=232)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8837/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8837/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8837/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 19 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12907641 - PreCommit-HIVE-Build
> Query results caching
> -
>
> Key: HIVE-18513
> URL: https://issues.apache.org/jira/browse/HIVE-18513
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-18513.1.patch, HIVE-18513.2.patch,
> HIVE-18513.3.patch
>
>
> Add a query results cache that can save the results of an executed Hive query
> for reuse on subsequent queries. This may be useful in cases where the same
> query is issued many times, since Hive can return back the results of a
> cached query rather than having to execute the full query on the cluster.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-18549) Option to create null session handle for hive server.
[ https://issues.apache.org/jira/browse/HIVE-18549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Purshotam Shah reassigned HIVE-18549: - Assignee: Purshotam Shah > Option to create null session handle for hive server. > - > > Key: HIVE-18549 > URL: https://issues.apache.org/jira/browse/HIVE-18549 > Project: Hive > Issue Type: Bug >Reporter: Purshotam Shah >Assignee: Purshotam Shah >Priority: Major > > Every time we open a hive connection, it creates a session while includes > creating HDFS file. > To monitor hs2 health status, we have a web service api exposed, which > creates a connection to check the hs2 health status. > If there is any slowness/issue with NN, health check fails. > It will be nice if there an option to create a connection without creating a > session (no NN calls). > The connection with null handle should not be allowed to do any operation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18549) Option to create null session handle for hive server.
[ https://issues.apache.org/jira/browse/HIVE-18549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Purshotam Shah updated HIVE-18549: -- Description: Every time we open a hive connection, it creates a session while includes creating HDFS file. To monitor hs2 health status, we have a web service api exposed, which creates a connection to check the hs2 health status. If there is any slowness/issue with NN, health check fails. It will be nice if there an option to create a connection without creating a session (no NN calls). The connection with null handle should not be allowed to do any operation. > Option to create null session handle for hive server. > - > > Key: HIVE-18549 > URL: https://issues.apache.org/jira/browse/HIVE-18549 > Project: Hive > Issue Type: Bug >Reporter: Purshotam Shah >Priority: Major > > Every time we open a hive connection, it creates a session while includes > creating HDFS file. > To monitor hs2 health status, we have a web service api exposed, which > creates a connection to check the hs2 health status. > If there is any slowness/issue with NN, health check fails. > It will be nice if there an option to create a connection without creating a > session (no NN calls). > The connection with null handle should not be allowed to do any operation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17848) Bucket Map Join : Implement an efficient way to minimize loading hash table
[ https://issues.apache.org/jira/browse/HIVE-17848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340287#comment-16340287 ] Jason Dere commented on HIVE-17848: --- +1 pending tests. > Bucket Map Join : Implement an efficient way to minimize loading hash table > --- > > Key: HIVE-17848 > URL: https://issues.apache.org/jira/browse/HIVE-17848 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-17848.2.patch, HIVE-17848.4.patch, > HIVE-17848.5.patch, HIVE-17848.6.patch > > > In bucket mapjoin, each task loads its own copy of hash table which is > inefficient as load is IO heavy and due to multiple copies of same hash > table, the tables may get GCed on a busy system. > Implement a subcache with softreference to each hash table corresponding to > its bucketID such that it can be reused by a task. > This needs changes from Tez side to push bucket id to TezProcessor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18513) Query results caching
[
https://issues.apache.org/jira/browse/HIVE-18513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340288#comment-16340288
]
Hive QA commented on HIVE-18513:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
43s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
13s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
47s{color} | {color:red} ql: The patch generated 17 new + 1334 unchanged - 2
fixed = 1351 total (was 1336) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
57s{color} | {color:red} root: The patch generated 17 new + 1886 unchanged - 2
fixed = 1903 total (was 1888) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
10s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
11s{color} | {color:red} The patch generated 6 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 53m 1s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile
xml |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 948684a |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8837/yetus/diff-checkstyle-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8837/yetus/diff-checkstyle-root.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8837/yetus/patch-asflicense-problems.txt
|
| modules | C: common ql service . itests itests/hive-blobstore itests/util U:
. |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8837/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Query results caching
> -
>
> Key: HIVE-18513
> URL: https://issues.apache.org/jira/browse/HIVE-18513
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-18513.1.patch, HIVE-18513.2.patch,
> HIVE-18513.3.patch
>
>
> Add a query results cache that can save the results of an executed Hive query
> for reuse on subsequent queries. This may be useful in cases where the same
> query is issued many times, since Hive can return back the results of a
> cached query rather than having to execute the full query on the cluster.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18548) Fix log4j import
[
https://issues.apache.org/jira/browse/HIVE-18548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340271#comment-16340271
]
Prasanth Jayachandran commented on HIVE-18548:
--
lgtm, +1
> Fix log4j import
>
>
> Key: HIVE-18548
> URL: https://issues.apache.org/jira/browse/HIVE-18548
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18548.1.patch
>
>
> LLAP server code could run into invalid reference to log4j.MDC due to code
> importing {{org.apache.log4j.MDC}} instead of {{org.slf4j.MDC}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18548) Fix log4j import
[
https://issues.apache.org/jira/browse/HIVE-18548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18548:
---
Attachment: HIVE-18548.1.patch
> Fix log4j import
>
>
> Key: HIVE-18548
> URL: https://issues.apache.org/jira/browse/HIVE-18548
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18548.1.patch
>
>
> LLAP server code could run into invalid reference to log4j.MDC due to code
> importing {{org.apache.log4j.MDC}} instead of {{org.slf4j.MDC}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18548) Fix log4j import
[
https://issues.apache.org/jira/browse/HIVE-18548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18548:
---
Status: Patch Available (was: Open)
> Fix log4j import
>
>
> Key: HIVE-18548
> URL: https://issues.apache.org/jira/browse/HIVE-18548
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18548.1.patch
>
>
> LLAP server code could run into invalid reference to log4j.MDC due to code
> importing {{org.apache.log4j.MDC}} instead of {{org.slf4j.MDC}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18548) Fix log4j import
[
https://issues.apache.org/jira/browse/HIVE-18548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18548:
---
Description: LLAP server code could run into invalid reference to log4j.MDC
due to code importing {{org.apache.log4j.MDC}} instead of {org.slf4j.MDC }.
(was: LLAP server code could run into invalid reference to log4j.MDC due to
code importing {{org.apache.log4j.MDC}} instead of {{org.slf4j.MDC }}.)
> Fix log4j import
>
>
> Key: HIVE-18548
> URL: https://issues.apache.org/jira/browse/HIVE-18548
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
>
> LLAP server code could run into invalid reference to log4j.MDC due to code
> importing {{org.apache.log4j.MDC}} instead of {org.slf4j.MDC }.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18548) Fix log4j import
[
https://issues.apache.org/jira/browse/HIVE-18548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18548:
---
Description: LLAP server code could run into invalid reference to log4j.MDC
due to code importing {{org.apache.log4j.MDC}} instead of {{org.slf4j.MDC}}
(was: LLAP server code could run into invalid reference to log4j.MDC due to
code importing {{org.apache.log4j.MDC}} instead of {org.slf4j.MDC }.)
> Fix log4j import
>
>
> Key: HIVE-18548
> URL: https://issues.apache.org/jira/browse/HIVE-18548
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
>
> LLAP server code could run into invalid reference to log4j.MDC due to code
> importing {{org.apache.log4j.MDC}} instead of {{org.slf4j.MDC}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18548) Fix log4j import
[
https://issues.apache.org/jira/browse/HIVE-18548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340233#comment-16340233
]
Vineet Garg commented on HIVE-18548:
cc [~prasanth_j]
> Fix log4j import
>
>
> Key: HIVE-18548
> URL: https://issues.apache.org/jira/browse/HIVE-18548
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
>
> LLAP server code could run into invalid reference to log4j.MDC due to code
> importing {{org.apache.log4j.MDC}} instead of {{org.slf4j.MDC }}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-18548) Fix log4j import
[
https://issues.apache.org/jira/browse/HIVE-18548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg reassigned HIVE-18548:
--
> Fix log4j import
>
>
> Key: HIVE-18548
> URL: https://issues.apache.org/jira/browse/HIVE-18548
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
>
> LLAP server code could run into invalid reference to log4j.MDC due to code
> importing {{org.apache.log4j.MDC}} instead of {{org.slf4j.MDC }}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18426) Memory leak in RoutingAppender for every hive operation
[
https://issues.apache.org/jira/browse/HIVE-18426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340208#comment-16340208
]
Hive QA commented on HIVE-18426:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12907547/HIVE-18426.006.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 20 failed/errored test(s), 11661 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=35)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_input_format_excludes]
(batchId=162)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_part]
(batchId=94)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_aggregator_error_1]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=121)
org.apache.hadoop.hive.metastore.TestMarkPartition.testMarkingPartitionSet
(batchId=212)
org.apache.hadoop.hive.ql.TestCreateUdfEntities.testUdfWithDfsResource
(batchId=226)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=254)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=186)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=232)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=232)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=232)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8836/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8836/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8836/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 20 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12907547 - PreCommit-HIVE-Build
> Memory leak in RoutingAppender for every hive operation
> ---
>
> Key: HIVE-18426
> URL: https://issues.apache.org/jira/browse/HIVE-18426
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.0.0
>Reporter: kalyan kumar kalvagadda
>Assignee: kalyan kumar kalvagadda
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18426.001.patch, HIVE-18426.002.patch,
> HIVE-18426.003.patch, HIVE-18426.004.patch, HIVE-18426.005.patch,
> HIVE-18426.006.patch, Screen Shot 2018-01-10 at 9.17.19 AM.png
>
>
> Each new operation creates new entry in the ConcurrentMap in RoutingAppender
> but when the operation ends, AppenderControl stored in the map is retrieved
> and stopped but the entry in ConcurrentMap is never cleaned up.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18519) do not create materialized CTEs with ACID/MM
[ https://issues.apache.org/jira/browse/HIVE-18519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18519: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Committed to master. Thanks for the review! > do not create materialized CTEs with ACID/MM > > > Key: HIVE-18519 > URL: https://issues.apache.org/jira/browse/HIVE-18519 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18519.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18547) WM: trigger test may fail
[
https://issues.apache.org/jira/browse/HIVE-18547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-18547:
Description:
https://builds.apache.org/job/PreCommit-HIVE-Build/8818/testReport/org.apache.hive.jdbc/TestTriggersMoveWorkloadManager/testTriggerMoveAndKill/
Looks like the cluster allocation assignment and WM event creation race,
probably because WM returns session to the caller ASAP and then makes the
changes after that.
{noformat}
'Event: GET Pool: BI Cluster %: 80.00' expected in STDERR capture, but not
found.
...
2018-01-24T15:07:31,746 INFO [Workload management master] tez.WorkloadManager:
Processing changes for pool BI: [BI, query parallelism 1, fraction of the
cluster 0.80011920929, fraction used by child pools 0.0, active sessions 0,
initializing sessions 0]
2018-01-24T15:07:31,746 INFO [Workload management master] tez.WorkloadManager:
Starting 1 queries in pool [BI, query parallelism 1, fraction of the cluster
0.80011920929, fraction used by child pools 0.0, active sessions 0,
initializing sessions 0]
2018-01-24T15:07:31,746 INFO [Workload management master] tez.WorkloadManager:
Received a session from AM pool sessionId=2be29c62-9f2c-40b7-a5eb-6298baf83a34,
queueName=default, user=hiveptest, doAs=false, isOpen=true, isDefault=true,
expires in 588529859ms, WM state poolName=null, clusterFraction=0.0,
queryId=null, killReason=null
2018-01-24T15:07:31,746 INFO [HiveServer2-Background-Pool: Thread-1377]
tez.WmEvent: Added WMEvent: EventType: GET EventStartTimestamp: 1516835251746
elapsedTime: 0 wmTezSessionInfo:SessionId: 2be29c62-9f2c-40b7-a5eb-6298baf83a34
Pool: BI Cluster %: 0.0
2018-01-24T15:07:31,746 INFO [Workload management master]
tez.GuaranteedTasksAllocator: Updating 2be29c62-9f2c-40b7-a5eb-6298baf83a34
with 3 guaranteed tasks
{noformat}
was:
https://builds.apache.org/job/PreCommit-HIVE-Build/8818/testReport/org.apache.hive.jdbc/TestTriggersMoveWorkloadManager/testTriggerMoveAndKill/
Looks like the cluster allocation assignment and WM event creation race somehow
{noformat}
'Event: GET Pool: BI Cluster %: 80.00' expected in STDERR capture, but not
found.
...
2018-01-24T15:07:31,746 INFO [Workload management master] tez.WorkloadManager:
Processing changes for pool BI: [BI, query parallelism 1, fraction of the
cluster 0.80011920929, fraction used by child pools 0.0, active sessions 0,
initializing sessions 0]
2018-01-24T15:07:31,746 INFO [Workload management master] tez.WorkloadManager:
Starting 1 queries in pool [BI, query parallelism 1, fraction of the cluster
0.80011920929, fraction used by child pools 0.0, active sessions 0,
initializing sessions 0]
2018-01-24T15:07:31,746 INFO [Workload management master] tez.WorkloadManager:
Received a session from AM pool sessionId=2be29c62-9f2c-40b7-a5eb-6298baf83a34,
queueName=default, user=hiveptest, doAs=false, isOpen=true, isDefault=true,
expires in 588529859ms, WM state poolName=null, clusterFraction=0.0,
queryId=null, killReason=null
2018-01-24T15:07:31,746 INFO [HiveServer2-Background-Pool: Thread-1377]
tez.WmEvent: Added WMEvent: EventType: GET EventStartTimestamp: 1516835251746
elapsedTime: 0 wmTezSessionInfo:SessionId: 2be29c62-9f2c-40b7-a5eb-6298baf83a34
Pool: BI Cluster %: 0.0
2018-01-24T15:07:31,746 INFO [Workload management master]
tez.GuaranteedTasksAllocator: Updating 2be29c62-9f2c-40b7-a5eb-6298baf83a34
with 3 guaranteed tasks
{noformat}
> WM: trigger test may fail
> -
>
> Key: HIVE-18547
> URL: https://issues.apache.org/jira/browse/HIVE-18547
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Priority: Major
>
> https://builds.apache.org/job/PreCommit-HIVE-Build/8818/testReport/org.apache.hive.jdbc/TestTriggersMoveWorkloadManager/testTriggerMoveAndKill/
> Looks like the cluster allocation assignment and WM event creation race,
> probably because WM returns session to the caller ASAP and then makes the
> changes after that.
> {noformat}
> 'Event: GET Pool: BI Cluster %: 80.00' expected in STDERR capture, but not
> found.
> ...
> 2018-01-24T15:07:31,746 INFO [Workload management master]
> tez.WorkloadManager: Processing changes for pool BI: [BI, query parallelism
> 1, fraction of the cluster 0.80011920929, fraction used by child pools
> 0.0, active sessions 0, initializing sessions 0]
> 2018-01-24T15:07:31,746 INFO [Workload management master]
> tez.WorkloadManager: Starting 1 queries in pool [BI, query parallelism 1,
> fraction of the cluster 0.80011920929, fraction used by child pools 0.0,
> active sessions 0, initializing sessions 0]
> 2018-01-24T15:07:31,746 INFO [Workload management master]
> tez.WorkloadManager: Received a session from AM pool
> sessionId=2be29c62-9f2c-40b7-a5eb-6298baf83a34, queueName=default,
> user=hive
[jira] [Commented] (HIVE-18547) WM: trigger test may fail
[
https://issues.apache.org/jira/browse/HIVE-18547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340160#comment-16340160
]
Sergey Shelukhin commented on HIVE-18547:
-
cc [~prasanth_j]
> WM: trigger test may fail
> -
>
> Key: HIVE-18547
> URL: https://issues.apache.org/jira/browse/HIVE-18547
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Priority: Major
>
> https://builds.apache.org/job/PreCommit-HIVE-Build/8818/testReport/org.apache.hive.jdbc/TestTriggersMoveWorkloadManager/testTriggerMoveAndKill/
> Looks like the cluster allocation assignment and WM event creation race
> somehow
> {noformat}
> 'Event: GET Pool: BI Cluster %: 80.00' expected in STDERR capture, but not
> found.
> ...
> 2018-01-24T15:07:31,746 INFO [Workload management master]
> tez.WorkloadManager: Processing changes for pool BI: [BI, query parallelism
> 1, fraction of the cluster 0.80011920929, fraction used by child pools
> 0.0, active sessions 0, initializing sessions 0]
> 2018-01-24T15:07:31,746 INFO [Workload management master]
> tez.WorkloadManager: Starting 1 queries in pool [BI, query parallelism 1,
> fraction of the cluster 0.80011920929, fraction used by child pools 0.0,
> active sessions 0, initializing sessions 0]
> 2018-01-24T15:07:31,746 INFO [Workload management master]
> tez.WorkloadManager: Received a session from AM pool
> sessionId=2be29c62-9f2c-40b7-a5eb-6298baf83a34, queueName=default,
> user=hiveptest, doAs=false, isOpen=true, isDefault=true, expires in
> 588529859ms, WM state poolName=null, clusterFraction=0.0, queryId=null,
> killReason=null
> 2018-01-24T15:07:31,746 INFO [HiveServer2-Background-Pool: Thread-1377]
> tez.WmEvent: Added WMEvent: EventType: GET EventStartTimestamp: 1516835251746
> elapsedTime: 0 wmTezSessionInfo:SessionId:
> 2be29c62-9f2c-40b7-a5eb-6298baf83a34 Pool: BI Cluster %: 0.0
> 2018-01-24T15:07:31,746 INFO [Workload management master]
> tez.GuaranteedTasksAllocator: Updating 2be29c62-9f2c-40b7-a5eb-6298baf83a34
> with 3 guaranteed tasks
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18501) Typo in beeline code
[ https://issues.apache.org/jira/browse/HIVE-18501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam updated HIVE-18501: - Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Fix has been pushed to master. Thanks for the review [~ychena] > Typo in beeline code > > > Key: HIVE-18501 > URL: https://issues.apache.org/jira/browse/HIVE-18501 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 3.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Trivial > Fix For: 3.0.0 > > Attachments: HIVE-18501.patch, HIVE-18501.patch, HIVE-18501.patch > > > [https://github.com/apache/hive/blob/master/beeline/src/java/org/apache/hive/beeline/BeeLine.java#L744] > the string literal used here should be "silent", not "slient". There is no > functional bug here just a silly typo. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18426) Memory leak in RoutingAppender for every hive operation
[
https://issues.apache.org/jira/browse/HIVE-18426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340108#comment-16340108
]
Hive QA commented on HIVE-18426:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} The patch common passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} itests/hive-unit: The patch generated 0 new + 4
unchanged - 3 fixed = 4 total (was 7) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
13s{color} | {color:red} The patch generated 6 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 35s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 3f7127f |
| Default Java | 1.8.0_111 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8836/yetus/patch-asflicense-problems.txt
|
| modules | C: common itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8836/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Memory leak in RoutingAppender for every hive operation
> ---
>
> Key: HIVE-18426
> URL: https://issues.apache.org/jira/browse/HIVE-18426
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.0.0
>Reporter: kalyan kumar kalvagadda
>Assignee: kalyan kumar kalvagadda
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18426.001.patch, HIVE-18426.002.patch,
> HIVE-18426.003.patch, HIVE-18426.004.patch, HIVE-18426.005.patch,
> HIVE-18426.006.patch, Screen Shot 2018-01-10 at 9.17.19 AM.png
>
>
> Each new operation creates new entry in the ConcurrentMap in RoutingAppender
> but when the operation ends, AppenderControl stored in the map is retrieved
> and stopped but the entry in ConcurrentMap is never cleaned up.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18530) Replication should skip MM table (for now)
[
https://issues.apache.org/jira/browse/HIVE-18530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340104#comment-16340104
]
Thejas M Nair commented on HIVE-18530:
--
+1 pending tests
> Replication should skip MM table (for now)
> --
>
> Key: HIVE-18530
> URL: https://issues.apache.org/jira/browse/HIVE-18530
> Project: Hive
> Issue Type: Bug
> Components: repl
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-18530.1.patch
>
>
> Currently replication cannot handle transactional table (including MM table)
> until HIVE-18320. HIVE-17504 skips table with transactional=true explicitly.
> HIVE-18352 changes the logic to use AcidUtils.isAcidTable for the same
> purpose. However, isAcidTable returns false for mm table, thus Hive still
> dump mm table during replication.
> Here is an error message during dump mm table:
> {code}
> ERROR : FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.repl.ReplDumpTask.
> java.io.FileNotFoundException: Path is not a file:
> /apps/hive/warehouse/testrepldb5.db/test1/delta_261_261_
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:89)
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:75)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:154)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1920)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:731)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:424)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:869)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:815)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1965)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2675)
> INFO : Completed executing
> command(queryId=hive_20180119203438_293813df-7630-47fa-bc30-5ef7cbb42842);
> Time taken: 1.119 seconds
> Error: Error while processing statement: FAILED: Execution Error, return code
> 1 from org.apache.hadoop.hive.ql.exec.repl.ReplDumpTask.
> java.io.FileNotFoundException: Path is not a file:
> /apps/hive/warehouse/testrepldb5.db/test1/delta_261_261_
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:89)
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:75)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:154)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1920)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:731)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:424)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:869)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:815)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1965)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2675)
> (state=08S01,code=1)
> 0: jdbc:hive2://ctr-e137-1514896590304-25219-> Closing: 0:
> jdbc:hive2://ctr-e137-1514896590304-25219-02-05.hwx.site:2181,ctr-e137-1514896590304-25219-02-12.hwx.site:2181,ctr-e137-1514896590304-25219-02-09.hwx.site:2181,ctr-e137-1514896590304-25219-02-04.hwx.site:2181/;servi
[jira] [Commented] (HIVE-18436) Upgrade to Spark 2.3.0
[
https://issues.apache.org/jira/browse/HIVE-18436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340091#comment-16340091
]
Sahil Takiar commented on HIVE-18436:
-
Found another, separate issue. This one isn't a Spark bug though. Looks like
SPARK-21422 added a new dependency on ORC libraries, so now the Spark
distribution contains ORC jars. The version they are using is different from
ours (1.4.1 vs. 1.4.0), which leads to some {{NoSuchMethodError}} when running
{{TestSparkCliDriver}}. Luckily, SPARK-21422 added an {{orc-provided}} profile,
so we just have to build the Spark distribution with {{orc-provided}} going
forward. Should probably add it to the wiki too. Will update the distribution
and re-test.
> Upgrade to Spark 2.3.0
> --
>
> Key: HIVE-18436
> URL: https://issues.apache.org/jira/browse/HIVE-18436
> Project: Hive
> Issue Type: Task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
>
> Branching has been completed. Release candidates should be published soon.
> Might be a while before the actual release, but at least we get to identify
> any issues early.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18247) Use DB auto-increment for indexes
[
https://issues.apache.org/jira/browse/HIVE-18247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340088#comment-16340088
]
Hive QA commented on HIVE-18247:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12907546/HIVE-18247.02.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 20 failed/errored test(s), 11654 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=35)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_input_format_excludes]
(batchId=162)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_part]
(batchId=94)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_aggregator_error_1]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=121)
org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver.org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver
(batchId=248)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=254)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testExchangePartition
(batchId=226)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=186)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=232)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=232)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=232)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8835/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8835/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8835/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 20 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12907546 - PreCommit-HIVE-Build
> Use DB auto-increment for indexes
> -
>
> Key: HIVE-18247
> URL: https://issues.apache.org/jira/browse/HIVE-18247
> Project: Hive
> Issue Type: Bug
> Components: Hive, Metastore
>Affects Versions: 3.0.0
>Reporter: Alexander Kolbasov
>Assignee: Alexander Kolbasov
>Priority: Major
> Labels: datanucleus, perfomance
> Attachments: HIVE-18247.02.patch
>
>
> I initially noticed this problem in Apache Sentry - see SENTRY-1960. Hive has
> the same issue. DataNucleus uses SEQUENCE table to allocate IDs which
> requires raw locks on multiple tables during transactions and this creates
> scalability problems.
> Instead DN should rely on DB auto-increment mechanisms which are much more
> scalable.
> See SENTRY-1960 for extra details.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18061) q.outs: be more selective with masking hdfs paths
[
https://issues.apache.org/jira/browse/HIVE-18061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16339817#comment-16339817
]
kalyan kumar kalvagadda commented on HIVE-18061:
[~abstractdog] You have a Jira to handle the failures reported because of new
{{out_spark}} files added,
> q.outs: be more selective with masking hdfs paths
> -
>
> Key: HIVE-18061
> URL: https://issues.apache.org/jira/browse/HIVE-18061
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Laszlo Bodor
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18061.01.patch, HIVE-18061.02.patch,
> HIVE-18061.03.patch, HIVE-18061.04.patch, HIVE-18061.05.patch,
> HIVE-18061.06.patch, HIVE-18061.07.patch, HIVE-18061.08.patch,
> HIVE-18061.09.patch, HIVE-18061.10.patch, HIVE-18061.11.patch,
> HIVE-18061.12.patch, HIVE-18061.13.patch, HIVE-18061.14.patch
>
>
> currently any line which contains a path which looks like an hdfs location is
> replaced with a "masked pattern was here"...
> it might be releavant to record these messages; since even an exception
> message might contain an hdfs location
> noticed in
> HIVE-18012
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18501) Typo in beeline code
[ https://issues.apache.org/jira/browse/HIVE-18501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16339816#comment-16339816 ] Naveen Gangam commented on HIVE-18501: -- The test failures do not appear to be related to the fix. The prior builds also have the same failures. So +1 for me. > Typo in beeline code > > > Key: HIVE-18501 > URL: https://issues.apache.org/jira/browse/HIVE-18501 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 3.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Trivial > Attachments: HIVE-18501.patch, HIVE-18501.patch, HIVE-18501.patch > > > [https://github.com/apache/hive/blob/master/beeline/src/java/org/apache/hive/beeline/BeeLine.java#L744] > the string literal used here should be "silent", not "slient". There is no > functional bug here just a silly typo. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18524) Vectorization: Execution failure related to non-standard embedding of IfExprConditionalFilter inside VectorUDFAdaptor (Revert HIVE-17139)
[
https://issues.apache.org/jira/browse/HIVE-18524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-18524:
Attachment: HIVE-18524.01.patch
> Vectorization: Execution failure related to non-standard embedding of
> IfExprConditionalFilter inside VectorUDFAdaptor (Revert HIVE-17139)
> -
>
> Key: HIVE-18524
> URL: https://issues.apache.org/jira/browse/HIVE-18524
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HIVE-18524.01.patch
>
>
> {noformat}
> insert overwrite table insert_10_1
> select cast(gpa as float),
>age,
>IF(age>40,cast('2011-01-01 01:01:01' as timestamp),NULL),
>IF(LENGTH(name)>10,cast(name as binary),NULL)
> from studentnull10k
> vectorizationSchemaColumns: [0:name:string, 1:age:int, 2:gpa:double]
> ExprNodeDescs:
> UDFToFloat(gpa) (type: float),
> age (type: int),
> if((age > 40), 2011-01-01 01:01:01.0, null) (type: timestamp),
> if((length(name) > 10), CAST( name AS BINARY), null) (type: binary)
> selectExpressions:
> VectorUDFAdaptor(if((age > 40), 2011-01-01 01:01:01.0, null))
> (children: LongColGreaterLongScalar(col 1:int, val 40) -> 4:boolean)
> -> 5:timestamp,
> VectorUDFAdaptor(if((length(name) > 10), CAST( name AS BINARY), null))
> (children: LongColGreaterLongScalar(col 4:int, val 10)(children:
> StringLength(col 0:string) -> 4:int) -> 6:boolean,
> VectorUDFAdaptor(CAST( name AS BINARY)) -> 7:binary) -> 8:binary
> {noformat}
> *// Notice there is no vector expression shown for the last IF stmt.* It has
> been magically embedded inside the VectorUDFAdaptor object...
> Execution results in this call stack.
> {nocode}
> Caused by: java.lang.NullPointerException
> at java.util.Arrays.copyOfRange(Arrays.java:3521)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$9.writeValue(VectorExpressionWriterFactory.java:1101)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$VectorExpressionWriterBytes.writeValue(VectorExpressionWriterFactory.java:343)
> at
> org.apache.hadoop.hive.ql.exec.vector.udf.VectorUDFArgDesc.getDeferredJavaObject(VectorUDFArgDesc.java:123)
> at
> org.apache.hadoop.hive.ql.exec.vector.udf.VectorUDFAdaptor.setResult(VectorUDFAdaptor.java:211)
> at
> org.apache.hadoop.hive.ql.exec.vector.udf.VectorUDFAdaptor.evaluate(VectorUDFAdaptor.java:177)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:145)
> ... 22 more
> {nocode}
> Change is due to:
> HIVE-17139: Conditional expressions optimization: skip the expression
> evaluation if the condition is not satisfied for vectorization engine. (Jia
> Ke, reviewed by Ferdinand Xu)
> Embedding a raw vector expression outside of VectorizationContext is quite
> non-standard and evidently buggy.
> [~Ferd] [~Ke Jia] I am inclined to revert this change. Comments? CC:
> [~ashutoshc] [~hagleitn]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-18426) Memory leak in RoutingAppender for every hive operation
[
https://issues.apache.org/jira/browse/HIVE-18426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16339811#comment-16339811
]
kalyan kumar kalvagadda edited comment on HIVE-18426 at 1/25/18 8:19 PM:
-
Failures reported below are introduced by HIVE-18061.
|{color:#FF}-1{color}|{color:#FF}asflicense{color}|{color:#FF}0m
12s{color}|{color:#FF}The patch generated 6 ASF License warnings.{color}|
was (Author: kkalyan):
Failures reported below are introduced by HIVE-18061.
{noformat}
|{color:#FF}-1{color}|{color:#FF}asflicense{color}|{color:#FF}0m
12s{color}|{color:#FF}The patch generated 6 ASF License warnings.{color}|
{noformat}
> Memory leak in RoutingAppender for every hive operation
> ---
>
> Key: HIVE-18426
> URL: https://issues.apache.org/jira/browse/HIVE-18426
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.0.0
>Reporter: kalyan kumar kalvagadda
>Assignee: kalyan kumar kalvagadda
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18426.001.patch, HIVE-18426.002.patch,
> HIVE-18426.003.patch, HIVE-18426.004.patch, HIVE-18426.005.patch,
> HIVE-18426.006.patch, Screen Shot 2018-01-10 at 9.17.19 AM.png
>
>
> Each new operation creates new entry in the ConcurrentMap in RoutingAppender
> but when the operation ends, AppenderControl stored in the map is retrieved
> and stopped but the entry in ConcurrentMap is never cleaned up.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18524) Vectorization: Execution failure related to non-standard embedding of IfExprConditionalFilter inside VectorUDFAdaptor (Revert HIVE-17139)
[
https://issues.apache.org/jira/browse/HIVE-18524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-18524:
Attachment: (was: HIVE-18524.01.patch)
> Vectorization: Execution failure related to non-standard embedding of
> IfExprConditionalFilter inside VectorUDFAdaptor (Revert HIVE-17139)
> -
>
> Key: HIVE-18524
> URL: https://issues.apache.org/jira/browse/HIVE-18524
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Fix For: 3.0.0
>
>
> {noformat}
> insert overwrite table insert_10_1
> select cast(gpa as float),
>age,
>IF(age>40,cast('2011-01-01 01:01:01' as timestamp),NULL),
>IF(LENGTH(name)>10,cast(name as binary),NULL)
> from studentnull10k
> vectorizationSchemaColumns: [0:name:string, 1:age:int, 2:gpa:double]
> ExprNodeDescs:
> UDFToFloat(gpa) (type: float),
> age (type: int),
> if((age > 40), 2011-01-01 01:01:01.0, null) (type: timestamp),
> if((length(name) > 10), CAST( name AS BINARY), null) (type: binary)
> selectExpressions:
> VectorUDFAdaptor(if((age > 40), 2011-01-01 01:01:01.0, null))
> (children: LongColGreaterLongScalar(col 1:int, val 40) -> 4:boolean)
> -> 5:timestamp,
> VectorUDFAdaptor(if((length(name) > 10), CAST( name AS BINARY), null))
> (children: LongColGreaterLongScalar(col 4:int, val 10)(children:
> StringLength(col 0:string) -> 4:int) -> 6:boolean,
> VectorUDFAdaptor(CAST( name AS BINARY)) -> 7:binary) -> 8:binary
> {noformat}
> *// Notice there is no vector expression shown for the last IF stmt.* It has
> been magically embedded inside the VectorUDFAdaptor object...
> Execution results in this call stack.
> {nocode}
> Caused by: java.lang.NullPointerException
> at java.util.Arrays.copyOfRange(Arrays.java:3521)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$9.writeValue(VectorExpressionWriterFactory.java:1101)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$VectorExpressionWriterBytes.writeValue(VectorExpressionWriterFactory.java:343)
> at
> org.apache.hadoop.hive.ql.exec.vector.udf.VectorUDFArgDesc.getDeferredJavaObject(VectorUDFArgDesc.java:123)
> at
> org.apache.hadoop.hive.ql.exec.vector.udf.VectorUDFAdaptor.setResult(VectorUDFAdaptor.java:211)
> at
> org.apache.hadoop.hive.ql.exec.vector.udf.VectorUDFAdaptor.evaluate(VectorUDFAdaptor.java:177)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:145)
> ... 22 more
> {nocode}
> Change is due to:
> HIVE-17139: Conditional expressions optimization: skip the expression
> evaluation if the condition is not satisfied for vectorization engine. (Jia
> Ke, reviewed by Ferdinand Xu)
> Embedding a raw vector expression outside of VectorizationContext is quite
> non-standard and evidently buggy.
> [~Ferd] [~Ke Jia] I am inclined to revert this change. Comments? CC:
> [~ashutoshc] [~hagleitn]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18426) Memory leak in RoutingAppender for every hive operation
[
https://issues.apache.org/jira/browse/HIVE-18426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16339811#comment-16339811
]
kalyan kumar kalvagadda commented on HIVE-18426:
Failures reported below are introduced by HIVE-18061.
{noformat}
|{color:#FF}-1{color}|{color:#FF}asflicense{color}|{color:#FF}0m
12s{color}|{color:#FF}The patch generated 6 ASF License warnings.{color}|
{noformat}
> Memory leak in RoutingAppender for every hive operation
> ---
>
> Key: HIVE-18426
> URL: https://issues.apache.org/jira/browse/HIVE-18426
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.0.0
>Reporter: kalyan kumar kalvagadda
>Assignee: kalyan kumar kalvagadda
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18426.001.patch, HIVE-18426.002.patch,
> HIVE-18426.003.patch, HIVE-18426.004.patch, HIVE-18426.005.patch,
> HIVE-18426.006.patch, Screen Shot 2018-01-10 at 9.17.19 AM.png
>
>
> Each new operation creates new entry in the ConcurrentMap in RoutingAppender
> but when the operation ends, AppenderControl stored in the map is retrieved
> and stopped but the entry in ConcurrentMap is never cleaned up.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18247) Use DB auto-increment for indexes
[
https://issues.apache.org/jira/browse/HIVE-18247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16339800#comment-16339800
]
Hive QA commented on HIVE-18247:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
12s{color} | {color:red} The patch generated 6 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 9m 23s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 3f7127f |
| Default Java | 1.8.0_111 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8835/yetus/patch-asflicense-problems.txt
|
| modules | C: standalone-metastore U: standalone-metastore |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8835/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Use DB auto-increment for indexes
> -
>
> Key: HIVE-18247
> URL: https://issues.apache.org/jira/browse/HIVE-18247
> Project: Hive
> Issue Type: Bug
> Components: Hive, Metastore
>Affects Versions: 3.0.0
>Reporter: Alexander Kolbasov
>Assignee: Alexander Kolbasov
>Priority: Major
> Labels: datanucleus, perfomance
> Attachments: HIVE-18247.02.patch
>
>
> I initially noticed this problem in Apache Sentry - see SENTRY-1960. Hive has
> the same issue. DataNucleus uses SEQUENCE table to allocate IDs which
> requires raw locks on multiple tables during transactions and this creates
> scalability problems.
> Instead DN should rely on DB auto-increment mechanisms which are much more
> scalable.
> See SENTRY-1960 for extra details.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18469) HS2UI: Introduce separate option to show query on web ui
[
https://issues.apache.org/jira/browse/HIVE-18469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-18469:
Issue Type: Improvement (was: Bug)
> HS2UI: Introduce separate option to show query on web ui
>
>
> Key: HIVE-18469
> URL: https://issues.apache.org/jira/browse/HIVE-18469
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Priority: Major
>
> currently {{ConfVars.HIVE_LOG_EXPLAIN_OUTPUT}} enables 2 features:
> * log the query to the console (even thru beeline)
> * shows the query on the web ui
> I've enabled it...and ever since then my beeline is always flooded with an
> {{explain extended}} output...which is very verbose; even for simple queries.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18469) HS2UI: Introduce separate option to show query on web ui
[
https://issues.apache.org/jira/browse/HIVE-18469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-18469:
Description:
currently {{ConfVars.HIVE_LOG_EXPLAIN_OUTPUT}} enables 2 features:
* log the query to the console (even thru beeline)
* shows the query on the web ui
I've enabled it...and ever since then my beeline is always flooded with an
{{explain extended}} output...which is very verbose; even for simple queries.
> HS2UI: Introduce separate option to show query on web ui
>
>
> Key: HIVE-18469
> URL: https://issues.apache.org/jira/browse/HIVE-18469
> Project: Hive
> Issue Type: Bug
> Environment: currently {{ConfVars.HIVE_LOG_EXPLAIN_OUTPUT}} enables 2
> features:
> * log the query to the console (even thru beeline)
> * shows the query on the web ui
> I've enabled it...and ever since then my beeline is always flooded with an
> {{explain extended}} output...which is very verbose; even for simple queries.
>Reporter: Zoltan Haindrich
>Priority: Major
>
> currently {{ConfVars.HIVE_LOG_EXPLAIN_OUTPUT}} enables 2 features:
> * log the query to the console (even thru beeline)
> * shows the query on the web ui
> I've enabled it...and ever since then my beeline is always flooded with an
> {{explain extended}} output...which is very verbose; even for simple queries.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18469) HS2UI: Introduce separate option to show query on web ui
[
https://issues.apache.org/jira/browse/HIVE-18469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-18469:
Environment: (was: currently {{ConfVars.HIVE_LOG_EXPLAIN_OUTPUT}}
enables 2 features:
* log the query to the console (even thru beeline)
* shows the query on the web ui
I've enabled it...and ever since then my beeline is always flooded with an
{{explain extended}} output...which is very verbose; even for simple queries.)
> HS2UI: Introduce separate option to show query on web ui
>
>
> Key: HIVE-18469
> URL: https://issues.apache.org/jira/browse/HIVE-18469
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Priority: Major
>
> currently {{ConfVars.HIVE_LOG_EXPLAIN_OUTPUT}} enables 2 features:
> * log the query to the console (even thru beeline)
> * shows the query on the web ui
> I've enabled it...and ever since then my beeline is always flooded with an
> {{explain extended}} output...which is very verbose; even for simple queries.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18426) Memory leak in RoutingAppender for every hive operation
[
https://issues.apache.org/jira/browse/HIVE-18426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16339777#comment-16339777
]
Hive QA commented on HIVE-18426:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12907547/HIVE-18426.006.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 19 failed/errored test(s), 11661 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=35)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_input_format_excludes]
(batchId=162)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_part]
(batchId=94)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_aggregator_error_1]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=121)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query64]
(batchId=246)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=254)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=186)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=232)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=232)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=232)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8834/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8834/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8834/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 19 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12907547 - PreCommit-HIVE-Build
> Memory leak in RoutingAppender for every hive operation
> ---
>
> Key: HIVE-18426
> URL: https://issues.apache.org/jira/browse/HIVE-18426
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.0.0
>Reporter: kalyan kumar kalvagadda
>Assignee: kalyan kumar kalvagadda
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18426.001.patch, HIVE-18426.002.patch,
> HIVE-18426.003.patch, HIVE-18426.004.patch, HIVE-18426.005.patch,
> HIVE-18426.006.patch, Screen Shot 2018-01-10 at 9.17.19 AM.png
>
>
> Each new operation creates new entry in the ConcurrentMap in RoutingAppender
> but when the operation ends, AppenderControl stored in the map is retrieved
> and stopped but the entry in ConcurrentMap is never cleaned up.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18546) Remove unnecessary code introduced in HIVE-14498
[ https://issues.apache.org/jira/browse/HIVE-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-18546: --- Attachment: (was: HIVE-18546.01.patch) > Remove unnecessary code introduced in HIVE-14498 > > > Key: HIVE-18546 > URL: https://issues.apache.org/jira/browse/HIVE-18546 > Project: Hive > Issue Type: Bug > Components: Materialized views >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-18546.02.patch > > > HIVE-14498 introduced some code to check the invalidation of materialized > views that can be simplified, relying instead on existing transaction ids. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18546) Remove unnecessary code introduced in HIVE-14498
[ https://issues.apache.org/jira/browse/HIVE-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-18546: --- Attachment: HIVE-18546.02.patch > Remove unnecessary code introduced in HIVE-14498 > > > Key: HIVE-18546 > URL: https://issues.apache.org/jira/browse/HIVE-18546 > Project: Hive > Issue Type: Bug > Components: Materialized views >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-18546.02.patch > > > HIVE-14498 introduced some code to check the invalidation of materialized > views that can be simplified, relying instead on existing transaction ids. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18426) Memory leak in RoutingAppender for every hive operation
[
https://issues.apache.org/jira/browse/HIVE-18426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16339639#comment-16339639
]
Hive QA commented on HIVE-18426:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
30s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} The patch common passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} itests/hive-unit: The patch generated 0 new + 4
unchanged - 3 fixed = 4 total (was 7) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
12s{color} | {color:red} The patch generated 6 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 13s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 3f7127f |
| Default Java | 1.8.0_111 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8834/yetus/patch-asflicense-problems.txt
|
| modules | C: common itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8834/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Memory leak in RoutingAppender for every hive operation
> ---
>
> Key: HIVE-18426
> URL: https://issues.apache.org/jira/browse/HIVE-18426
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.0.0
>Reporter: kalyan kumar kalvagadda
>Assignee: kalyan kumar kalvagadda
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18426.001.patch, HIVE-18426.002.patch,
> HIVE-18426.003.patch, HIVE-18426.004.patch, HIVE-18426.005.patch,
> HIVE-18426.006.patch, Screen Shot 2018-01-10 at 9.17.19 AM.png
>
>
> Each new operation creates new entry in the ConcurrentMap in RoutingAppender
> but when the operation ends, AppenderControl stored in the map is retrieved
> and stopped but the entry in ConcurrentMap is never cleaned up.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18501) Typo in beeline code
[
https://issues.apache.org/jira/browse/HIVE-18501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16339613#comment-16339613
]
Hive QA commented on HIVE-18501:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12907525/HIVE-18501.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 17 failed/errored test(s), 11660 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=35)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_input_format_excludes]
(batchId=162)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_part]
(batchId=94)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_aggregator_error_1]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=121)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query39]
(batchId=248)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=254)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=186)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=232)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=232)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=232)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8833/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8833/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8833/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 17 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12907525 - PreCommit-HIVE-Build
> Typo in beeline code
>
>
> Key: HIVE-18501
> URL: https://issues.apache.org/jira/browse/HIVE-18501
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 3.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Trivial
> Attachments: HIVE-18501.patch, HIVE-18501.patch, HIVE-18501.patch
>
>
> [https://github.com/apache/hive/blob/master/beeline/src/java/org/apache/hive/beeline/BeeLine.java#L744]
> the string literal used here should be "silent", not "slient". There is no
> functional bug here just a silly typo.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18532) Hive Connection can't be closed if there is an error while creating session
[
https://issues.apache.org/jira/browse/HIVE-18532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Purshotam Shah updated HIVE-18532:
--
Status: Patch Available (was: Open)
> Hive Connection can't be closed if there is an error while creating session
> ---
>
> Key: HIVE-18532
> URL: https://issues.apache.org/jira/browse/HIVE-18532
> Project: Hive
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Purshotam Shah
>Priority: Major
> Attachments: HIVE-18532-V1.patch
>
>
> To open a hiveconnection we have to
> {code}
> HiveConnection connection = null;
> try {
> connection = new HiveConnection(uri, new Properties());
> } finally {
> if (connection != null)
> try {
> connection.close();
> } catch (SQLException e) {
> LOGGER.error("Error closing connection, ignored.", e);
> }
> }
> {code}
> HiveConnection opens the opensTransport and tries to create a session in the
> constructor.
> If opensTransport succeed, but session creates fails because of namenode
> error or any other error, HiveConnection can't be closed.
> Since construction fails, the hiveconnection object will be null and the
> client can't close it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18532) Hive Connection can't be closed if there is an error while creating session
[
https://issues.apache.org/jira/browse/HIVE-18532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Purshotam Shah updated HIVE-18532:
--
Attachment: HIVE-18532-V1.patch
> Hive Connection can't be closed if there is an error while creating session
> ---
>
> Key: HIVE-18532
> URL: https://issues.apache.org/jira/browse/HIVE-18532
> Project: Hive
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Purshotam Shah
>Priority: Major
> Attachments: HIVE-18532-V1.patch
>
>
> To open a hiveconnection we have to
> {code}
> HiveConnection connection = null;
> try {
> connection = new HiveConnection(uri, new Properties());
> } finally {
> if (connection != null)
> try {
> connection.close();
> } catch (SQLException e) {
> LOGGER.error("Error closing connection, ignored.", e);
> }
> }
> {code}
> HiveConnection opens the opensTransport and tries to create a session in the
> constructor.
> If opensTransport succeed, but session creates fails because of namenode
> error or any other error, HiveConnection can't be closed.
> Since construction fails, the hiveconnection object will be null and the
> client can't close it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18546) Remove unnecessary code introduced in HIVE-14498
[ https://issues.apache.org/jira/browse/HIVE-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-18546: --- Attachment: (was: HIVE-18546.patch) > Remove unnecessary code introduced in HIVE-14498 > > > Key: HIVE-18546 > URL: https://issues.apache.org/jira/browse/HIVE-18546 > Project: Hive > Issue Type: Bug > Components: Materialized views >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-18546.01.patch > > > HIVE-14498 introduced some code to check the invalidation of materialized > views that can be simplified, relying instead on existing transaction ids. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18546) Remove unnecessary code introduced in HIVE-14498
[ https://issues.apache.org/jira/browse/HIVE-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-18546: --- Attachment: HIVE-18546.01.patch > Remove unnecessary code introduced in HIVE-14498 > > > Key: HIVE-18546 > URL: https://issues.apache.org/jira/browse/HIVE-18546 > Project: Hive > Issue Type: Bug > Components: Materialized views >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-18546.01.patch > > > HIVE-14498 introduced some code to check the invalidation of materialized > views that can be simplified, relying instead on existing transaction ids. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18546) Remove unnecessary code introduced in HIVE-14498
[ https://issues.apache.org/jira/browse/HIVE-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-18546: --- Attachment: HIVE-18546.patch > Remove unnecessary code introduced in HIVE-14498 > > > Key: HIVE-18546 > URL: https://issues.apache.org/jira/browse/HIVE-18546 > Project: Hive > Issue Type: Bug > Components: Materialized views >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-18546.patch > > > HIVE-14498 introduced some code to check the invalidation of materialized > views that can be simplified, relying instead on existing transaction ids. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18546) Remove unnecessary code introduced in HIVE-14498
[ https://issues.apache.org/jira/browse/HIVE-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-18546: --- Status: Patch Available (was: In Progress) > Remove unnecessary code introduced in HIVE-14498 > > > Key: HIVE-18546 > URL: https://issues.apache.org/jira/browse/HIVE-18546 > Project: Hive > Issue Type: Bug > Components: Materialized views >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > > HIVE-14498 introduced some code to check the invalidation of materialized > views that can be simplified, relying instead on existing transaction ids. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HIVE-18546) Remove unnecessary code introduced in HIVE-14498
[ https://issues.apache.org/jira/browse/HIVE-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-18546 started by Jesus Camacho Rodriguez. -- > Remove unnecessary code introduced in HIVE-14498 > > > Key: HIVE-18546 > URL: https://issues.apache.org/jira/browse/HIVE-18546 > Project: Hive > Issue Type: Bug > Components: Materialized views >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > > HIVE-14498 introduced some code to check the invalidation of materialized > views that can be simplified, relying instead on existing transaction ids. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18546) Remove unnecessary code introduced in HIVE-14498
[ https://issues.apache.org/jira/browse/HIVE-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-18546: --- Description: HIVE-14498 introduced some code to check the invalidation of materialized views that can be simplified, relying instead on existing transaction ids. > Remove unnecessary code introduced in HIVE-14498 > > > Key: HIVE-18546 > URL: https://issues.apache.org/jira/browse/HIVE-18546 > Project: Hive > Issue Type: Bug > Components: Materialized views >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > > HIVE-14498 introduced some code to check the invalidation of materialized > views that can be simplified, relying instead on existing transaction ids. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-18546) Remove unnecessary code introduced in HIVE-14498
[ https://issues.apache.org/jira/browse/HIVE-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez reassigned HIVE-18546: -- > Remove unnecessary code introduced in HIVE-14498 > > > Key: HIVE-18546 > URL: https://issues.apache.org/jira/browse/HIVE-18546 > Project: Hive > Issue Type: Bug > Components: Materialized views >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18501) Typo in beeline code
[
https://issues.apache.org/jira/browse/HIVE-18501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16339537#comment-16339537
]
Hive QA commented on HIVE-18501:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
12s{color} | {color:red} The patch generated 6 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 8m 58s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 3f7127f |
| Default Java | 1.8.0_111 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8833/yetus/patch-asflicense-problems.txt
|
| modules | C: beeline U: beeline |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8833/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Typo in beeline code
>
>
> Key: HIVE-18501
> URL: https://issues.apache.org/jira/browse/HIVE-18501
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 3.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Trivial
> Attachments: HIVE-18501.patch, HIVE-18501.patch, HIVE-18501.patch
>
>
> [https://github.com/apache/hive/blob/master/beeline/src/java/org/apache/hive/beeline/BeeLine.java#L744]
> the string literal used here should be "silent", not "slient". There is no
> functional bug here just a silly typo.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18472) Beeline gives log4j warnings
[
https://issues.apache.org/jira/browse/HIVE-18472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16339536#comment-16339536
]
Vihang Karajgaonkar commented on HIVE-18472:
I think we should still fix the hive scripts so that atleast the warning coming
from hbase and hadoop are not seen when we issue {{hadoop version}} and hbase
commands. For the hadoop-config.sh changes we should file a HADOOP JIRA to get
that fix. In order to get rid of these warning we will need changes to both
hive and hadoop-config.sh scripts.
> Beeline gives log4j warnings
>
>
> Key: HIVE-18472
> URL: https://issues.apache.org/jira/browse/HIVE-18472
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Janaki Lahorani
>Assignee: Janaki Lahorani
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18472.1.patch, HIVE-18472.2.patch
>
>
> Starting Beeline gives the following warnings multiple times:
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/opt/cloudera/parcels/CDH-6.x-1.cdh6.x.p0.215261/jars/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/opt/cloudera/parcels/CDH-6.x-1.cdh6.x.p0.215261/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See [http://www.slf4j.org/codes.html#multiple_bindings] for an
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> ERROR StatusLogger No log4j2 configuration file found. Using default
> configuration: logging only errors to the console. Set system property
> 'org.apache.logging.log4j.simplelog.StatusLogger.level' to TRACE to show
> Log4j2 internal initialization logging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-18532) Hive Connection can't be closed if there is an error while creating session
[
https://issues.apache.org/jira/browse/HIVE-18532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Purshotam Shah reassigned HIVE-18532:
-
Assignee: Purshotam Shah
> Hive Connection can't be closed if there is an error while creating session
> ---
>
> Key: HIVE-18532
> URL: https://issues.apache.org/jira/browse/HIVE-18532
> Project: Hive
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Purshotam Shah
>Priority: Major
>
> To open a hiveconnection we have to
> {code}
> HiveConnection connection = null;
> try {
> connection = new HiveConnection(uri, new Properties());
> } finally {
> if (connection != null)
> try {
> connection.close();
> } catch (SQLException e) {
> LOGGER.error("Error closing connection, ignored.", e);
> }
> }
> {code}
> HiveConnection opens the opensTransport and tries to create a session in the
> constructor.
> If opensTransport succeed, but session creates fails because of namenode
> error or any other error, HiveConnection can't be closed.
> Since construction fails, the hiveconnection object will be null and the
> client can't close it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18498) Create tests to cover get and list index methods
[
https://issues.apache.org/jira/browse/HIVE-18498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16339520#comment-16339520
]
Hive QA commented on HIVE-18498:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12907522/HIVE-18498.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 27 failed/errored test(s), 11714 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=35)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_input_format_excludes]
(batchId=162)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[archive_partspec2]
(batchId=94)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_alter_table_exchange_partition_fail]
(batchId=94)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_part]
(batchId=94)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[ctas_noemptyfolder]
(batchId=94)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[materialized_view_authorization_create_no_grant]
(batchId=94)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[materialized_view_authorization_drop_other]
(batchId=94)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[materialized_view_no_transactional_rewrite_2]
(batchId=94)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_aggregator_error_1]
(batchId=94)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_publisher_error_1]
(batchId=94)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[subquery_notin_implicit_gby]
(batchId=94)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[truncate_bucketed_column]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=121)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=254)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=186)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=232)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=232)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=232)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8832/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8832/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8832/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 27 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12907522 - PreCommit-HIVE-Build
> Create tests to cover get and list index methods
>
>
> Key: HIVE-18498
> URL: https://issues.apache.org/jira/browse/HIVE-18498
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Marta Kuczora
>Assignee: Marta Kuczora
>Priority: Major
> Attachments: HIVE-18498.1.patch, HIVE-18498.2.patch,
> HIVE-18498.3.patch
>
>
> The following methods of IMetaStoreClient are covered in this Jira:
> {code:java}
> - Index getIndex(String, String, String)
> - List listIndexes(String, String, short)
> - List listIndexNames(String, String, short){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18478) Drop of temp table creating recycle files at CM path
[ https://issues.apache.org/jira/browse/HIVE-18478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-18478: --- Status: Patch Available (was: Open) In case of drop table, truncate table, load table etc, the table info is deleted which can cause issues during replication. To solve this, the old files are stored in the CM directory to be used by replication later. But for temporary tables, replication is not done and thus these files creation is not required. So extra checks are added to avoid creation of recycle files in case of temp tables. > Drop of temp table creating recycle files at CM path > > > Key: HIVE-18478 > URL: https://issues.apache.org/jira/browse/HIVE-18478 > Project: Hive > Issue Type: Sub-task > Components: Hive, HiveServer2 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Minor > Labels: pull-request-available > Fix For: 3.0.0 > > > Drop TEMP table operation invokes deleteDir which moves the file to $CMROOT > which is not needed as temp tables need not be replicated -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18498) Create tests to cover get and list index methods
[
https://issues.apache.org/jira/browse/HIVE-18498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16339447#comment-16339447
]
Hive QA commented on HIVE-18498:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
13s{color} | {color:red} standalone-metastore: The patch generated 3 new + 0
unchanged - 0 fixed = 3 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
12s{color} | {color:red} The patch generated 6 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 10m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 3f7127f |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8832/yetus/diff-checkstyle-standalone-metastore.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8832/yetus/patch-asflicense-problems.txt
|
| modules | C: standalone-metastore U: standalone-metastore |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8832/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Create tests to cover get and list index methods
>
>
> Key: HIVE-18498
> URL: https://issues.apache.org/jira/browse/HIVE-18498
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Marta Kuczora
>Assignee: Marta Kuczora
>Priority: Major
> Attachments: HIVE-18498.1.patch, HIVE-18498.2.patch,
> HIVE-18498.3.patch
>
>
> The following methods of IMetaStoreClient are covered in this Jira:
> {code:java}
> - Index getIndex(String, String, String)
> - List listIndexes(String, String, short)
> - List listIndexNames(String, String, short){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18541) Secure HS2 web UI with PAM
[ https://issues.apache.org/jira/browse/HIVE-18541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleksiy Sayankin updated HIVE-18541: Description: Secure HS2 web UI with PAM. Add two new properties * hive.server2.webui.use.pam * Default value: false * Description: If true, the HiveServer2 WebUI will be secured with PAM * hive.server2.webui.pam.authenticator * Default value: org.apache.hive.http.security.PamAuthenticator * Description: Class for PAM authentication was: Secure HS2 web UI with PAM. Add two new properties * hive.server2.webui.use.pam * Default value: false * Description: If true, the HiveServer2 WebUI will be secured with PAM * hive.server2.webui.pam.authenticator * Default value: org.apache.hive.http.security.PamAuthenticator * Description: Class for PAM authenticatio > Secure HS2 web UI with PAM > -- > > Key: HIVE-18541 > URL: https://issues.apache.org/jira/browse/HIVE-18541 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2 >Reporter: Oleksiy Sayankin >Assignee: Oleksiy Sayankin >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18541.1.patch > > > Secure HS2 web UI with PAM. Add two new properties > * hive.server2.webui.use.pam > * Default value: false > * Description: If true, the HiveServer2 WebUI will be secured with PAM > * hive.server2.webui.pam.authenticator > * Default value: org.apache.hive.http.security.PamAuthenticator > * Description: Class for PAM authentication -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18495) JUnit rule to enable Driver level testing
[
https://issues.apache.org/jira/browse/HIVE-18495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16339436#comment-16339436
]
Zoltan Haindrich commented on HIVE-18495:
-
test failures doesn't seem related
[~jcamachorodriguez], [~ashutoshc]: Could you please take a look?
> JUnit rule to enable Driver level testing
> -
>
> Key: HIVE-18495
> URL: https://issues.apache.org/jira/browse/HIVE-18495
> Project: Hive
> Issue Type: Sub-task
> Components: Testing Infrastructure
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-18495.01.patch, HIVE-18495.02.patch,
> HIVE-18495.03.patch
>
>
> I've tried to write a case for a sophisticated check...it worked so well that
> I've started using it and eventually created a junit rule to make it easier
> to reuse
> Currently it takes ~15-25sec to run a test case with this framework (from
> which most of the time is the launch time of all kind of stuff which are
> needed to run a driver command).
> * enable to write JUnit tests which has access to the {{IDriver}} level
> * leave out the cli-driver; it sometimes causes problems
> * write tests at the {{ql}} module
> * it should also work from the IDE without changing anything
> Note: JUnit 5 would be great for this task; but unfortunately junit5 needs
> maven-surefire 2.19.1 ; which causes all kinds of problems for hive devs
> using idea...so that's not an option.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18478) Drop of temp table creating recycle files at CM path
[ https://issues.apache.org/jira/browse/HIVE-18478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-18478: -- Labels: pull-request-available (was: ) > Drop of temp table creating recycle files at CM path > > > Key: HIVE-18478 > URL: https://issues.apache.org/jira/browse/HIVE-18478 > Project: Hive > Issue Type: Sub-task > Components: Hive, HiveServer2 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Minor > Labels: pull-request-available > Fix For: 3.0.0 > > > Drop TEMP table operation invokes deleteDir which moves the file to $CMROOT > which is not needed as temp tables need not be replicated -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18478) Drop of temp table creating recycle files at CM path
[ https://issues.apache.org/jira/browse/HIVE-18478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16339437#comment-16339437 ] ASF GitHub Bot commented on HIVE-18478: --- GitHub user maheshk114 opened a pull request: https://github.com/apache/hive/pull/298 HIVE-18478: Avoiding creation of CM recycle file in case of temp table In case of drop table, truncate table, load table etc, the table info is deleted which can cause issues during replication. To solve this, the old files are stored in the CM directory to be used by replication later. But for temporary tables, replication is not done and thus these files creation is not required. So extra checks are added to avoid creation of recycle files in case of temp tables. You can merge this pull request into a Git repository by running: $ git pull https://github.com/maheshk114/hive HIVE-18478 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hive/pull/298.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #298 commit 2a5dd87618c1a323a230eafe1906a7ad8b9e7af7 Author: Mahesh Kumar Behera Date: 2018-01-25T16:06:27Z HIVE-18478: Avoiding creation of CM recycle file in case of temp table > Drop of temp table creating recycle files at CM path > > > Key: HIVE-18478 > URL: https://issues.apache.org/jira/browse/HIVE-18478 > Project: Hive > Issue Type: Sub-task > Components: Hive, HiveServer2 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Minor > Labels: pull-request-available > Fix For: 3.0.0 > > > Drop TEMP table operation invokes deleteDir which moves the file to $CMROOT > which is not needed as temp tables need not be replicated -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18545) Add UDF to parse complex types from json
[ https://issues.apache.org/jira/browse/HIVE-18545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-18545: Attachment: HIVE-18545.02.patch > Add UDF to parse complex types from json > > > Key: HIVE-18545 > URL: https://issues.apache.org/jira/browse/HIVE-18545 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-18545.02.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18545) Add UDF to parse complex types from json
[ https://issues.apache.org/jira/browse/HIVE-18545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-18545: Attachment: (was: HIVE-18545.01.patch) > Add UDF to parse complex types from json > > > Key: HIVE-18545 > URL: https://issues.apache.org/jira/browse/HIVE-18545 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-18545.02.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
