[jira] [Commented] (HIVE-21061) CTAS query fails with IllegalStateException for empty source
[
https://issues.apache.org/jira/browse/HIVE-21061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829962#comment-16829962
]
Vaibhav Gumashta commented on HIVE-21061:
-
+1
> CTAS query fails with IllegalStateException for empty source
>
>
> Key: HIVE-21061
> URL: https://issues.apache.org/jira/browse/HIVE-21061
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.1.0
>Reporter: Riju Trivedi
>Assignee: Riju Trivedi
>Priority: Major
> Attachments: HIVE-21061.1.patch, HIVE-21061.2.patch,
> HIVE-21061.2.patch, HIVE-21061.patch
>
>
> Creating a table using CTAS from an empty source table with predicate
> condition evaluating to False
> {code}
> create table testctas1 (id int);
> create table testctas2 as select * from testctas1 where 1=2;
> {code}
> Fails with below exception:
> {code}
> Caused by: java.lang.IllegalStateException: null
> at com.google.common.base.Preconditions.checkState(Preconditions.java:159)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.verifyAndSetVectorPartDesc(Vectorizer.java:1312)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.validateInputFormatAndSchemaEvolution(Vectorizer.java:1654)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.validateAndVectorizeMapWork(Vectorizer.java:1865)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.convertMapWork(Vectorizer.java:1109)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.dispatch(Vectorizer.java:961)
> at
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.dispatch(TaskGraphWalker.java:111)
> at
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.walk(TaskGraphWalker.java:180)
> at
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.startWalking(TaskGraphWalker.java:125)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.resolve(Vectorizer.java:2442)
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeTaskPlan(TezCompiler.java:717)
> at
> org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:258)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12443)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:358)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:285)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1863)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1810)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1805)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:197)
> ... 36 more
> {code}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21586) Thrift generated cpp files for metastore do not compile
[
https://issues.apache.org/jira/browse/HIVE-21586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829956#comment-16829956
]
Hive QA commented on HIVE-21586:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
24s{color} | {color:blue} standalone-metastore/metastore-common in master has
29 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17076/dev-support/hive-personality.sh
|
| git revision | master / 0c38cee |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore/metastore-common U:
standalone-metastore/metastore-common |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17076/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Thrift generated cpp files for metastore do not compile
> ---
>
> Key: HIVE-21586
> URL: https://issues.apache.org/jira/browse/HIVE-21586
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Affects Versions: 3.1.0
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Fix For: 4.0.0, 3.2.0
>
> Attachments: HIVE-21586.01.patch, HIVE-21586.02.patch,
> HIVE-21586.03.patch, HIVE-21586.04.branch-3.patch, HIVE-21586.04.patch,
> HIVE-21586.05.patch, HIVE-21586.nothrift.patch
>
>
> The way some structs like CreationMetadata, CompactionInfo, ColumnStatistics
> are defined in hive_metastore.thrift is that these structs are used before
> they are defined. While this works for the java code which is generated, it
> does not work for the generated cpp code since Thrift does not use
> pointer/references to the forward declared classes.
> The easy fix for this would be to reorder the struct definitions in the
> hive_metastore.thrift so that they are always defined before they are used.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21061) CTAS query fails with IllegalStateException for empty source
[
https://issues.apache.org/jira/browse/HIVE-21061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829952#comment-16829952
]
Riju Trivedi commented on HIVE-21061:
-
Failed test seems to be unrelated.
> CTAS query fails with IllegalStateException for empty source
>
>
> Key: HIVE-21061
> URL: https://issues.apache.org/jira/browse/HIVE-21061
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.1.0
>Reporter: Riju Trivedi
>Assignee: Riju Trivedi
>Priority: Major
> Attachments: HIVE-21061.1.patch, HIVE-21061.2.patch,
> HIVE-21061.2.patch, HIVE-21061.patch
>
>
> Creating a table using CTAS from an empty source table with predicate
> condition evaluating to False
> {code}
> create table testctas1 (id int);
> create table testctas2 as select * from testctas1 where 1=2;
> {code}
> Fails with below exception:
> {code}
> Caused by: java.lang.IllegalStateException: null
> at com.google.common.base.Preconditions.checkState(Preconditions.java:159)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.verifyAndSetVectorPartDesc(Vectorizer.java:1312)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.validateInputFormatAndSchemaEvolution(Vectorizer.java:1654)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.validateAndVectorizeMapWork(Vectorizer.java:1865)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.convertMapWork(Vectorizer.java:1109)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.dispatch(Vectorizer.java:961)
> at
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.dispatch(TaskGraphWalker.java:111)
> at
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.walk(TaskGraphWalker.java:180)
> at
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.startWalking(TaskGraphWalker.java:125)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.resolve(Vectorizer.java:2442)
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeTaskPlan(TezCompiler.java:717)
> at
> org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:258)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12443)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:358)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:285)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1863)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1810)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1805)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:197)
> ... 36 more
> {code}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-13582) E061-07 and E061-12: Quantified Comparison Predicates
[
https://issues.apache.org/jira/browse/HIVE-13582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829943#comment-16829943
]
Hive QA commented on HIVE-13582:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12967420/HIVE-13582.4.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17075/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17075/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17075/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12967420/HIVE-13582.4.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12967420 - PreCommit-HIVE-Build
> E061-07 and E061-12: Quantified Comparison Predicates
> -
>
> Key: HIVE-13582
> URL: https://issues.apache.org/jira/browse/HIVE-13582
> Project: Hive
> Issue Type: Sub-task
>Reporter: Carter Shanklin
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-13582.1.patch, HIVE-13582.2.patch,
> HIVE-13582.3.patch, HIVE-13582.4.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This is a part of the SQL:2011 Analytics Complete Umbrella JIRA HIVE-13554.
> Quantified comparison predicates (ANY/SOME/ALL) are mandatory in the SQL
> standard. Hive should support the predicates (E061-07) and you should be able
> to use these with subqueries (E061-12)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-13582) E061-07 and E061-12: Quantified Comparison Predicates
[
https://issues.apache.org/jira/browse/HIVE-13582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829919#comment-16829919
]
Hive QA commented on HIVE-13582:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12967420/HIVE-13582.4.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 15938 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_ALL]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_ANY]
(batchId=176)
org.apache.hadoop.hive.llap.security.TestLlapSignerImpl.testSigning
(batchId=345)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17074/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17074/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17074/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12967420 - PreCommit-HIVE-Build
> E061-07 and E061-12: Quantified Comparison Predicates
> -
>
> Key: HIVE-13582
> URL: https://issues.apache.org/jira/browse/HIVE-13582
> Project: Hive
> Issue Type: Sub-task
>Reporter: Carter Shanklin
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-13582.1.patch, HIVE-13582.2.patch,
> HIVE-13582.3.patch, HIVE-13582.4.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This is a part of the SQL:2011 Analytics Complete Umbrella JIRA HIVE-13554.
> Quantified comparison predicates (ANY/SOME/ALL) are mandatory in the SQL
> standard. Hive should support the predicates (E061-07) and you should be able
> to use these with subqueries (E061-12)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21643) Fix Broken support for ISO Time with Zone in Hive UDFs
[
https://issues.apache.org/jira/browse/HIVE-21643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829915#comment-16829915
]
RAJKAMAL commented on HIVE-21643:
-
[~NavyaS] Can you please look at the patch ? Also let me know, if you need more
information for the patch.
> Fix Broken support for ISO Time with Zone in Hive UDFs
> --
>
> Key: HIVE-21643
> URL: https://issues.apache.org/jira/browse/HIVE-21643
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 3.0.0, 3.1.0, 3.1.1
>Reporter: RAJKAMAL
>Assignee: Navya Sruthi Sunkarapalli
>Priority: Major
> Labels: patch-available, pull-request-available
> Attachments: HIVE-21643.1.patch, Hive-21643.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The followings UDFs date_format and to_date used to support ISO dates with
> timezone and the support has been broken since Hive 3.x release.
> Example:
> date_format('2017-03-16T00:10:42Z', 'y')
> date_format('2017-03-16T00:10:42+01:00', 'y')
> date_format('2017-03-16T00:10:42-01:00', 'y')
> to_date('2015-04-11T01:30:45Z')
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-13582) E061-07 and E061-12: Quantified Comparison Predicates
[
https://issues.apache.org/jira/browse/HIVE-13582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829895#comment-16829895
]
Hive QA commented on HIVE-13582:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
1s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
57s{color} | {color:blue} ql in master has 2253 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
44s{color} | {color:red} ql: The patch generated 29 new + 462 unchanged - 24
fixed = 491 total (was 486) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 10 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 8 line(s) with tabs. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
12s{color} | {color:red} ql generated 14 new + 2244 unchanged - 9 fixed = 2258
total (was 2253) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 25s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Switch statement found in
org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveSubQueryRemoveRule.rewriteInExists(RexSubQuery,
Set, RelOptUtil$Logic, HiveSubQRemoveRelBuilder, int, boolean) where one case
falls through to the next case At HiveSubQueryRemoveRule.java:Set,
RelOptUtil$Logic, HiveSubQRemoveRelBuilder, int, boolean) where one case falls
through to the next case At HiveSubQueryRemoveRule.java:[lines 460-463] |
| | Switch statement found in
org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveSubQueryRemoveRule.rewriteInExists(RexSubQuery,
Set, RelOptUtil$Logic, HiveSubQRemoveRelBuilder, int, boolean) where default
case is missing At HiveSubQueryRemoveRule.java:Set, RelOptUtil$Logic,
HiveSubQRemoveRelBuilder, int, boolean) where default case is missing At
HiveSubQueryRemoveRule.java:[lines 356-378] |
| | Dead store to stream_retval in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceSimilarExpressionQuantifierPredicate(CommonTree)
At
HiveParser_IdentifiersParser.java:org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceSimilarExpressionQuantifierPredicate(CommonTree)
At HiveParser_IdentifiersParser.java:[line 9822] |
| | Dead store to stream_retval in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.quantifierType()
At
HiveParser_IdentifiersParser.java:org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.quantifierType()
At HiveParser_IdentifiersParser.java:[line 9940] |
| | Redundant nullcheck of nonReserved311, which is known to be non-null in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.identifier()
Redundant null check at HiveParser_IdentifiersParser.java:is known to be
[jira] [Commented] (HIVE-21598) CTAS on ACID table during incremental does not replicate data
[
https://issues.apache.org/jira/browse/HIVE-21598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829889#comment-16829889
]
Ashutosh Bapat commented on HIVE-21598:
---
[~NavyaS], this has been already solved. The patch needs to be reviewed and
committed. [~maheshk114] is reviewing it and will commit it, if he finds it
committable.
> CTAS on ACID table during incremental does not replicate data
> -
>
> Key: HIVE-21598
> URL: https://issues.apache.org/jira/browse/HIVE-21598
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, repl
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: CTAS, Repl
> Attachments: HIVE-21598.01.patch, HIVE-21598.02.patch,
> HIVE-21598.03.patch, HIVE-21598.04.patch, HIVE-21598.05.patch
>
>
> Scenario
> create database dumpdb with dbproperties('repl.source.for'='1,2,3');
> use dumpdb;
> create table t1 (id int) clustered by(id) into 3 buckets stored as orc
> tblproperties ("transactional"="true");
> insert into t1 values(1);
> insert into t1 values(2);
> repl dump dumpdb;
> repl load loaddb from ;
> use loaddb;
> select * from t1;
> ++
> | t6.id |
> ++
> | 1 |
> | 2 |
> +
> use dumpdb;
> create table t6 stored as orc tblproperties ("transactional"="true") as
> select * from t1;
> select * from t6;
> ++
> | t6.id |
> ++
> | 1 |
> | 2 |
> ++
> repl dump dumpdb from
> repl load loaddb from ;
> use loaddb;
> select * from t6;
> ++
> | t6.id |
> ++
> ++
> t6 gets created but there's no data.
>
> On further investigation, I see that the CommitTxnEvent's dump directory has
> _files but it is empty. Looks like we do not log names of the files created
> as part of CTAS.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-21598) CTAS on ACID table during incremental does not replicate data
[
https://issues.apache.org/jira/browse/HIVE-21598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Bapat reassigned HIVE-21598:
-
Assignee: Ashutosh Bapat (was: Navya Sruthi Sunkarapalli)
> CTAS on ACID table during incremental does not replicate data
> -
>
> Key: HIVE-21598
> URL: https://issues.apache.org/jira/browse/HIVE-21598
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, repl
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: CTAS, Repl
> Attachments: HIVE-21598.01.patch, HIVE-21598.02.patch,
> HIVE-21598.03.patch, HIVE-21598.04.patch, HIVE-21598.05.patch
>
>
> Scenario
> create database dumpdb with dbproperties('repl.source.for'='1,2,3');
> use dumpdb;
> create table t1 (id int) clustered by(id) into 3 buckets stored as orc
> tblproperties ("transactional"="true");
> insert into t1 values(1);
> insert into t1 values(2);
> repl dump dumpdb;
> repl load loaddb from ;
> use loaddb;
> select * from t1;
> ++
> | t6.id |
> ++
> | 1 |
> | 2 |
> +
> use dumpdb;
> create table t6 stored as orc tblproperties ("transactional"="true") as
> select * from t1;
> select * from t6;
> ++
> | t6.id |
> ++
> | 1 |
> | 2 |
> ++
> repl dump dumpdb from
> repl load loaddb from ;
> use loaddb;
> select * from t6;
> ++
> | t6.id |
> ++
> ++
> t6 gets created but there's no data.
>
> On further investigation, I see that the CommitTxnEvent's dump directory has
> _files but it is empty. Looks like we do not log names of the files created
> as part of CTAS.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21586) Thrift generated cpp files for metastore do not compile
[
https://issues.apache.org/jira/browse/HIVE-21586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829881#comment-16829881
]
Hive QA commented on HIVE-21586:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12967415/HIVE-21586.04.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15934 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17073/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17073/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17073/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12967415 - PreCommit-HIVE-Build
> Thrift generated cpp files for metastore do not compile
> ---
>
> Key: HIVE-21586
> URL: https://issues.apache.org/jira/browse/HIVE-21586
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Affects Versions: 3.1.0
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Fix For: 4.0.0, 3.2.0
>
> Attachments: HIVE-21586.01.patch, HIVE-21586.02.patch,
> HIVE-21586.03.patch, HIVE-21586.04.branch-3.patch, HIVE-21586.04.patch,
> HIVE-21586.05.patch, HIVE-21586.nothrift.patch
>
>
> The way some structs like CreationMetadata, CompactionInfo, ColumnStatistics
> are defined in hive_metastore.thrift is that these structs are used before
> they are defined. While this works for the java code which is generated, it
> does not work for the generated cpp code since Thrift does not use
> pointer/references to the forward declared classes.
> The easy fix for this would be to reorder the struct definitions in the
> hive_metastore.thrift so that they are always defined before they are used.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-21586) Thrift generated cpp files for metastore do not compile
[
https://issues.apache.org/jira/browse/HIVE-21586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829850#comment-16829850
]
Vihang Karajgaonkar edited comment on HIVE-21586 at 4/30/19 1:35 AM:
-
Looks like I missed moving about {{ColumnStatistics}} Reopening this jira. Will
attach a addendum soon
was (Author: vihangk1):
Looks like I missed moving about {{ColumnStatistics}} and {{ReplLastIdInfo}}
Reopening this jira. Will attach a addendum soon
> Thrift generated cpp files for metastore do not compile
> ---
>
> Key: HIVE-21586
> URL: https://issues.apache.org/jira/browse/HIVE-21586
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Affects Versions: 3.1.0
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Fix For: 4.0.0, 3.2.0
>
> Attachments: HIVE-21586.01.patch, HIVE-21586.02.patch,
> HIVE-21586.03.patch, HIVE-21586.04.branch-3.patch, HIVE-21586.04.patch,
> HIVE-21586.05.patch, HIVE-21586.nothrift.patch
>
>
> The way some structs like CreationMetadata, CompactionInfo, ColumnStatistics
> are defined in hive_metastore.thrift is that these structs are used before
> they are defined. While this works for the java code which is generated, it
> does not work for the generated cpp code since Thrift does not use
> pointer/references to the forward declared classes.
> The easy fix for this would be to reorder the struct definitions in the
> hive_metastore.thrift so that they are always defined before they are used.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-21586) Thrift generated cpp files for metastore do not compile
[
https://issues.apache.org/jira/browse/HIVE-21586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829850#comment-16829850
]
Vihang Karajgaonkar edited comment on HIVE-21586 at 4/30/19 1:30 AM:
-
Looks like I missed moving about {{ColumnStatistics}} and {{ReplLastIdInfo}}
Reopening this jira. Will attach a addendum soon
was (Author: vihangk1):
Looks like I missed moving about {{ColumnStatistics}} Reopening this jira. Will
attach a addendum soon
> Thrift generated cpp files for metastore do not compile
> ---
>
> Key: HIVE-21586
> URL: https://issues.apache.org/jira/browse/HIVE-21586
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Affects Versions: 3.1.0
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Fix For: 4.0.0, 3.2.0
>
> Attachments: HIVE-21586.01.patch, HIVE-21586.02.patch,
> HIVE-21586.03.patch, HIVE-21586.04.branch-3.patch, HIVE-21586.04.patch,
> HIVE-21586.05.patch, HIVE-21586.nothrift.patch
>
>
> The way some structs like CreationMetadata, CompactionInfo, ColumnStatistics
> are defined in hive_metastore.thrift is that these structs are used before
> they are defined. While this works for the java code which is generated, it
> does not work for the generated cpp code since Thrift does not use
> pointer/references to the forward declared classes.
> The easy fix for this would be to reorder the struct definitions in the
> hive_metastore.thrift so that they are always defined before they are used.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21586) Thrift generated cpp files for metastore do not compile
[ https://issues.apache.org/jira/browse/HIVE-21586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar updated HIVE-21586: --- Attachment: HIVE-21586.05.patch > Thrift generated cpp files for metastore do not compile > --- > > Key: HIVE-21586 > URL: https://issues.apache.org/jira/browse/HIVE-21586 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Affects Versions: 3.1.0 >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-21586.01.patch, HIVE-21586.02.patch, > HIVE-21586.03.patch, HIVE-21586.04.branch-3.patch, HIVE-21586.04.patch, > HIVE-21586.05.patch, HIVE-21586.nothrift.patch > > > The way some structs like CreationMetadata, CompactionInfo, ColumnStatistics > are defined in hive_metastore.thrift is that these structs are used before > they are defined. While this works for the java code which is generated, it > does not work for the generated cpp code since Thrift does not use > pointer/references to the forward declared classes. > The easy fix for this would be to reorder the struct definitions in the > hive_metastore.thrift so that they are always defined before they are used. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21586) Thrift generated cpp files for metastore do not compile
[
https://issues.apache.org/jira/browse/HIVE-21586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829872#comment-16829872
]
Hive QA commented on HIVE-21586:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
35s{color} | {color:blue} standalone-metastore/metastore-common in master has
29 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 24s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17073/dev-support/hive-personality.sh
|
| git revision | master / 0c38cee |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore/metastore-common U:
standalone-metastore/metastore-common |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17073/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Thrift generated cpp files for metastore do not compile
> ---
>
> Key: HIVE-21586
> URL: https://issues.apache.org/jira/browse/HIVE-21586
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Affects Versions: 3.1.0
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Fix For: 4.0.0, 3.2.0
>
> Attachments: HIVE-21586.01.patch, HIVE-21586.02.patch,
> HIVE-21586.03.patch, HIVE-21586.04.branch-3.patch, HIVE-21586.04.patch,
> HIVE-21586.nothrift.patch
>
>
> The way some structs like CreationMetadata, CompactionInfo, ColumnStatistics
> are defined in hive_metastore.thrift is that these structs are used before
> they are defined. While this works for the java code which is generated, it
> does not work for the generated cpp code since Thrift does not use
> pointer/references to the forward declared classes.
> The easy fix for this would be to reorder the struct definitions in the
> hive_metastore.thrift so that they are always defined before they are used.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-13582) E061-07 and E061-12: Quantified Comparison Predicates
[ https://issues.apache.org/jira/browse/HIVE-13582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-13582: -- Labels: pull-request-available (was: ) > E061-07 and E061-12: Quantified Comparison Predicates > - > > Key: HIVE-13582 > URL: https://issues.apache.org/jira/browse/HIVE-13582 > Project: Hive > Issue Type: Sub-task >Reporter: Carter Shanklin >Assignee: Vineet Garg >Priority: Major > Labels: pull-request-available > Attachments: HIVE-13582.1.patch, HIVE-13582.2.patch, > HIVE-13582.3.patch, HIVE-13582.4.patch > > > This is a part of the SQL:2011 Analytics Complete Umbrella JIRA HIVE-13554. > Quantified comparison predicates (ANY/SOME/ALL) are mandatory in the SQL > standard. Hive should support the predicates (E061-07) and you should be able > to use these with subqueries (E061-12) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-13582) E061-07 and E061-12: Quantified Comparison Predicates
[ https://issues.apache.org/jira/browse/HIVE-13582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829869#comment-16829869 ] Vineet Garg commented on HIVE-13582: Pull request: https://github.com/apache/hive/pull/612 > E061-07 and E061-12: Quantified Comparison Predicates > - > > Key: HIVE-13582 > URL: https://issues.apache.org/jira/browse/HIVE-13582 > Project: Hive > Issue Type: Sub-task >Reporter: Carter Shanklin >Assignee: Vineet Garg >Priority: Major > Labels: pull-request-available > Attachments: HIVE-13582.1.patch, HIVE-13582.2.patch, > HIVE-13582.3.patch, HIVE-13582.4.patch > > Time Spent: 10m > Remaining Estimate: 0h > > This is a part of the SQL:2011 Analytics Complete Umbrella JIRA HIVE-13554. > Quantified comparison predicates (ANY/SOME/ALL) are mandatory in the SQL > standard. Hive should support the predicates (E061-07) and you should be able > to use these with subqueries (E061-12) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (HIVE-13582) E061-07 and E061-12: Quantified Comparison Predicates
[ https://issues.apache.org/jira/browse/HIVE-13582?focusedWorklogId=235019=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-235019 ] ASF GitHub Bot logged work on HIVE-13582: - Author: ASF GitHub Bot Created on: 30/Apr/19 01:06 Start Date: 30/Apr/19 01:06 Worklog Time Spent: 10m Work Description: vineetgarg02 commented on pull request #612: HIVE-13582: Support for Quantified predicates for non-correlated queries URL: https://github.com/apache/hive/pull/612 This adds the support for Quantified predicates (ANY/SOME, ALL) in WHERE and SELECT. Note that `<>ANY` and `=ALL` is not yet supported. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 235019) Time Spent: 10m Remaining Estimate: 0h > E061-07 and E061-12: Quantified Comparison Predicates > - > > Key: HIVE-13582 > URL: https://issues.apache.org/jira/browse/HIVE-13582 > Project: Hive > Issue Type: Sub-task >Reporter: Carter Shanklin >Assignee: Vineet Garg >Priority: Major > Labels: pull-request-available > Attachments: HIVE-13582.1.patch, HIVE-13582.2.patch, > HIVE-13582.3.patch, HIVE-13582.4.patch > > Time Spent: 10m > Remaining Estimate: 0h > > This is a part of the SQL:2011 Analytics Complete Umbrella JIRA HIVE-13554. > Quantified comparison predicates (ANY/SOME/ALL) are mandatory in the SQL > standard. Hive should support the predicates (E061-07) and you should be able > to use these with subqueries (E061-12) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-13582) E061-07 and E061-12: Quantified Comparison Predicates
[ https://issues.apache.org/jira/browse/HIVE-13582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829868#comment-16829868 ] Vineet Garg commented on HIVE-13582: [~ashutoshc] [~jcamachorodriguez] Can you take a look? > E061-07 and E061-12: Quantified Comparison Predicates > - > > Key: HIVE-13582 > URL: https://issues.apache.org/jira/browse/HIVE-13582 > Project: Hive > Issue Type: Sub-task >Reporter: Carter Shanklin >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-13582.1.patch, HIVE-13582.2.patch, > HIVE-13582.3.patch, HIVE-13582.4.patch > > > This is a part of the SQL:2011 Analytics Complete Umbrella JIRA HIVE-13554. > Quantified comparison predicates (ANY/SOME/ALL) are mandatory in the SQL > standard. Hive should support the predicates (E061-07) and you should be able > to use these with subqueries (E061-12) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-13582) E061-07 and E061-12: Quantified Comparison Predicates
[ https://issues.apache.org/jira/browse/HIVE-13582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-13582: --- Attachment: HIVE-13582.4.patch > E061-07 and E061-12: Quantified Comparison Predicates > - > > Key: HIVE-13582 > URL: https://issues.apache.org/jira/browse/HIVE-13582 > Project: Hive > Issue Type: Sub-task >Reporter: Carter Shanklin >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-13582.1.patch, HIVE-13582.2.patch, > HIVE-13582.3.patch, HIVE-13582.4.patch > > > This is a part of the SQL:2011 Analytics Complete Umbrella JIRA HIVE-13554. > Quantified comparison predicates (ANY/SOME/ALL) are mandatory in the SQL > standard. Hive should support the predicates (E061-07) and you should be able > to use these with subqueries (E061-12) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-13582) E061-07 and E061-12: Quantified Comparison Predicates
[ https://issues.apache.org/jira/browse/HIVE-13582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-13582: --- Attachment: (was: HIVE-13582.4.patch) > E061-07 and E061-12: Quantified Comparison Predicates > - > > Key: HIVE-13582 > URL: https://issues.apache.org/jira/browse/HIVE-13582 > Project: Hive > Issue Type: Sub-task >Reporter: Carter Shanklin >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-13582.1.patch, HIVE-13582.2.patch, > HIVE-13582.3.patch, HIVE-13582.4.patch > > > This is a part of the SQL:2011 Analytics Complete Umbrella JIRA HIVE-13554. > Quantified comparison predicates (ANY/SOME/ALL) are mandatory in the SQL > standard. Hive should support the predicates (E061-07) and you should be able > to use these with subqueries (E061-12) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-13582) E061-07 and E061-12: Quantified Comparison Predicates
[ https://issues.apache.org/jira/browse/HIVE-13582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-13582: --- Status: Patch Available (was: Open) > E061-07 and E061-12: Quantified Comparison Predicates > - > > Key: HIVE-13582 > URL: https://issues.apache.org/jira/browse/HIVE-13582 > Project: Hive > Issue Type: Sub-task >Reporter: Carter Shanklin >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-13582.1.patch, HIVE-13582.2.patch, > HIVE-13582.3.patch, HIVE-13582.4.patch > > > This is a part of the SQL:2011 Analytics Complete Umbrella JIRA HIVE-13554. > Quantified comparison predicates (ANY/SOME/ALL) are mandatory in the SQL > standard. Hive should support the predicates (E061-07) and you should be able > to use these with subqueries (E061-12) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-13582) E061-07 and E061-12: Quantified Comparison Predicates
[ https://issues.apache.org/jira/browse/HIVE-13582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-13582: --- Attachment: HIVE-13582.4.patch > E061-07 and E061-12: Quantified Comparison Predicates > - > > Key: HIVE-13582 > URL: https://issues.apache.org/jira/browse/HIVE-13582 > Project: Hive > Issue Type: Sub-task >Reporter: Carter Shanklin >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-13582.1.patch, HIVE-13582.2.patch, > HIVE-13582.3.patch, HIVE-13582.4.patch > > > This is a part of the SQL:2011 Analytics Complete Umbrella JIRA HIVE-13554. > Quantified comparison predicates (ANY/SOME/ALL) are mandatory in the SQL > standard. Hive should support the predicates (E061-07) and you should be able > to use these with subqueries (E061-12) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-13582) E061-07 and E061-12: Quantified Comparison Predicates
[ https://issues.apache.org/jira/browse/HIVE-13582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-13582: --- Attachment: (was: HIVE-13582.4.patch) > E061-07 and E061-12: Quantified Comparison Predicates > - > > Key: HIVE-13582 > URL: https://issues.apache.org/jira/browse/HIVE-13582 > Project: Hive > Issue Type: Sub-task >Reporter: Carter Shanklin >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-13582.1.patch, HIVE-13582.2.patch, > HIVE-13582.3.patch, HIVE-13582.4.patch > > > This is a part of the SQL:2011 Analytics Complete Umbrella JIRA HIVE-13554. > Quantified comparison predicates (ANY/SOME/ALL) are mandatory in the SQL > standard. Hive should support the predicates (E061-07) and you should be able > to use these with subqueries (E061-12) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-13582) E061-07 and E061-12: Quantified Comparison Predicates
[ https://issues.apache.org/jira/browse/HIVE-13582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-13582: --- Status: Open (was: Patch Available) > E061-07 and E061-12: Quantified Comparison Predicates > - > > Key: HIVE-13582 > URL: https://issues.apache.org/jira/browse/HIVE-13582 > Project: Hive > Issue Type: Sub-task >Reporter: Carter Shanklin >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-13582.1.patch, HIVE-13582.2.patch, > HIVE-13582.3.patch, HIVE-13582.4.patch > > > This is a part of the SQL:2011 Analytics Complete Umbrella JIRA HIVE-13554. > Quantified comparison predicates (ANY/SOME/ALL) are mandatory in the SQL > standard. Hive should support the predicates (E061-07) and you should be able > to use these with subqueries (E061-12) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-13582) E061-07 and E061-12: Quantified Comparison Predicates
[ https://issues.apache.org/jira/browse/HIVE-13582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-13582: --- Status: Patch Available (was: Open) > E061-07 and E061-12: Quantified Comparison Predicates > - > > Key: HIVE-13582 > URL: https://issues.apache.org/jira/browse/HIVE-13582 > Project: Hive > Issue Type: Sub-task >Reporter: Carter Shanklin >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-13582.1.patch, HIVE-13582.2.patch, > HIVE-13582.3.patch, HIVE-13582.4.patch > > > This is a part of the SQL:2011 Analytics Complete Umbrella JIRA HIVE-13554. > Quantified comparison predicates (ANY/SOME/ALL) are mandatory in the SQL > standard. Hive should support the predicates (E061-07) and you should be able > to use these with subqueries (E061-12) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-13582) E061-07 and E061-12: Quantified Comparison Predicates
[ https://issues.apache.org/jira/browse/HIVE-13582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-13582: --- Status: Open (was: Patch Available) > E061-07 and E061-12: Quantified Comparison Predicates > - > > Key: HIVE-13582 > URL: https://issues.apache.org/jira/browse/HIVE-13582 > Project: Hive > Issue Type: Sub-task >Reporter: Carter Shanklin >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-13582.1.patch, HIVE-13582.2.patch, > HIVE-13582.3.patch, HIVE-13582.4.patch > > > This is a part of the SQL:2011 Analytics Complete Umbrella JIRA HIVE-13554. > Quantified comparison predicates (ANY/SOME/ALL) are mandatory in the SQL > standard. Hive should support the predicates (E061-07) and you should be able > to use these with subqueries (E061-12) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-13582) E061-07 and E061-12: Quantified Comparison Predicates
[ https://issues.apache.org/jira/browse/HIVE-13582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-13582: --- Attachment: HIVE-13582.4.patch > E061-07 and E061-12: Quantified Comparison Predicates > - > > Key: HIVE-13582 > URL: https://issues.apache.org/jira/browse/HIVE-13582 > Project: Hive > Issue Type: Sub-task >Reporter: Carter Shanklin >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-13582.1.patch, HIVE-13582.2.patch, > HIVE-13582.3.patch, HIVE-13582.4.patch > > > This is a part of the SQL:2011 Analytics Complete Umbrella JIRA HIVE-13554. > Quantified comparison predicates (ANY/SOME/ALL) are mandatory in the SQL > standard. Hive should support the predicates (E061-07) and you should be able > to use these with subqueries (E061-12) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21586) Thrift generated cpp files for metastore do not compile
[ https://issues.apache.org/jira/browse/HIVE-21586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar updated HIVE-21586: --- Status: Patch Available (was: Reopened) > Thrift generated cpp files for metastore do not compile > --- > > Key: HIVE-21586 > URL: https://issues.apache.org/jira/browse/HIVE-21586 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Affects Versions: 3.1.0 >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-21586.01.patch, HIVE-21586.02.patch, > HIVE-21586.03.patch, HIVE-21586.04.branch-3.patch, HIVE-21586.04.patch, > HIVE-21586.nothrift.patch > > > The way some structs like CreationMetadata, CompactionInfo, ColumnStatistics > are defined in hive_metastore.thrift is that these structs are used before > they are defined. While this works for the java code which is generated, it > does not work for the generated cpp code since Thrift does not use > pointer/references to the forward declared classes. > The easy fix for this would be to reorder the struct definitions in the > hive_metastore.thrift so that they are always defined before they are used. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21586) Thrift generated cpp files for metastore do not compile
[ https://issues.apache.org/jira/browse/HIVE-21586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar updated HIVE-21586: --- Attachment: (was: HIVE-21586.03.patch) > Thrift generated cpp files for metastore do not compile > --- > > Key: HIVE-21586 > URL: https://issues.apache.org/jira/browse/HIVE-21586 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Affects Versions: 3.1.0 >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-21586.01.patch, HIVE-21586.02.patch, > HIVE-21586.03.patch, HIVE-21586.04.branch-3.patch, HIVE-21586.04.patch, > HIVE-21586.nothrift.patch > > > The way some structs like CreationMetadata, CompactionInfo, ColumnStatistics > are defined in hive_metastore.thrift is that these structs are used before > they are defined. While this works for the java code which is generated, it > does not work for the generated cpp code since Thrift does not use > pointer/references to the forward declared classes. > The easy fix for this would be to reorder the struct definitions in the > hive_metastore.thrift so that they are always defined before they are used. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21586) Thrift generated cpp files for metastore do not compile
[ https://issues.apache.org/jira/browse/HIVE-21586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar updated HIVE-21586: --- Attachment: HIVE-21586.03.patch > Thrift generated cpp files for metastore do not compile > --- > > Key: HIVE-21586 > URL: https://issues.apache.org/jira/browse/HIVE-21586 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Affects Versions: 3.1.0 >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-21586.01.patch, HIVE-21586.02.patch, > HIVE-21586.03.patch, HIVE-21586.04.branch-3.patch, HIVE-21586.04.patch, > HIVE-21586.nothrift.patch > > > The way some structs like CreationMetadata, CompactionInfo, ColumnStatistics > are defined in hive_metastore.thrift is that these structs are used before > they are defined. While this works for the java code which is generated, it > does not work for the generated cpp code since Thrift does not use > pointer/references to the forward declared classes. > The easy fix for this would be to reorder the struct definitions in the > hive_metastore.thrift so that they are always defined before they are used. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21586) Thrift generated cpp files for metastore do not compile
[ https://issues.apache.org/jira/browse/HIVE-21586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar updated HIVE-21586: --- Attachment: HIVE-21586.04.patch > Thrift generated cpp files for metastore do not compile > --- > > Key: HIVE-21586 > URL: https://issues.apache.org/jira/browse/HIVE-21586 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Affects Versions: 3.1.0 >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-21586.01.patch, HIVE-21586.02.patch, > HIVE-21586.03.patch, HIVE-21586.04.branch-3.patch, HIVE-21586.04.patch, > HIVE-21586.nothrift.patch > > > The way some structs like CreationMetadata, CompactionInfo, ColumnStatistics > are defined in hive_metastore.thrift is that these structs are used before > they are defined. While this works for the java code which is generated, it > does not work for the generated cpp code since Thrift does not use > pointer/references to the forward declared classes. > The easy fix for this would be to reorder the struct definitions in the > hive_metastore.thrift so that they are always defined before they are used. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (HIVE-21586) Thrift generated cpp files for metastore do not compile
[
https://issues.apache.org/jira/browse/HIVE-21586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vihang Karajgaonkar reopened HIVE-21586:
Looks like I missed moving about {{ColumnStatistics}} Reopening this jira. Will
attach a addendum soon
> Thrift generated cpp files for metastore do not compile
> ---
>
> Key: HIVE-21586
> URL: https://issues.apache.org/jira/browse/HIVE-21586
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Affects Versions: 3.1.0
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Fix For: 4.0.0, 3.2.0
>
> Attachments: HIVE-21586.01.patch, HIVE-21586.02.patch,
> HIVE-21586.03.patch, HIVE-21586.04.branch-3.patch, HIVE-21586.nothrift.patch
>
>
> The way some structs like CreationMetadata, CompactionInfo, ColumnStatistics
> are defined in hive_metastore.thrift is that these structs are used before
> they are defined. While this works for the java code which is generated, it
> does not work for the generated cpp code since Thrift does not use
> pointer/references to the forward declared classes.
> The easy fix for this would be to reorder the struct definitions in the
> hive_metastore.thrift so that they are always defined before they are used.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (HIVE-21656) Vectorize UDF mask
[ https://issues.apache.org/jira/browse/HIVE-21656?focusedWorklogId=234988=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-234988 ] ASF GitHub Bot logged work on HIVE-21656: - Author: ASF GitHub Bot Created on: 29/Apr/19 23:51 Start Date: 29/Apr/19 23:51 Worklog Time Spent: 10m Work Description: t3rmin4t0r commented on pull request #611: HIVE-21656: Vectorized Mask UDF URL: https://github.com/apache/hive/pull/611#discussion_r279579554 ## File path: ql/src/test/queries/clientpositive/udf_mask_vectorized.q ## @@ -0,0 +1,38 @@ +set hive.mapred.mode=nonstrict; +set hive.explain.user=false; +SET hive.vectorized.execution.enabled=true; +set hive.fetch.task.conversion=minimal; + +create table tmask(t tinyint, s smallint, i int, bg bigint, f float, db double, dc decimal (10,3), n numeric, + d date, ts timestamp, + str string, vr varchar(10), ch char(4), + b boolean, bin binary); + +insert into tmask values(1,2,345,4455433,5.6,5644.455,10.20, 579.00, '2019-09-09', current_timestamp(), 'string1', 'varchar1', 'ch1', true, 'bin'), + (NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL), + (9,7,3450,7455433,5.08,5944.455,10.20, 579.00, '1019-09-09', current_timestamp(), 'string2', 'varchar2', 'ch2', false, 'bin2'), + (NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL); + +-- MASK UDF with single argument +-- date types (timestamp is not supported for masking) +set hive.vectorized.execution.enabled=true; +explain VECTORIZATION DETAIL select mask(d), mask(ts) from tmask where s > 0 and i < 1000; +select mask(d), mask(ts) from tmask where s > 0 and i < 1000; +set hive.vectorized.execution.enabled=false; +select mask(d), mask(ts) from tmask where s > 0 and i < 1000; + +-- numeric types, double, float, demical and numeric are not supported for masking +set hive.vectorized.execution.enabled=true; +explain VECTORIZATION DETAIL select mask(t), mask(s), mask(i), mask(bg), mask(f), mask(db), mask(dc), mask(n) from tmask; +select mask(t), mask(s), mask(i), mask(bg), mask(f), mask(db), mask(dc), mask(n) from tmask; +set hive.vectorized.execution.enabled=false; +select mask(t), mask(s), mask(i), mask(bg), mask(f), mask(db), mask(dc), mask(n) from tmask; + +-- string + misc types +set hive.vectorized.execution.enabled=true; +explain VECTORIZATION DETAIL select mask(str), mask(vr), mask(ch), mask(b), mask(bin) from tmask ; +select mask(str), mask(vr), mask(ch), mask(b), mask(bin) from tmask ; +set hive.vectorized.execution.enabled=false; +select mask(str), mask(vr), mask(ch), mask(b), mask(bin) from tmask ; Review comment: Add test to compare the two results - instead of golden files This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 234988) Time Spent: 0.5h (was: 20m) > Vectorize UDF mask > -- > > Key: HIVE-21656 > URL: https://issues.apache.org/jira/browse/HIVE-21656 > Project: Hive > Issue Type: Improvement > Components: Vectorization >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21656.1.patch, HIVE-21656.2.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (HIVE-21656) Vectorize UDF mask
[
https://issues.apache.org/jira/browse/HIVE-21656?focusedWorklogId=234987=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-234987
]
ASF GitHub Bot logged work on HIVE-21656:
-
Author: ASF GitHub Bot
Created on: 29/Apr/19 23:48
Start Date: 29/Apr/19 23:48
Worklog Time Spent: 10m
Work Description: t3rmin4t0r commented on pull request #611: HIVE-21656:
Vectorized Mask UDF
URL: https://github.com/apache/hive/pull/611#discussion_r279579121
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/UDFMaskVectorLong.java
##
@@ -0,0 +1,89 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.exec.vector.expressions;
+
+import org.apache.hadoop.hive.ql.exec.vector.ColumnVector;
+import org.apache.hadoop.hive.ql.exec.vector.LongColumnVector;
+import org.apache.hadoop.hive.ql.exec.vector.VectorExpressionDescriptor;
+import org.apache.hadoop.hive.serde.serdeConstants;
+
+public class UDFMaskVectorLong extends UDFMaskBaseVector {
+
+ public UDFMaskVectorLong(int colNum, int outputColumnNum) {
+super(colNum, outputColumnNum);
+ }
+
+ public UDFMaskVectorLong() {
+super();
+ }
+
+ public void transform(ColumnVector outputColVector, ColumnVector
inputVector, int idx) {
+boolean isBooleanType =
+
this.getInputTypeInfos()[0].getTypeName().equals(serdeConstants.BOOLEAN_TYPE_NAME);
+
+// boolean data type is represented as long and masking is not supported
for boolean
+if(inputVector.isNull[idx] || isBooleanType) {
Review comment:
noNulls checking
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 234987)
Time Spent: 20m (was: 10m)
> Vectorize UDF mask
> --
>
> Key: HIVE-21656
> URL: https://issues.apache.org/jira/browse/HIVE-21656
> Project: Hive
> Issue Type: Improvement
> Components: Vectorization
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21656.1.patch, HIVE-21656.2.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21665) Unable to reconstruct valid SQL query from AST when back ticks are used
[
https://issues.apache.org/jira/browse/HIVE-21665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829831#comment-16829831
]
slim bouguerra commented on HIVE-21665:
---
cc [~vgarg] / [~jdere]
> Unable to reconstruct valid SQL query from AST when back ticks are used
> ---
>
> Key: HIVE-21665
> URL: https://issues.apache.org/jira/browse/HIVE-21665
> Project: Hive
> Issue Type: Bug
>Reporter: slim bouguerra
>Priority: Major
>
> Hive-6013 have introduced a parser rule that removes all the
> {code:java}
> `{code}
> from identifiers or query alias, this can result in some issue when we need
> to reconstruct the actual SQL query from the AST.
> To reproduce the bug you can use explain analyze statement as the following
> query
> {code:java}
> explain analyze select 'literal' as `alias with space`;
> {code}
> This bugs will affect how Ranger plugin and probably result cache, where in
> both places we need to reconstruct the query from AST.
> The current work around is to avoid white spaces within Aliases.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21665) Unable to reconstruct valid SQL query from AST when back ticks are used
[
https://issues.apache.org/jira/browse/HIVE-21665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
slim bouguerra updated HIVE-21665:
--
Affects Version/s: 4.0.0
> Unable to reconstruct valid SQL query from AST when back ticks are used
> ---
>
> Key: HIVE-21665
> URL: https://issues.apache.org/jira/browse/HIVE-21665
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: slim bouguerra
>Priority: Major
>
> Hive-6013 have introduced a parser rule that removes all the
> {code:java}
> `{code}
> from identifiers or query alias, this can result in some issue when we need
> to reconstruct the actual SQL query from the AST.
> To reproduce the bug you can use explain analyze statement as the following
> query
> {code:java}
> explain analyze select 'literal' as `alias with space`;
> {code}
> This bugs will affect how Ranger plugin and probably result cache, where in
> both places we need to reconstruct the query from AST.
> The current work around is to avoid white spaces within Aliases.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21656) Vectorize UDF mask
[
https://issues.apache.org/jira/browse/HIVE-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829794#comment-16829794
]
Hive QA commented on HIVE-21656:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12967404/HIVE-21656.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15935 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17072/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17072/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17072/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12967404 - PreCommit-HIVE-Build
> Vectorize UDF mask
> --
>
> Key: HIVE-21656
> URL: https://issues.apache.org/jira/browse/HIVE-21656
> Project: Hive
> Issue Type: Improvement
> Components: Vectorization
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21656.1.patch, HIVE-21656.2.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21656) Vectorize UDF mask
[
https://issues.apache.org/jira/browse/HIVE-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829752#comment-16829752
]
Hive QA commented on HIVE-21656:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
53s{color} | {color:blue} ql in master has 2253 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
37s{color} | {color:red} ql: The patch generated 6 new + 42 unchanged - 0 fixed
= 48 total (was 42) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
1s{color} | {color:red} The patch 6 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17072/dev-support/hive-personality.sh
|
| git revision | master / 0c38cee |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17072/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17072/yetus/whitespace-tabs.txt
|
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17072/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Vectorize UDF mask
> --
>
> Key: HIVE-21656
> URL: https://issues.apache.org/jira/browse/HIVE-21656
> Project: Hive
> Issue Type: Improvement
> Components: Vectorization
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21656.1.patch, HIVE-21656.2.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-21664) HMS Translation layer - Thrift API changes
[ https://issues.apache.org/jira/browse/HIVE-21664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam reassigned HIVE-21664: > HMS Translation layer - Thrift API changes > -- > > Key: HIVE-21664 > URL: https://issues.apache.org/jira/browse/HIVE-21664 > Project: Hive > Issue Type: Sub-task > Components: Standalone Metastore >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Major > > This jira is to track the HMS side changes of this feature. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21663) Hive Metastore Translation Layer
[ https://issues.apache.org/jira/browse/HIVE-21663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam updated HIVE-21663: - Attachment: HMS Translation Layer_v1.0.pdf > Hive Metastore Translation Layer > > > Key: HIVE-21663 > URL: https://issues.apache.org/jira/browse/HIVE-21663 > Project: Hive > Issue Type: New Feature > Components: Standalone Metastore >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Major > Attachments: HMS Translation Layer_v1.0.pdf > > > This task is for the implementation of the default provider for translation, > that is extensible if needed for a custom translator. Please refer the spec > for additional details on the translation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21663) Hive Metastore Translation Layer
[ https://issues.apache.org/jira/browse/HIVE-21663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam reassigned HIVE-21663: > Hive Metastore Translation Layer > > > Key: HIVE-21663 > URL: https://issues.apache.org/jira/browse/HIVE-21663 > Project: Hive > Issue Type: New Feature > Components: Standalone Metastore >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Major > > This task is for the implementation of the default provider for translation, > that is extensible if needed for a custom translator. Please refer the spec > for additional details on the translation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21656) Vectorize UDF mask
[ https://issues.apache.org/jira/browse/HIVE-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829716#comment-16829716 ] Vineet Garg commented on HIVE-21656: [~gopalv] Can you take a look? [GitHub Pull Request #611|https://github.com/apache/hive/pull/611] > Vectorize UDF mask > -- > > Key: HIVE-21656 > URL: https://issues.apache.org/jira/browse/HIVE-21656 > Project: Hive > Issue Type: Improvement > Components: Vectorization >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21656.1.patch, HIVE-21656.2.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21656) Vectorize UDF mask
[ https://issues.apache.org/jira/browse/HIVE-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-21656: --- Attachment: HIVE-21656.2.patch > Vectorize UDF mask > -- > > Key: HIVE-21656 > URL: https://issues.apache.org/jira/browse/HIVE-21656 > Project: Hive > Issue Type: Improvement > Components: Vectorization >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-21656.1.patch, HIVE-21656.2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21656) Vectorize UDF mask
[ https://issues.apache.org/jira/browse/HIVE-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-21656: -- Labels: pull-request-available (was: ) > Vectorize UDF mask > -- > > Key: HIVE-21656 > URL: https://issues.apache.org/jira/browse/HIVE-21656 > Project: Hive > Issue Type: Improvement > Components: Vectorization >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21656.1.patch, HIVE-21656.2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21656) Vectorize UDF mask
[ https://issues.apache.org/jira/browse/HIVE-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-21656: --- Attachment: (was: HIVE-21656.2.patch) > Vectorize UDF mask > -- > > Key: HIVE-21656 > URL: https://issues.apache.org/jira/browse/HIVE-21656 > Project: Hive > Issue Type: Improvement > Components: Vectorization >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-21656.1.patch, HIVE-21656.2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21656) Vectorize UDF mask
[ https://issues.apache.org/jira/browse/HIVE-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-21656: --- Status: Patch Available (was: Open) > Vectorize UDF mask > -- > > Key: HIVE-21656 > URL: https://issues.apache.org/jira/browse/HIVE-21656 > Project: Hive > Issue Type: Improvement > Components: Vectorization >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-21656.1.patch, HIVE-21656.2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (HIVE-21656) Vectorize UDF mask
[ https://issues.apache.org/jira/browse/HIVE-21656?focusedWorklogId=234905=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-234905 ] ASF GitHub Bot logged work on HIVE-21656: - Author: ASF GitHub Bot Created on: 29/Apr/19 21:05 Start Date: 29/Apr/19 21:05 Worklog Time Spent: 10m Work Description: vineetgarg02 commented on pull request #611: HIVE-21656: Vectorized Mask UDF URL: https://github.com/apache/hive/pull/611 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 234905) Time Spent: 10m Remaining Estimate: 0h > Vectorize UDF mask > -- > > Key: HIVE-21656 > URL: https://issues.apache.org/jira/browse/HIVE-21656 > Project: Hive > Issue Type: Improvement > Components: Vectorization >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21656.1.patch, HIVE-21656.2.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21656) Vectorize UDF mask
[ https://issues.apache.org/jira/browse/HIVE-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-21656: --- Status: Open (was: Patch Available) > Vectorize UDF mask > -- > > Key: HIVE-21656 > URL: https://issues.apache.org/jira/browse/HIVE-21656 > Project: Hive > Issue Type: Improvement > Components: Vectorization >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-21656.1.patch, HIVE-21656.2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21656) Vectorize UDF mask
[ https://issues.apache.org/jira/browse/HIVE-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-21656: --- Attachment: HIVE-21656.2.patch > Vectorize UDF mask > -- > > Key: HIVE-21656 > URL: https://issues.apache.org/jira/browse/HIVE-21656 > Project: Hive > Issue Type: Improvement > Components: Vectorization >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-21656.1.patch, HIVE-21656.2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21661) Add option to check q file syntax before execution
[
https://issues.apache.org/jira/browse/HIVE-21661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829694#comment-16829694
]
Hive QA commented on HIVE-21661:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12967397/HIVE-21661.2.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15934 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17071/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17071/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17071/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12967397 - PreCommit-HIVE-Build
> Add option to check q file syntax before execution
> --
>
> Key: HIVE-21661
> URL: https://issues.apache.org/jira/browse/HIVE-21661
> Project: Hive
> Issue Type: Improvement
> Components: Testing Infrastructure
>Reporter: Bruno Pusztahazi
>Assignee: Bruno Pusztahazi
>Priority: Major
> Attachments: HIVE-21661.1.patch, HIVE-21661.2.patch
>
>
> Currently q file tests are executed line-by-line. This could lead to
> problems, e.g. when a q file has some long running queries and the last line
> contains a syntax error. In this case everything before the wrong line will
> be executed first, which could take a lot of time and only then will the test
> fail due to the syntax error.
> I propose a simple syntax checker that will check the q file for errors
> before executing any statements. This check can will be turned off by default
> and can be enabled with the following option:
> {code:java}
> -Dtest.check.syntax=true{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21637) Synchronized metastore cache
[ https://issues.apache.org/jira/browse/HIVE-21637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829688#comment-16829688 ] Daniel Dai commented on HIVE-21637: --- Yes, I will need to create writeIds in DDL operations as well. > Synchronized metastore cache > > > Key: HIVE-21637 > URL: https://issues.apache.org/jira/browse/HIVE-21637 > Project: Hive > Issue Type: New Feature >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-21637-1.patch > > > Currently, HMS has a cache implemented by CachedStore. The cache is > asynchronized and in HMS HA setting, we can only get eventual consistency. In > this Jira, we try to make it synchronized. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21619) Print timestamp type without precision in SQL explain extended
[ https://issues.apache.org/jira/browse/HIVE-21619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-21619: --- Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) Pushed to master. > Print timestamp type without precision in SQL explain extended > -- > > Key: HIVE-21619 > URL: https://issues.apache.org/jira/browse/HIVE-21619 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21619.01.patch, HIVE-21619.01.patch, > HIVE-21619.01.patch, HIVE-21619.01.patch, HIVE-21619.01.patch, > HIVE-21619.02.patch, HIVE-21619.02.patch, HIVE-21619.02.patch, > HIVE-21619.02.patch, HIVE-21619.03.patch, HIVE-21619.patch > > > Hive dialect should print timestamp type without precision in generated SQL, > since currently Hive does not support user-defined precision. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21661) Add option to check q file syntax before execution
[
https://issues.apache.org/jira/browse/HIVE-21661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829652#comment-16829652
]
Hive QA commented on HIVE-21661:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
44s{color} | {color:blue} itests/util in master has 46 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} itests/util: The patch generated 1 new + 53 unchanged
- 0 fixed = 54 total (was 53) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
54s{color} | {color:red} itests/util generated 1 new + 46 unchanged - 0 fixed =
47 total (was 46) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 35s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:itests/util |
| | org.apache.hadoop.hive.ql.QTestSyntaxUtil.checkQFileSyntax(List)
concatenates strings using + in a loop At QTestSyntaxUtil.java:in a loop At
QTestSyntaxUtil.java:[line 68] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17071/dev-support/hive-personality.sh
|
| git revision | master / 98e2a35 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17071/yetus/diff-checkstyle-itests_util.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17071/yetus/new-findbugs-itests_util.html
|
| modules | C: itests/util U: itests/util |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17071/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add option to check q file syntax before execution
> --
>
> Key: HIVE-21661
> URL: https://issues.apache.org/jira/browse/HIVE-21661
> Project: Hive
> Issue Type: Improvement
> Components: Testing Infrastructure
>Reporter: Bruno Pusztahazi
>Assignee: Bruno Pusztahazi
>Priority: Major
> Attachments: HIVE-21661.1.patch, HIVE-21661.2.patch
>
>
> Currently q file tests are executed line-by-line. This could lead to
> problems, e.g. when a q file has some long running queries and the last line
> contains a syntax error. In this case everything before the wrong line will
> be executed first, which could take a lot of time and only then will the test
> fail due to the syntax error.
> I propose a simple syntax checker that will check the q file for errors
> before executing any statements. This check can will be turned off by default
> and can be enabled with the following option:
> {code:java}
> -Dtest.check.syntax=true{code}
--
This message was sent by
[jira] [Commented] (HIVE-20967) Handle alter events when replicate to cluster with hive.strict.managed.tables enabled.
[
https://issues.apache.org/jira/browse/HIVE-20967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829641#comment-16829641
]
Hive QA commented on HIVE-20967:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12967396/HIVE-20967.01.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15936 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17070/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17070/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17070/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12967396 - PreCommit-HIVE-Build
> Handle alter events when replicate to cluster with hive.strict.managed.tables
> enabled.
> --
>
> Key: HIVE-20967
> URL: https://issues.apache.org/jira/browse/HIVE-20967
> Project: Hive
> Issue Type: Sub-task
> Components: repl
>Affects Versions: 4.0.0
>Reporter: mahesh kumar behera
>Assignee: Ashutosh Bapat
>Priority: Minor
> Labels: DR
> Attachments: HIVE-20967.01.patch
>
>
> Some of the events from Hive2 may cause conflicts in Hive3
> (hive.strict.managed.tables=true) when applied. So, need to handle them
> properly.
> 1. Alter table to convert non-acid to acid.
> - Do not allow this conversion on source of replication if strict.managed is
> false.
> 2. Alter table or partition that changes the location.
> - For managed tables at source, the table location shouldn't be changed for
> the given non-partitioned table and partition location shouldn't be changed
> for partitioned table as alter event doesn't capture the new files list. So,
> it may cause data inconsistsency. So, if database is enabled for replication
> at source, then alter location on managed tables should be blocked.
> - For external partitioned tables, if location is changed at source, the the
> location should be changed for the table and any partitions which reside
> within the table location, but not for the partitions which are not within
> the table location. (may be we just need the test).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work stopped] (HIVE-21660) Wrong result when union all and later view with explode is used
[
https://issues.apache.org/jira/browse/HIVE-21660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-21660 stopped by Navya Sruthi Sunkarapalli.
> Wrong result when union all and later view with explode is used
> ---
>
> Key: HIVE-21660
> URL: https://issues.apache.org/jira/browse/HIVE-21660
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.1
>Reporter: Ganesha Shreedhara
>Assignee: Navya Sruthi Sunkarapalli
>Priority: Major
>
> There is a data loss when the data is inserted to a partitioned table using
> union all and lateral view with explode.
>
> *Steps to reproduce:*
>
> {code:java}
> create table t1 (id int, dt string);
> insert into t1 values (2, '2019-04-01');
> create table t3( id int, dates array);
> insert into t2 select 1 as id, array('2019-01-01','2019-01-02','2019-01-03')
> as dates;
> create table dst (id int) partitioned by (dt string);
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.exec.dynamic.partition=true;
> insert overwrite table dst partition (dt)
> select t.id, t.dt from (
> select id, dt from t1
> union all
> select id, dts as dt from t2 tt2 lateral view explode(tt2.dates) dd as dts )
> t;
> select * from dst_hdfs;
> {code}
>
>
> *Actual Result:*
> {code:java}
> +--+--+
> | 2| 2019-04-01 |
> +--+--+{code}
>
> *Expected Result* (Run only the select part from the above insert query)*:*
> {code:java}
> +---++
> | 2 | 2019-04-01 |
> | 1 | 2019-01-01 |
> | 1 | 2019-01-02 |
> | 1 | 2019-01-03 |
> +---++{code}
>
> Data retrieved using union all and lateral view with explode from second
> table is missing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work started] (HIVE-21660) Wrong result when union all and later view with explode is used
[
https://issues.apache.org/jira/browse/HIVE-21660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-21660 started by Navya Sruthi Sunkarapalli.
> Wrong result when union all and later view with explode is used
> ---
>
> Key: HIVE-21660
> URL: https://issues.apache.org/jira/browse/HIVE-21660
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.1
>Reporter: Ganesha Shreedhara
>Assignee: Navya Sruthi Sunkarapalli
>Priority: Major
>
> There is a data loss when the data is inserted to a partitioned table using
> union all and lateral view with explode.
>
> *Steps to reproduce:*
>
> {code:java}
> create table t1 (id int, dt string);
> insert into t1 values (2, '2019-04-01');
> create table t3( id int, dates array);
> insert into t2 select 1 as id, array('2019-01-01','2019-01-02','2019-01-03')
> as dates;
> create table dst (id int) partitioned by (dt string);
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.exec.dynamic.partition=true;
> insert overwrite table dst partition (dt)
> select t.id, t.dt from (
> select id, dt from t1
> union all
> select id, dts as dt from t2 tt2 lateral view explode(tt2.dates) dd as dts )
> t;
> select * from dst_hdfs;
> {code}
>
>
> *Actual Result:*
> {code:java}
> +--+--+
> | 2| 2019-04-01 |
> +--+--+{code}
>
> *Expected Result* (Run only the select part from the above insert query)*:*
> {code:java}
> +---++
> | 2 | 2019-04-01 |
> | 1 | 2019-01-01 |
> | 1 | 2019-01-02 |
> | 1 | 2019-01-03 |
> +---++{code}
>
> Data retrieved using union all and lateral view with explode from second
> table is missing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21660) Wrong result when union all and later view with explode is used
[
https://issues.apache.org/jira/browse/HIVE-21660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829632#comment-16829632
]
Navya Sruthi Sunkarapalli commented on HIVE-21660:
--
Can you provide me with the data to solve this issue.
> Wrong result when union all and later view with explode is used
> ---
>
> Key: HIVE-21660
> URL: https://issues.apache.org/jira/browse/HIVE-21660
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.1
>Reporter: Ganesha Shreedhara
>Assignee: Navya Sruthi Sunkarapalli
>Priority: Major
>
> There is a data loss when the data is inserted to a partitioned table using
> union all and lateral view with explode.
>
> *Steps to reproduce:*
>
> {code:java}
> create table t1 (id int, dt string);
> insert into t1 values (2, '2019-04-01');
> create table t3( id int, dates array);
> insert into t2 select 1 as id, array('2019-01-01','2019-01-02','2019-01-03')
> as dates;
> create table dst (id int) partitioned by (dt string);
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.exec.dynamic.partition=true;
> insert overwrite table dst partition (dt)
> select t.id, t.dt from (
> select id, dt from t1
> union all
> select id, dts as dt from t2 tt2 lateral view explode(tt2.dates) dd as dts )
> t;
> select * from dst_hdfs;
> {code}
>
>
> *Actual Result:*
> {code:java}
> +--+--+
> | 2| 2019-04-01 |
> +--+--+{code}
>
> *Expected Result* (Run only the select part from the above insert query)*:*
> {code:java}
> +---++
> | 2 | 2019-04-01 |
> | 1 | 2019-01-01 |
> | 1 | 2019-01-02 |
> | 1 | 2019-01-03 |
> +---++{code}
>
> Data retrieved using union all and lateral view with explode from second
> table is missing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-21643) Fix Broken support for ISO Time with Zone in Hive UDFs
[
https://issues.apache.org/jira/browse/HIVE-21643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navya Sruthi Sunkarapalli reassigned HIVE-21643:
Assignee: Navya Sruthi Sunkarapalli
> Fix Broken support for ISO Time with Zone in Hive UDFs
> --
>
> Key: HIVE-21643
> URL: https://issues.apache.org/jira/browse/HIVE-21643
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 3.0.0, 3.1.0, 3.1.1
>Reporter: RAJKAMAL
>Assignee: Navya Sruthi Sunkarapalli
>Priority: Major
> Labels: patch-available, pull-request-available
> Attachments: HIVE-21643.1.patch, Hive-21643.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The followings UDFs date_format and to_date used to support ISO dates with
> timezone and the support has been broken since Hive 3.x release.
> Example:
> date_format('2017-03-16T00:10:42Z', 'y')
> date_format('2017-03-16T00:10:42+01:00', 'y')
> date_format('2017-03-16T00:10:42-01:00', 'y')
> to_date('2015-04-11T01:30:45Z')
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-21660) Wrong result when union all and later view with explode is used
[
https://issues.apache.org/jira/browse/HIVE-21660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navya Sruthi Sunkarapalli reassigned HIVE-21660:
Assignee: Navya Sruthi Sunkarapalli
> Wrong result when union all and later view with explode is used
> ---
>
> Key: HIVE-21660
> URL: https://issues.apache.org/jira/browse/HIVE-21660
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.1
>Reporter: Ganesha Shreedhara
>Assignee: Navya Sruthi Sunkarapalli
>Priority: Major
>
> There is a data loss when the data is inserted to a partitioned table using
> union all and lateral view with explode.
>
> *Steps to reproduce:*
>
> {code:java}
> create table t1 (id int, dt string);
> insert into t1 values (2, '2019-04-01');
> create table t3( id int, dates array);
> insert into t2 select 1 as id, array('2019-01-01','2019-01-02','2019-01-03')
> as dates;
> create table dst (id int) partitioned by (dt string);
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.exec.dynamic.partition=true;
> insert overwrite table dst partition (dt)
> select t.id, t.dt from (
> select id, dt from t1
> union all
> select id, dts as dt from t2 tt2 lateral view explode(tt2.dates) dd as dts )
> t;
> select * from dst_hdfs;
> {code}
>
>
> *Actual Result:*
> {code:java}
> +--+--+
> | 2| 2019-04-01 |
> +--+--+{code}
>
> *Expected Result* (Run only the select part from the above insert query)*:*
> {code:java}
> +---++
> | 2 | 2019-04-01 |
> | 1 | 2019-01-01 |
> | 1 | 2019-01-02 |
> | 1 | 2019-01-03 |
> +---++{code}
>
> Data retrieved using union all and lateral view with explode from second
> table is missing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-21649) Passing an non-existent jar in HIVE_AUX_JARS_PATH produces incorrect error message
[
https://issues.apache.org/jira/browse/HIVE-21649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navya Sruthi Sunkarapalli reassigned HIVE-21649:
Assignee: Navya Sruthi Sunkarapalli
> Passing an non-existent jar in HIVE_AUX_JARS_PATH produces incorrect error
> message
> --
>
> Key: HIVE-21649
> URL: https://issues.apache.org/jira/browse/HIVE-21649
> Project: Hive
> Issue Type: Bug
> Components: Tez
>Reporter: Todd Lipcon
>Assignee: Navya Sruthi Sunkarapalli
>Priority: Minor
>
> I had configured HS2 with HIVE_AUX_JARS_PATH pointing to a non-existent
> postgres jar. This resulted in queries failing with the following error:
> {code}
> tez.DagUtils: Localizing resource because it does not exist:
> file:/data/1/todd/impala/fe/target/dependency/postgresql-42.2.5.jar to dest:
>
> hdfs://localhost:20500/tmp/hive/todd/_tez_session_dir/9de357d5-59bf-4faa-8973-5212a08bc41a-resources/postgresql-42.2.5.jar
> tez.DagUtils: Looks like another thread or process is writing the same file
> tez.DagUtils: Waiting for the file
> hdfs://localhost:20500/tmp/hive/todd/_tez_session_dir/9de357d5-59bf-4faa-8973-5212a08bc41a-resources/postgresql-42.2.5.jar
> (5 attempts, with 5000ms interval)
> tez.DagUtils: Could not find the jar that was being uploaded
> {code}
> This incorrect logging sent me on a wild goose chase looking for concurrency
> issues, HDFS issues, etc, before I realized that the jar in fact didn't exist
> on the local FS. This is due to some sketchy code which presumes that any
> IOException is due to a writing conflict.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20967) Handle alter events when replicate to cluster with hive.strict.managed.tables enabled.
[
https://issues.apache.org/jira/browse/HIVE-20967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829618#comment-16829618
]
Hive QA commented on HIVE-20967:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
28s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
27s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
30s{color} | {color:blue} standalone-metastore/metastore-common in master has
29 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
8s{color} | {color:blue} standalone-metastore/metastore-server in master has
180 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
11s{color} | {color:blue} ql in master has 2253 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
38s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
39s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
41s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
9s{color} | {color:red} standalone-metastore/metastore-common: The patch
generated 1 new + 10 unchanged - 0 fixed = 11 total (was 10) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
17s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 7 new + 26 unchanged - 0 fixed = 33 total (was 26) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
5s{color} | {color:red} ql generated 1 new + 2252 unchanged - 1 fixed = 2253
total (was 2253) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
39s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 44m 6s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | org.apache.hadoop.hive.ql.exec.DDLTask.alterTableOrSinglePartition(Hive,
AlterTableDesc, Table, Partition) makes inefficient use of keySet iterator
instead of entrySet iterator At DDLTask.java:inefficient use of keySet
iterator instead of entrySet iterator At DDLTask.java:[line 2582] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17070/dev-support/hive-personality.sh
|
| git revision | master / 98e2a35 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17070/yetus/diff-checkstyle-standalone-metastore_metastore-common.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17070/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17070/yetus/new-findbugs-ql.html
|
| modules |
[jira] [Comment Edited] (HIVE-20967) Handle alter events when replicate to cluster with hive.strict.managed.tables enabled.
[ https://issues.apache.org/jira/browse/HIVE-20967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829562#comment-16829562 ] Ashutosh Bapat edited comment on HIVE-20967 at 4/29/19 6:24 PM: Per current behaviour, if we change the table location of a partitioned external table, we do not update the locations of the existing partitions even if those are within the table location directory. So, on replica as well, we just update the table location but not the location of existing partitions even if those are within the table location directory. The attached patch .01 has tests for the same. was (Author: ashutosh.bapat): Per current behaviour, if we change the table location of a partitioned external table, we do not update the locations of the existing partitions even if those are within the table location directory. So, on replica as well, we just update the table location but not the location of existing partitions even if those are within the table location directory. > Handle alter events when replicate to cluster with hive.strict.managed.tables > enabled. > -- > > Key: HIVE-20967 > URL: https://issues.apache.org/jira/browse/HIVE-20967 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 4.0.0 >Reporter: mahesh kumar behera >Assignee: Ashutosh Bapat >Priority: Minor > Labels: DR > Attachments: HIVE-20967.01.patch > > > Some of the events from Hive2 may cause conflicts in Hive3 > (hive.strict.managed.tables=true) when applied. So, need to handle them > properly. > 1. Alter table to convert non-acid to acid. > - Do not allow this conversion on source of replication if strict.managed is > false. > 2. Alter table or partition that changes the location. > - For managed tables at source, the table location shouldn't be changed for > the given non-partitioned table and partition location shouldn't be changed > for partitioned table as alter event doesn't capture the new files list. So, > it may cause data inconsistsency. So, if database is enabled for replication > at source, then alter location on managed tables should be blocked. > - For external partitioned tables, if location is changed at source, the the > location should be changed for the table and any partitions which reside > within the table location, but not for the partitions which are not within > the table location. (may be we just need the test). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21661) Add option to check q file syntax before execution
[
https://issues.apache.org/jira/browse/HIVE-21661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bruno Pusztahazi updated HIVE-21661:
Status: Open (was: Patch Available)
> Add option to check q file syntax before execution
> --
>
> Key: HIVE-21661
> URL: https://issues.apache.org/jira/browse/HIVE-21661
> Project: Hive
> Issue Type: Improvement
> Components: Testing Infrastructure
>Reporter: Bruno Pusztahazi
>Assignee: Bruno Pusztahazi
>Priority: Major
> Attachments: HIVE-21661.1.patch, HIVE-21661.2.patch
>
>
> Currently q file tests are executed line-by-line. This could lead to
> problems, e.g. when a q file has some long running queries and the last line
> contains a syntax error. In this case everything before the wrong line will
> be executed first, which could take a lot of time and only then will the test
> fail due to the syntax error.
> I propose a simple syntax checker that will check the q file for errors
> before executing any statements. This check can will be turned off by default
> and can be enabled with the following option:
> {code:java}
> -Dtest.check.syntax=true{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20967) Handle alter events when replicate to cluster with hive.strict.managed.tables enabled.
[ https://issues.apache.org/jira/browse/HIVE-20967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829562#comment-16829562 ] Ashutosh Bapat commented on HIVE-20967: --- Per current behaviour, if we change the table location of a partitioned external table, we do not update the locations of the existing partitions even if those are within the table location directory. So, on replica as well, we just update the table location but not the location of existing partitions even if those are within the table location directory. > Handle alter events when replicate to cluster with hive.strict.managed.tables > enabled. > -- > > Key: HIVE-20967 > URL: https://issues.apache.org/jira/browse/HIVE-20967 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 4.0.0 >Reporter: mahesh kumar behera >Assignee: Ashutosh Bapat >Priority: Minor > Labels: DR > Attachments: HIVE-20967.01.patch > > > Some of the events from Hive2 may cause conflicts in Hive3 > (hive.strict.managed.tables=true) when applied. So, need to handle them > properly. > 1. Alter table to convert non-acid to acid. > - Do not allow this conversion on source of replication if strict.managed is > false. > 2. Alter table or partition that changes the location. > - For managed tables at source, the table location shouldn't be changed for > the given non-partitioned table and partition location shouldn't be changed > for partitioned table as alter event doesn't capture the new files list. So, > it may cause data inconsistsency. So, if database is enabled for replication > at source, then alter location on managed tables should be blocked. > - For external partitioned tables, if location is changed at source, the the > location should be changed for the table and any partitions which reside > within the table location, but not for the partitions which are not within > the table location. (may be we just need the test). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21661) Add option to check q file syntax before execution
[
https://issues.apache.org/jira/browse/HIVE-21661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bruno Pusztahazi updated HIVE-21661:
Attachment: HIVE-21661.2.patch
> Add option to check q file syntax before execution
> --
>
> Key: HIVE-21661
> URL: https://issues.apache.org/jira/browse/HIVE-21661
> Project: Hive
> Issue Type: Improvement
> Components: Testing Infrastructure
>Reporter: Bruno Pusztahazi
>Assignee: Bruno Pusztahazi
>Priority: Major
> Attachments: HIVE-21661.1.patch, HIVE-21661.2.patch
>
>
> Currently q file tests are executed line-by-line. This could lead to
> problems, e.g. when a q file has some long running queries and the last line
> contains a syntax error. In this case everything before the wrong line will
> be executed first, which could take a lot of time and only then will the test
> fail due to the syntax error.
> I propose a simple syntax checker that will check the q file for errors
> before executing any statements. This check can will be turned off by default
> and can be enabled with the following option:
> {code:java}
> -Dtest.check.syntax=true{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21661) Add option to check q file syntax before execution
[
https://issues.apache.org/jira/browse/HIVE-21661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bruno Pusztahazi updated HIVE-21661:
Status: Patch Available (was: Open)
> Add option to check q file syntax before execution
> --
>
> Key: HIVE-21661
> URL: https://issues.apache.org/jira/browse/HIVE-21661
> Project: Hive
> Issue Type: Improvement
> Components: Testing Infrastructure
>Reporter: Bruno Pusztahazi
>Assignee: Bruno Pusztahazi
>Priority: Major
> Attachments: HIVE-21661.1.patch, HIVE-21661.2.patch
>
>
> Currently q file tests are executed line-by-line. This could lead to
> problems, e.g. when a q file has some long running queries and the last line
> contains a syntax error. In this case everything before the wrong line will
> be executed first, which could take a lot of time and only then will the test
> fail due to the syntax error.
> I propose a simple syntax checker that will check the q file for errors
> before executing any statements. This check can will be turned off by default
> and can be enabled with the following option:
> {code:java}
> -Dtest.check.syntax=true{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20967) Handle alter events when replicate to cluster with hive.strict.managed.tables enabled.
[ https://issues.apache.org/jira/browse/HIVE-20967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Bapat updated HIVE-20967: -- Attachment: HIVE-20967.01.patch Status: Patch Available (was: Open) > Handle alter events when replicate to cluster with hive.strict.managed.tables > enabled. > -- > > Key: HIVE-20967 > URL: https://issues.apache.org/jira/browse/HIVE-20967 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 4.0.0 >Reporter: mahesh kumar behera >Assignee: Ashutosh Bapat >Priority: Minor > Labels: DR > Attachments: HIVE-20967.01.patch > > > Some of the events from Hive2 may cause conflicts in Hive3 > (hive.strict.managed.tables=true) when applied. So, need to handle them > properly. > 1. Alter table to convert non-acid to acid. > - Do not allow this conversion on source of replication if strict.managed is > false. > 2. Alter table or partition that changes the location. > - For managed tables at source, the table location shouldn't be changed for > the given non-partitioned table and partition location shouldn't be changed > for partitioned table as alter event doesn't capture the new files list. So, > it may cause data inconsistsency. So, if database is enabled for replication > at source, then alter location on managed tables should be blocked. > - For external partitioned tables, if location is changed at source, the the > location should be changed for the table and any partitions which reside > within the table location, but not for the partitions which are not within > the table location. (may be we just need the test). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21619) Print timestamp type without precision in SQL explain extended
[
https://issues.apache.org/jira/browse/HIVE-21619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829544#comment-16829544
]
Hive QA commented on HIVE-21619:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12967384/HIVE-21619.03.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15934 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17069/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17069/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17069/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12967384 - PreCommit-HIVE-Build
> Print timestamp type without precision in SQL explain extended
> --
>
> Key: HIVE-21619
> URL: https://issues.apache.org/jira/browse/HIVE-21619
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-21619.01.patch, HIVE-21619.01.patch,
> HIVE-21619.01.patch, HIVE-21619.01.patch, HIVE-21619.01.patch,
> HIVE-21619.02.patch, HIVE-21619.02.patch, HIVE-21619.02.patch,
> HIVE-21619.02.patch, HIVE-21619.03.patch, HIVE-21619.patch
>
>
> Hive dialect should print timestamp type without precision in generated SQL,
> since currently Hive does not support user-defined precision.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21619) Print timestamp type without precision in SQL explain extended
[
https://issues.apache.org/jira/browse/HIVE-21619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829431#comment-16829431
]
Hive QA commented on HIVE-21619:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
6s{color} | {color:blue} ql in master has 2253 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} ql: The patch generated 0 new + 146 unchanged - 1
fixed = 146 total (was 147) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 54s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17069/dev-support/hive-personality.sh
|
| git revision | master / 98e2a35 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17069/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Print timestamp type without precision in SQL explain extended
> --
>
> Key: HIVE-21619
> URL: https://issues.apache.org/jira/browse/HIVE-21619
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-21619.01.patch, HIVE-21619.01.patch,
> HIVE-21619.01.patch, HIVE-21619.01.patch, HIVE-21619.01.patch,
> HIVE-21619.02.patch, HIVE-21619.02.patch, HIVE-21619.02.patch,
> HIVE-21619.02.patch, HIVE-21619.03.patch, HIVE-21619.patch
>
>
> Hive dialect should print timestamp type without precision in generated SQL,
> since currently Hive does not support user-defined precision.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21619) Print timestamp type without precision in SQL explain extended
[ https://issues.apache.org/jira/browse/HIVE-21619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-21619: --- Attachment: HIVE-21619.03.patch > Print timestamp type without precision in SQL explain extended > -- > > Key: HIVE-21619 > URL: https://issues.apache.org/jira/browse/HIVE-21619 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-21619.01.patch, HIVE-21619.01.patch, > HIVE-21619.01.patch, HIVE-21619.01.patch, HIVE-21619.01.patch, > HIVE-21619.02.patch, HIVE-21619.02.patch, HIVE-21619.02.patch, > HIVE-21619.02.patch, HIVE-21619.03.patch, HIVE-21619.patch > > > Hive dialect should print timestamp type without precision in generated SQL, > since currently Hive does not support user-defined precision. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21061) CTAS query fails with IllegalStateException for empty source
[
https://issues.apache.org/jira/browse/HIVE-21061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829366#comment-16829366
]
Hive QA commented on HIVE-21061:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12967381/HIVE-21061.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15935 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning]
(batchId=191)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17068/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17068/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17068/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12967381 - PreCommit-HIVE-Build
> CTAS query fails with IllegalStateException for empty source
>
>
> Key: HIVE-21061
> URL: https://issues.apache.org/jira/browse/HIVE-21061
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.1.0
>Reporter: Riju Trivedi
>Assignee: Riju Trivedi
>Priority: Major
> Attachments: HIVE-21061.1.patch, HIVE-21061.2.patch,
> HIVE-21061.2.patch, HIVE-21061.patch
>
>
> Creating a table using CTAS from an empty source table with predicate
> condition evaluating to False
> {code}
> create table testctas1 (id int);
> create table testctas2 as select * from testctas1 where 1=2;
> {code}
> Fails with below exception:
> {code}
> Caused by: java.lang.IllegalStateException: null
> at com.google.common.base.Preconditions.checkState(Preconditions.java:159)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.verifyAndSetVectorPartDesc(Vectorizer.java:1312)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.validateInputFormatAndSchemaEvolution(Vectorizer.java:1654)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.validateAndVectorizeMapWork(Vectorizer.java:1865)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.convertMapWork(Vectorizer.java:1109)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.dispatch(Vectorizer.java:961)
> at
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.dispatch(TaskGraphWalker.java:111)
> at
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.walk(TaskGraphWalker.java:180)
> at
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.startWalking(TaskGraphWalker.java:125)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.resolve(Vectorizer.java:2442)
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeTaskPlan(TezCompiler.java:717)
> at
> org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:258)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12443)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:358)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:285)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1863)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1810)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1805)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:197)
> ... 36 more
> {code}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21662) Refactor some functionality from QTestUtil to separate classes
[ https://issues.apache.org/jira/browse/HIVE-21662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-21662: Description: This may be a follow-up of HIVE-18560: new classes: QTestSystemProperties, QFileVersionHandler, QTestMiniClusters, QTestDatasetHandler was: This may be a follow-up of HIVE-18560: Implemented: 1. QTestSystemProperties class 2. unified DEBUG_HINT > Refactor some functionality from QTestUtil to separate classes > -- > > Key: HIVE-21662 > URL: https://issues.apache.org/jira/browse/HIVE-21662 > Project: Hive > Issue Type: Improvement >Reporter: Laszlo Bodor >Assignee: Laszlo Bodor >Priority: Major > > This may be a follow-up of HIVE-18560: > new classes: QTestSystemProperties, QFileVersionHandler, QTestMiniClusters, > QTestDatasetHandler -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21061) CTAS query fails with IllegalStateException for empty source
[
https://issues.apache.org/jira/browse/HIVE-21061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829331#comment-16829331
]
Hive QA commented on HIVE-21061:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
24s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
0s{color} | {color:blue} ql in master has 2253 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 54s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17068/dev-support/hive-personality.sh
|
| git revision | master / 98e2a35 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17068/yetus/whitespace-eol.txt
|
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17068/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> CTAS query fails with IllegalStateException for empty source
>
>
> Key: HIVE-21061
> URL: https://issues.apache.org/jira/browse/HIVE-21061
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.1.0
>Reporter: Riju Trivedi
>Assignee: Riju Trivedi
>Priority: Major
> Attachments: HIVE-21061.1.patch, HIVE-21061.2.patch,
> HIVE-21061.2.patch, HIVE-21061.patch
>
>
> Creating a table using CTAS from an empty source table with predicate
> condition evaluating to False
> {code}
> create table testctas1 (id int);
> create table testctas2 as select * from testctas1 where 1=2;
> {code}
> Fails with below exception:
> {code}
> Caused by: java.lang.IllegalStateException: null
> at com.google.common.base.Preconditions.checkState(Preconditions.java:159)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.verifyAndSetVectorPartDesc(Vectorizer.java:1312)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.validateInputFormatAndSchemaEvolution(Vectorizer.java:1654)
> at
>
[jira] [Updated] (HIVE-21061) CTAS query fails with IllegalStateException for empty source
[
https://issues.apache.org/jira/browse/HIVE-21061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Riju Trivedi updated HIVE-21061:
Attachment: HIVE-21061.2.patch
> CTAS query fails with IllegalStateException for empty source
>
>
> Key: HIVE-21061
> URL: https://issues.apache.org/jira/browse/HIVE-21061
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.1.0
>Reporter: Riju Trivedi
>Assignee: Riju Trivedi
>Priority: Major
> Attachments: HIVE-21061.1.patch, HIVE-21061.2.patch,
> HIVE-21061.2.patch, HIVE-21061.patch
>
>
> Creating a table using CTAS from an empty source table with predicate
> condition evaluating to False
> {code}
> create table testctas1 (id int);
> create table testctas2 as select * from testctas1 where 1=2;
> {code}
> Fails with below exception:
> {code}
> Caused by: java.lang.IllegalStateException: null
> at com.google.common.base.Preconditions.checkState(Preconditions.java:159)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.verifyAndSetVectorPartDesc(Vectorizer.java:1312)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.validateInputFormatAndSchemaEvolution(Vectorizer.java:1654)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.validateAndVectorizeMapWork(Vectorizer.java:1865)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.convertMapWork(Vectorizer.java:1109)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.dispatch(Vectorizer.java:961)
> at
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.dispatch(TaskGraphWalker.java:111)
> at
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.walk(TaskGraphWalker.java:180)
> at
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.startWalking(TaskGraphWalker.java:125)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.resolve(Vectorizer.java:2442)
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeTaskPlan(TezCompiler.java:717)
> at
> org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:258)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12443)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:358)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:285)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1863)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1810)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1805)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:197)
> ... 36 more
> {code}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21662) Refactor some functionality from QTestUtil to separate classes
[ https://issues.apache.org/jira/browse/HIVE-21662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-21662: Description: This may be a follow-up of HIVE-18560: Implemented: 1. QTestSystemProperties class 2. unified DEBUG_HINT was:This may be a follow-up of HIVE-18560 > Refactor some functionality from QTestUtil to separate classes > -- > > Key: HIVE-21662 > URL: https://issues.apache.org/jira/browse/HIVE-21662 > Project: Hive > Issue Type: Improvement >Reporter: Laszlo Bodor >Assignee: Laszlo Bodor >Priority: Major > > This may be a follow-up of HIVE-18560: > Implemented: > 1. QTestSystemProperties class > 2. unified DEBUG_HINT -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21061) CTAS query fails with IllegalStateException for empty source
[
https://issues.apache.org/jira/browse/HIVE-21061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829242#comment-16829242
]
Hive QA commented on HIVE-21061:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12967373/HIVE-21061.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17067/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17067/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17067/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2019-04-29 13:04:03.414
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-17067/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2019-04-29 13:04:03.418
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 98e2a35 HIVE-21654: External table location is not preserved at
target when base dir is set as / (Sankar Hariappan, reviewed by Mahesh Kumar
Behera)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 98e2a35 HIVE-21654: External table location is not preserved at
target when base dir is set as / (Sankar Hariappan, reviewed by Mahesh Kumar
Behera)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2019-04-29 13:04:04.144
+ rm -rf ../yetus_PreCommit-HIVE-Build-17067
+ mkdir ../yetus_PreCommit-HIVE-Build-17067
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-17067
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-17067/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: a/itests/src/test/resources/testconfiguration.properties: does not exist
in index
error:
a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java:
does not exist in index
error: patch failed: itests/src/test/resources/testconfiguration.properties:906
Falling back to three-way merge...
Applied patch to 'itests/src/test/resources/testconfiguration.properties' with
conflicts.
Going to apply patch with: git apply -p1
error: patch failed: itests/src/test/resources/testconfiguration.properties:906
Falling back to three-way merge...
Applied patch to 'itests/src/test/resources/testconfiguration.properties' with
conflicts.
U itests/src/test/resources/testconfiguration.properties
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-17067
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12967373 - PreCommit-HIVE-Build
> CTAS query fails with IllegalStateException for empty source
>
>
> Key: HIVE-21061
> URL: https://issues.apache.org/jira/browse/HIVE-21061
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.1.0
>Reporter: Riju Trivedi
>Assignee: Riju Trivedi
>Priority: Major
> Attachments: HIVE-21061.1.patch, HIVE-21061.2.patch, HIVE-21061.patch
>
>
> Creating a table using CTAS from an empty source table with predicate
> condition evaluating to False
> {code}
> create table testctas1 (id int);
> create table testctas2 as select * from testctas1 where 1=2;
> {code}
> Fails with below exception:
> {code}
> Caused by: java.lang.IllegalStateException: null
> at com.google.common.base.Preconditions.checkState(Preconditions.java:159)
> at
>
[jira] [Updated] (HIVE-21662) Refactor some functionality from QTestUtil to separate classes
[ https://issues.apache.org/jira/browse/HIVE-21662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-21662: Issue Type: Improvement (was: Bug) > Refactor some functionality from QTestUtil to separate classes > -- > > Key: HIVE-21662 > URL: https://issues.apache.org/jira/browse/HIVE-21662 > Project: Hive > Issue Type: Improvement >Reporter: Laszlo Bodor >Assignee: Laszlo Bodor >Priority: Major > > This may be a follow-up of HIVE-18560 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21662) Refactor some functionality from QTestUtil to separate classes
[ https://issues.apache.org/jira/browse/HIVE-21662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-21662: Description: This may be a follow-up of HIVE-18560 > Refactor some functionality from QTestUtil to separate classes > -- > > Key: HIVE-21662 > URL: https://issues.apache.org/jira/browse/HIVE-21662 > Project: Hive > Issue Type: Bug >Reporter: Laszlo Bodor >Assignee: Laszlo Bodor >Priority: Major > > This may be a follow-up of HIVE-18560 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21662) Refactor some functionality from QTestUtil to separate classes
[ https://issues.apache.org/jira/browse/HIVE-21662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor reassigned HIVE-21662: --- Assignee: Laszlo Bodor > Refactor some functionality from QTestUtil to separate classes > -- > > Key: HIVE-21662 > URL: https://issues.apache.org/jira/browse/HIVE-21662 > Project: Hive > Issue Type: Bug >Reporter: Laszlo Bodor >Assignee: Laszlo Bodor >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-21661) Add option to check q file syntax before execution
[
https://issues.apache.org/jira/browse/HIVE-21661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829209#comment-16829209
]
Laszlo Bodor edited comment on HIVE-21661 at 4/29/19 12:33 PM:
---
[~bpusztahazi]: I like the idea and the implementation as well, only one note:
could you please refactor the new functionality to a new class (syntax checker
or stuff)? However, QTestUtil is already a big monster, let's try not to fatten
it further (e.g. lots of new processor imports introduced by the patch which
QTestUtil shouldn't be aware of IMO)
was (Author: abstractdog):
[~bpusztahazi]: I like the idea and the implementation as well, only one note:
could you please refactor the new functionality to a new class (syntax checker
or stuff)? However, QTestUtil is already a big monster, let's try not to fatten
it further (e.g. lots of new processor imports which QTestUtils shouldn't be
aware of)
> Add option to check q file syntax before execution
> --
>
> Key: HIVE-21661
> URL: https://issues.apache.org/jira/browse/HIVE-21661
> Project: Hive
> Issue Type: Improvement
> Components: Testing Infrastructure
>Reporter: Bruno Pusztahazi
>Assignee: Bruno Pusztahazi
>Priority: Major
> Attachments: HIVE-21661.1.patch
>
>
> Currently q file tests are executed line-by-line. This could lead to
> problems, e.g. when a q file has some long running queries and the last line
> contains a syntax error. In this case everything before the wrong line will
> be executed first, which could take a lot of time and only then will the test
> fail due to the syntax error.
> I propose a simple syntax checker that will check the q file for errors
> before executing any statements. This check can will be turned off by default
> and can be enabled with the following option:
> {code:java}
> -Dtest.check.syntax=true{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21061) CTAS query fails with IllegalStateException for empty source
[
https://issues.apache.org/jira/browse/HIVE-21061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Riju Trivedi updated HIVE-21061:
Attachment: HIVE-21061.2.patch
> CTAS query fails with IllegalStateException for empty source
>
>
> Key: HIVE-21061
> URL: https://issues.apache.org/jira/browse/HIVE-21061
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.1.0
>Reporter: Riju Trivedi
>Assignee: Riju Trivedi
>Priority: Major
> Attachments: HIVE-21061.1.patch, HIVE-21061.2.patch, HIVE-21061.patch
>
>
> Creating a table using CTAS from an empty source table with predicate
> condition evaluating to False
> {code}
> create table testctas1 (id int);
> create table testctas2 as select * from testctas1 where 1=2;
> {code}
> Fails with below exception:
> {code}
> Caused by: java.lang.IllegalStateException: null
> at com.google.common.base.Preconditions.checkState(Preconditions.java:159)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.verifyAndSetVectorPartDesc(Vectorizer.java:1312)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.validateInputFormatAndSchemaEvolution(Vectorizer.java:1654)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.validateAndVectorizeMapWork(Vectorizer.java:1865)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.convertMapWork(Vectorizer.java:1109)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.dispatch(Vectorizer.java:961)
> at
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.dispatch(TaskGraphWalker.java:111)
> at
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.walk(TaskGraphWalker.java:180)
> at
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.startWalking(TaskGraphWalker.java:125)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.resolve(Vectorizer.java:2442)
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeTaskPlan(TezCompiler.java:717)
> at
> org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:258)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12443)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:358)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:285)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1863)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1810)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1805)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:197)
> ... 36 more
> {code}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21061) CTAS query fails with IllegalStateException for empty source
[
https://issues.apache.org/jira/browse/HIVE-21061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Riju Trivedi updated HIVE-21061:
Status: Patch Available (was: Open)
> CTAS query fails with IllegalStateException for empty source
>
>
> Key: HIVE-21061
> URL: https://issues.apache.org/jira/browse/HIVE-21061
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.1.0
>Reporter: Riju Trivedi
>Assignee: Riju Trivedi
>Priority: Major
> Attachments: HIVE-21061.1.patch, HIVE-21061.2.patch, HIVE-21061.patch
>
>
> Creating a table using CTAS from an empty source table with predicate
> condition evaluating to False
> {code}
> create table testctas1 (id int);
> create table testctas2 as select * from testctas1 where 1=2;
> {code}
> Fails with below exception:
> {code}
> Caused by: java.lang.IllegalStateException: null
> at com.google.common.base.Preconditions.checkState(Preconditions.java:159)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.verifyAndSetVectorPartDesc(Vectorizer.java:1312)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.validateInputFormatAndSchemaEvolution(Vectorizer.java:1654)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.validateAndVectorizeMapWork(Vectorizer.java:1865)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.convertMapWork(Vectorizer.java:1109)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.dispatch(Vectorizer.java:961)
> at
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.dispatch(TaskGraphWalker.java:111)
> at
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.walk(TaskGraphWalker.java:180)
> at
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.startWalking(TaskGraphWalker.java:125)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.resolve(Vectorizer.java:2442)
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeTaskPlan(TezCompiler.java:717)
> at
> org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:258)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12443)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:358)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:285)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1863)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1810)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1805)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:197)
> ... 36 more
> {code}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21661) Add option to check q file syntax before execution
[
https://issues.apache.org/jira/browse/HIVE-21661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829209#comment-16829209
]
Laszlo Bodor commented on HIVE-21661:
-
[~bpusztahazi]: I like the idea and the implementation as well, only one note:
could you please refactor the new functionality to a new class (syntax checker
or stuff)? However, QTestUtil is already a big monster, let's try not to fatten
it further (e.g. lots of new processor imports which QTestUtils shouldn't be
aware of)
> Add option to check q file syntax before execution
> --
>
> Key: HIVE-21661
> URL: https://issues.apache.org/jira/browse/HIVE-21661
> Project: Hive
> Issue Type: Improvement
> Components: Testing Infrastructure
>Reporter: Bruno Pusztahazi
>Assignee: Bruno Pusztahazi
>Priority: Major
> Attachments: HIVE-21661.1.patch
>
>
> Currently q file tests are executed line-by-line. This could lead to
> problems, e.g. when a q file has some long running queries and the last line
> contains a syntax error. In this case everything before the wrong line will
> be executed first, which could take a lot of time and only then will the test
> fail due to the syntax error.
> I propose a simple syntax checker that will check the q file for errors
> before executing any statements. This check can will be turned off by default
> and can be enabled with the following option:
> {code:java}
> -Dtest.check.syntax=true{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-7502) Writes to parquet tables via HCatalog fail with "java.lang.RuntimeException: Should never be used".
[
https://issues.apache.org/jira/browse/HIVE-7502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pramod Gapat reassigned HIVE-7502:
--
Assignee: (was: Pramod Gapat)
> Writes to parquet tables via HCatalog fail with "java.lang.RuntimeException:
> Should never be used".
> ---
>
> Key: HIVE-7502
> URL: https://issues.apache.org/jira/browse/HIVE-7502
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 0.13.1
>Reporter: Craig Condit
>Priority: Major
>
> Attempting to store data into a Parquet table via HCatalog (using Pig) fails.
> Based on the code, it appears that
> org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat.getRecordWriter()
> is unimplemented.
> To duplicate:
> Hive:
> {noformat}
> create table parquet_input_test stored as parquet as with x as (select test
> as value) select * from x;
> create table parquet_output_test (value int) stored as parquet;
> {noformat}
> Pig:
> {noformat}
> A = LOAD 'parquet_input_test' USING org.apache.hive.hcatalog.pig.HCatLoader();
> STORE A INTO 'parquet_output_test' USING
> org.apache.hive.hcatalog.pig.HCatStorer();
> {noformat}
> Result:
> {noformat}
> java.lang.RuntimeException: Should never be used
> at
> org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat.getRecordWriter(MapredParquetOutputFormat.java:76)
> at
> org.apache.hive.hcatalog.mapreduce.FileOutputFormatContainer.getRecordWriter(FileOutputFormatContainer.java:103)
> at
> org.apache.hive.hcatalog.mapreduce.HCatOutputFormat.getRecordWriter(HCatOutputFormat.java:260)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:84)
> at
> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.(MapTask.java:623)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:743)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:339)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-7502) Writes to parquet tables via HCatalog fail with "java.lang.RuntimeException: Should never be used".
[
https://issues.apache.org/jira/browse/HIVE-7502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pramod Gapat reassigned HIVE-7502:
--
Assignee: Pramod Gapat
> Writes to parquet tables via HCatalog fail with "java.lang.RuntimeException:
> Should never be used".
> ---
>
> Key: HIVE-7502
> URL: https://issues.apache.org/jira/browse/HIVE-7502
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 0.13.1
>Reporter: Craig Condit
>Assignee: Pramod Gapat
>Priority: Major
>
> Attempting to store data into a Parquet table via HCatalog (using Pig) fails.
> Based on the code, it appears that
> org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat.getRecordWriter()
> is unimplemented.
> To duplicate:
> Hive:
> {noformat}
> create table parquet_input_test stored as parquet as with x as (select test
> as value) select * from x;
> create table parquet_output_test (value int) stored as parquet;
> {noformat}
> Pig:
> {noformat}
> A = LOAD 'parquet_input_test' USING org.apache.hive.hcatalog.pig.HCatLoader();
> STORE A INTO 'parquet_output_test' USING
> org.apache.hive.hcatalog.pig.HCatStorer();
> {noformat}
> Result:
> {noformat}
> java.lang.RuntimeException: Should never be used
> at
> org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat.getRecordWriter(MapredParquetOutputFormat.java:76)
> at
> org.apache.hive.hcatalog.mapreduce.FileOutputFormatContainer.getRecordWriter(FileOutputFormatContainer.java:103)
> at
> org.apache.hive.hcatalog.mapreduce.HCatOutputFormat.getRecordWriter(HCatOutputFormat.java:260)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:84)
> at
> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.(MapTask.java:623)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:743)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:339)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21661) Add option to check q file syntax before execution
[
https://issues.apache.org/jira/browse/HIVE-21661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829163#comment-16829163
]
Hive QA commented on HIVE-21661:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12967364/HIVE-21661.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15934 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17066/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17066/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17066/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12967364 - PreCommit-HIVE-Build
> Add option to check q file syntax before execution
> --
>
> Key: HIVE-21661
> URL: https://issues.apache.org/jira/browse/HIVE-21661
> Project: Hive
> Issue Type: Improvement
> Components: Testing Infrastructure
>Reporter: Bruno Pusztahazi
>Assignee: Bruno Pusztahazi
>Priority: Major
> Attachments: HIVE-21661.1.patch
>
>
> Currently q file tests are executed line-by-line. This could lead to
> problems, e.g. when a q file has some long running queries and the last line
> contains a syntax error. In this case everything before the wrong line will
> be executed first, which could take a lot of time and only then will the test
> fail due to the syntax error.
> I propose a simple syntax checker that will check the q file for errors
> before executing any statements. This check can will be turned off by default
> and can be enabled with the following option:
> {code:java}
> -Dtest.check.syntax=true{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21061) CTAS query fails with IllegalStateException for empty source
[
https://issues.apache.org/jira/browse/HIVE-21061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829148#comment-16829148
]
Riju Trivedi commented on HIVE-21061:
-
[~vgumashta] Added test case.
> CTAS query fails with IllegalStateException for empty source
>
>
> Key: HIVE-21061
> URL: https://issues.apache.org/jira/browse/HIVE-21061
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.1.0
>Reporter: Riju Trivedi
>Assignee: Riju Trivedi
>Priority: Major
> Attachments: HIVE-21061.1.patch, HIVE-21061.patch
>
>
> Creating a table using CTAS from an empty source table with predicate
> condition evaluating to False
> {code}
> create table testctas1 (id int);
> create table testctas2 as select * from testctas1 where 1=2;
> {code}
> Fails with below exception:
> {code}
> Caused by: java.lang.IllegalStateException: null
> at com.google.common.base.Preconditions.checkState(Preconditions.java:159)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.verifyAndSetVectorPartDesc(Vectorizer.java:1312)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.validateInputFormatAndSchemaEvolution(Vectorizer.java:1654)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.validateAndVectorizeMapWork(Vectorizer.java:1865)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.convertMapWork(Vectorizer.java:1109)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.dispatch(Vectorizer.java:961)
> at
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.dispatch(TaskGraphWalker.java:111)
> at
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.walk(TaskGraphWalker.java:180)
> at
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.startWalking(TaskGraphWalker.java:125)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.resolve(Vectorizer.java:2442)
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeTaskPlan(TezCompiler.java:717)
> at
> org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:258)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12443)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:358)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:285)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1863)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1810)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1805)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:197)
> ... 36 more
> {code}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21061) CTAS query fails with IllegalStateException for empty source
[
https://issues.apache.org/jira/browse/HIVE-21061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Riju Trivedi updated HIVE-21061:
Attachment: HIVE-21061.1.patch
> CTAS query fails with IllegalStateException for empty source
>
>
> Key: HIVE-21061
> URL: https://issues.apache.org/jira/browse/HIVE-21061
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.1.0
>Reporter: Riju Trivedi
>Assignee: Riju Trivedi
>Priority: Major
> Attachments: HIVE-21061.1.patch, HIVE-21061.patch
>
>
> Creating a table using CTAS from an empty source table with predicate
> condition evaluating to False
> {code}
> create table testctas1 (id int);
> create table testctas2 as select * from testctas1 where 1=2;
> {code}
> Fails with below exception:
> {code}
> Caused by: java.lang.IllegalStateException: null
> at com.google.common.base.Preconditions.checkState(Preconditions.java:159)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.verifyAndSetVectorPartDesc(Vectorizer.java:1312)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.validateInputFormatAndSchemaEvolution(Vectorizer.java:1654)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.validateAndVectorizeMapWork(Vectorizer.java:1865)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.convertMapWork(Vectorizer.java:1109)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.dispatch(Vectorizer.java:961)
> at
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.dispatch(TaskGraphWalker.java:111)
> at
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.walk(TaskGraphWalker.java:180)
> at
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.startWalking(TaskGraphWalker.java:125)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.resolve(Vectorizer.java:2442)
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeTaskPlan(TezCompiler.java:717)
> at
> org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:258)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12443)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:358)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:285)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1863)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1810)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1805)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:197)
> ... 36 more
> {code}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21661) Add option to check q file syntax before execution
[
https://issues.apache.org/jira/browse/HIVE-21661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829136#comment-16829136
]
Hive QA commented on HIVE-21661:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
45s{color} | {color:blue} itests/util in master has 46 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 19s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17066/dev-support/hive-personality.sh
|
| git revision | master / 98e2a35 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: itests/util U: itests/util |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17066/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add option to check q file syntax before execution
> --
>
> Key: HIVE-21661
> URL: https://issues.apache.org/jira/browse/HIVE-21661
> Project: Hive
> Issue Type: Improvement
> Components: Testing Infrastructure
>Reporter: Bruno Pusztahazi
>Assignee: Bruno Pusztahazi
>Priority: Major
> Attachments: HIVE-21661.1.patch
>
>
> Currently q file tests are executed line-by-line. This could lead to
> problems, e.g. when a q file has some long running queries and the last line
> contains a syntax error. In this case everything before the wrong line will
> be executed first, which could take a lot of time and only then will the test
> fail due to the syntax error.
> I propose a simple syntax checker that will check the q file for errors
> before executing any statements. This check can will be turned off by default
> and can be enabled with the following option:
> {code:java}
> -Dtest.check.syntax=true{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21598) CTAS on ACID table during incremental does not replicate data
[
https://issues.apache.org/jira/browse/HIVE-21598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829110#comment-16829110
]
Hive QA commented on HIVE-21598:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12967356/HIVE-21598.05.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15970 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17065/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17065/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17065/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12967356 - PreCommit-HIVE-Build
> CTAS on ACID table during incremental does not replicate data
> -
>
> Key: HIVE-21598
> URL: https://issues.apache.org/jira/browse/HIVE-21598
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, repl
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: CTAS, Repl
> Attachments: HIVE-21598.01.patch, HIVE-21598.02.patch,
> HIVE-21598.03.patch, HIVE-21598.04.patch, HIVE-21598.05.patch
>
>
> Scenario
> create database dumpdb with dbproperties('repl.source.for'='1,2,3');
> use dumpdb;
> create table t1 (id int) clustered by(id) into 3 buckets stored as orc
> tblproperties ("transactional"="true");
> insert into t1 values(1);
> insert into t1 values(2);
> repl dump dumpdb;
> repl load loaddb from ;
> use loaddb;
> select * from t1;
> ++
> | t6.id |
> ++
> | 1 |
> | 2 |
> +
> use dumpdb;
> create table t6 stored as orc tblproperties ("transactional"="true") as
> select * from t1;
> select * from t6;
> ++
> | t6.id |
> ++
> | 1 |
> | 2 |
> ++
> repl dump dumpdb from
> repl load loaddb from ;
> use loaddb;
> select * from t6;
> ++
> | t6.id |
> ++
> ++
> t6 gets created but there's no data.
>
> On further investigation, I see that the CommitTxnEvent's dump directory has
> _files but it is empty. Looks like we do not log names of the files created
> as part of CTAS.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21598) CTAS on ACID table during incremental does not replicate data
[
https://issues.apache.org/jira/browse/HIVE-21598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829075#comment-16829075
]
Hive QA commented on HIVE-21598:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
45s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
8s{color} | {color:blue} ql in master has 2253 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
26s{color} | {color:blue} testutils/ptest2 in master has 24 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
34s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
56s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} itests/hive-unit: The patch generated 4 new + 117
unchanged - 7 fixed = 121 total (was 124) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 3 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 34m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17065/dev-support/hive-personality.sh
|
| git revision | master / 98e2a35 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17065/yetus/diff-checkstyle-itests_hive-unit.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17065/yetus/patch-asflicense-problems.txt
|
| modules | C: ql itests/hive-unit testutils/ptest2 U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17065/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> CTAS on ACID table during incremental does not replicate data
> -
>
> Key: HIVE-21598
> URL: https://issues.apache.org/jira/browse/HIVE-21598
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, repl
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: CTAS, Repl
> Attachments: HIVE-21598.01.patch, HIVE-21598.02.patch,
> HIVE-21598.03.patch, HIVE-21598.04.patch, HIVE-21598.05.patch
>
>
> Scenario
> create database dumpdb with dbproperties('repl.source.for'='1,2,3');
> use dumpdb;
> create table t1 (id int) clustered by(id) into 3 buckets stored as orc
>
[jira] [Updated] (HIVE-21661) Add option to check q file syntax before execution
[
https://issues.apache.org/jira/browse/HIVE-21661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bruno Pusztahazi updated HIVE-21661:
Attachment: HIVE-21661.1.patch
> Add option to check q file syntax before execution
> --
>
> Key: HIVE-21661
> URL: https://issues.apache.org/jira/browse/HIVE-21661
> Project: Hive
> Issue Type: Improvement
> Components: Testing Infrastructure
>Reporter: Bruno Pusztahazi
>Assignee: Bruno Pusztahazi
>Priority: Major
> Attachments: HIVE-21661.1.patch
>
>
> Currently q file tests are executed line-by-line. This could lead to
> problems, e.g. when a q file has some long running queries and the last line
> contains a syntax error. In this case everything before the wrong line will
> be executed first, which could take a lot of time and only then will the test
> fail due to the syntax error.
> I propose a simple syntax checker that will check the q file for errors
> before executing any statements. This check can will be turned off by default
> and can be enabled with the following option:
> {code:java}
> -Dtest.check.syntax=true{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21661) Add option to check q file syntax before execution
[
https://issues.apache.org/jira/browse/HIVE-21661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bruno Pusztahazi updated HIVE-21661:
Status: Patch Available (was: Open)
> Add option to check q file syntax before execution
> --
>
> Key: HIVE-21661
> URL: https://issues.apache.org/jira/browse/HIVE-21661
> Project: Hive
> Issue Type: Improvement
> Components: Testing Infrastructure
>Reporter: Bruno Pusztahazi
>Assignee: Bruno Pusztahazi
>Priority: Major
> Attachments: HIVE-21661.1.patch
>
>
> Currently q file tests are executed line-by-line. This could lead to
> problems, e.g. when a q file has some long running queries and the last line
> contains a syntax error. In this case everything before the wrong line will
> be executed first, which could take a lot of time and only then will the test
> fail due to the syntax error.
> I propose a simple syntax checker that will check the q file for errors
> before executing any statements. This check can will be turned off by default
> and can be enabled with the following option:
> {code:java}
> -Dtest.check.syntax=true{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-21661) Add option to check q file syntax before execution
[
https://issues.apache.org/jira/browse/HIVE-21661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bruno Pusztahazi reassigned HIVE-21661:
---
> Add option to check q file syntax before execution
> --
>
> Key: HIVE-21661
> URL: https://issues.apache.org/jira/browse/HIVE-21661
> Project: Hive
> Issue Type: Improvement
> Components: Testing Infrastructure
>Reporter: Bruno Pusztahazi
>Assignee: Bruno Pusztahazi
>Priority: Major
>
> Currently q file tests are executed line-by-line. This could lead to
> problems, e.g. when a q file has some long running queries and the last line
> contains a syntax error. In this case everything before the wrong line will
> be executed first, which could take a lot of time and only then will the test
> fail due to the syntax error.
> I propose a simple syntax checker that will check the q file for errors
> before executing any statements. This check can will be turned off by default
> and can be enabled with the following option:
> {code:java}
> -Dtest.check.syntax=true{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20967) Handle alter events when replicate to cluster with hive.strict.managed.tables enabled.
[ https://issues.apache.org/jira/browse/HIVE-20967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Bapat updated HIVE-20967: -- Description: Some of the events from Hive2 may cause conflicts in Hive3 (hive.strict.managed.tables=true) when applied. So, need to handle them properly. 1. Alter table to convert non-acid to acid. - Do not allow this conversion on source of replication if strict.managed is false. 2. Alter table or partition that changes the location. - For managed tables at source, the table location shouldn't be changed for the given non-partitioned table and partition location shouldn't be changed for partitioned table as alter event doesn't capture the new files list. So, it may cause data inconsistsency. So, if database is enabled for replication at source, then alter location on managed tables should be blocked. - For external partitioned tables, if location is changed at source, the the location should be changed for the table and any partitions which reside within the table location, but not for the partitions which are not within the table location. (may be we just need the test). was: Some of the events from Hive2 may cause conflicts in Hive3 (hive.strict.managed.tables=true) when applied. So, need to handle them properly. 1. Alter table to convert non-acid to acid. - This event should be no-op as the table in target might be already acid or MM or external table. 2. Alter table or partition that changes the location. - For managed tables at source, the table location shouldn't be changed for the given non-partitioned table and partition location shouldn't be changed for partitioned table as alter event doesn't capture the new files list. So, it may cause data inconsistsency. So, if database is enabled for replication at source, then alter location on managed tables should be blocked. - For external partitioned tables, if location is changed at source, the the location should be changed for the table and any partitions which reside within the table location, but not for the partitions which are not within the table location. (may be we just need the test). > Handle alter events when replicate to cluster with hive.strict.managed.tables > enabled. > -- > > Key: HIVE-20967 > URL: https://issues.apache.org/jira/browse/HIVE-20967 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 4.0.0 >Reporter: mahesh kumar behera >Assignee: Ashutosh Bapat >Priority: Minor > Labels: DR > > Some of the events from Hive2 may cause conflicts in Hive3 > (hive.strict.managed.tables=true) when applied. So, need to handle them > properly. > 1. Alter table to convert non-acid to acid. > - Do not allow this conversion on source of replication if strict.managed is > false. > 2. Alter table or partition that changes the location. > - For managed tables at source, the table location shouldn't be changed for > the given non-partitioned table and partition location shouldn't be changed > for partitioned table as alter event doesn't capture the new files list. So, > it may cause data inconsistsency. So, if database is enabled for replication > at source, then alter location on managed tables should be blocked. > - For external partitioned tables, if location is changed at source, the the > location should be changed for the table and any partitions which reside > within the table location, but not for the partitions which are not within > the table location. (may be we just need the test). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21598) CTAS on ACID table during incremental does not replicate data
[
https://issues.apache.org/jira/browse/HIVE-21598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Bapat updated HIVE-21598:
--
Attachment: HIVE-21598.05.patch
Status: Patch Available (was: In Progress)
Resubmit .04 as .05 for ptest trigger.
> CTAS on ACID table during incremental does not replicate data
> -
>
> Key: HIVE-21598
> URL: https://issues.apache.org/jira/browse/HIVE-21598
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, repl
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: CTAS, Repl
> Attachments: HIVE-21598.01.patch, HIVE-21598.02.patch,
> HIVE-21598.03.patch, HIVE-21598.04.patch, HIVE-21598.05.patch
>
>
> Scenario
> create database dumpdb with dbproperties('repl.source.for'='1,2,3');
> use dumpdb;
> create table t1 (id int) clustered by(id) into 3 buckets stored as orc
> tblproperties ("transactional"="true");
> insert into t1 values(1);
> insert into t1 values(2);
> repl dump dumpdb;
> repl load loaddb from ;
> use loaddb;
> select * from t1;
> ++
> | t6.id |
> ++
> | 1 |
> | 2 |
> +
> use dumpdb;
> create table t6 stored as orc tblproperties ("transactional"="true") as
> select * from t1;
> select * from t6;
> ++
> | t6.id |
> ++
> | 1 |
> | 2 |
> ++
> repl dump dumpdb from
> repl load loaddb from ;
> use loaddb;
> select * from t6;
> ++
> | t6.id |
> ++
> ++
> t6 gets created but there's no data.
>
> On further investigation, I see that the CommitTxnEvent's dump directory has
> _files but it is empty. Looks like we do not log names of the files created
> as part of CTAS.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-7502) Writes to parquet tables via HCatalog fail with "java.lang.RuntimeException: Should never be used".
[
https://issues.apache.org/jira/browse/HIVE-7502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829030#comment-16829030
]
Pramod Gapat commented on HIVE-7502:
Is the issue still open ?
has recent update solved this?
Ref -
[https://cwiki.apache.org/confluence/display/Hive/HCatalog+StorageFormats]
[Parquet|https://cwiki.apache.org/confluence/display/Hive/Parquet] was added
in Hive 0.10.0 (plug-in) and Hive 0.13.0 (native)
> Writes to parquet tables via HCatalog fail with "java.lang.RuntimeException:
> Should never be used".
> ---
>
> Key: HIVE-7502
> URL: https://issues.apache.org/jira/browse/HIVE-7502
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 0.13.1
>Reporter: Craig Condit
>Priority: Major
>
> Attempting to store data into a Parquet table via HCatalog (using Pig) fails.
> Based on the code, it appears that
> org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat.getRecordWriter()
> is unimplemented.
> To duplicate:
> Hive:
> {noformat}
> create table parquet_input_test stored as parquet as with x as (select test
> as value) select * from x;
> create table parquet_output_test (value int) stored as parquet;
> {noformat}
> Pig:
> {noformat}
> A = LOAD 'parquet_input_test' USING org.apache.hive.hcatalog.pig.HCatLoader();
> STORE A INTO 'parquet_output_test' USING
> org.apache.hive.hcatalog.pig.HCatStorer();
> {noformat}
> Result:
> {noformat}
> java.lang.RuntimeException: Should never be used
> at
> org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat.getRecordWriter(MapredParquetOutputFormat.java:76)
> at
> org.apache.hive.hcatalog.mapreduce.FileOutputFormatContainer.getRecordWriter(FileOutputFormatContainer.java:103)
> at
> org.apache.hive.hcatalog.mapreduce.HCatOutputFormat.getRecordWriter(HCatOutputFormat.java:260)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:84)
> at
> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.(MapTask.java:623)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:743)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:339)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21598) CTAS on ACID table during incremental does not replicate data
[
https://issues.apache.org/jira/browse/HIVE-21598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Bapat updated HIVE-21598:
--
Status: In Progress (was: Patch Available)
> CTAS on ACID table during incremental does not replicate data
> -
>
> Key: HIVE-21598
> URL: https://issues.apache.org/jira/browse/HIVE-21598
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, repl
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: CTAS, Repl
> Attachments: HIVE-21598.01.patch, HIVE-21598.02.patch,
> HIVE-21598.03.patch, HIVE-21598.04.patch
>
>
> Scenario
> create database dumpdb with dbproperties('repl.source.for'='1,2,3');
> use dumpdb;
> create table t1 (id int) clustered by(id) into 3 buckets stored as orc
> tblproperties ("transactional"="true");
> insert into t1 values(1);
> insert into t1 values(2);
> repl dump dumpdb;
> repl load loaddb from ;
> use loaddb;
> select * from t1;
> ++
> | t6.id |
> ++
> | 1 |
> | 2 |
> +
> use dumpdb;
> create table t6 stored as orc tblproperties ("transactional"="true") as
> select * from t1;
> select * from t6;
> ++
> | t6.id |
> ++
> | 1 |
> | 2 |
> ++
> repl dump dumpdb from
> repl load loaddb from ;
> use loaddb;
> select * from t6;
> ++
> | t6.id |
> ++
> ++
> t6 gets created but there's no data.
>
> On further investigation, I see that the CommitTxnEvent's dump directory has
> _files but it is empty. Looks like we do not log names of the files created
> as part of CTAS.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20967) Handle alter events when replicate to cluster with hive.strict.managed.tables enabled.
[ https://issues.apache.org/jira/browse/HIVE-20967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Bapat updated HIVE-20967: -- Description: Some of the events from Hive2 may cause conflicts in Hive3 (hive.strict.managed.tables=true) when applied. So, need to handle them properly. 1. Alter table to convert non-acid to acid. - This event should be no-op as the table in target might be already acid or MM or external table. 2. Alter table or partition that changes the location. - For managed tables at source, the table location shouldn't be changed for the given non-partitioned table and partition location shouldn't be changed for partitioned table as alter event doesn't capture the new files list. So, it may cause data inconsistsency. So, if database is enabled for replication at source, then alter location on managed tables should be blocked. - For external partitioned tables, if location is changed at source, the the location should be changed for the table and any partitions which reside within the table location, but not for the partitions which are not within the table location. (may be we just need the test). was: Some of the events from Hive2 may cause conflicts in Hive3 (hive.strict.managed.tables=true) when applied. So, need to handle them properly. 1. Alter table to convert non-acid to acid. - This event should be no-op as the table in target might be already acid or MM or external table. So, don't dump (and replicate) that event. Anyway, any event generated on the source on such an ALTERed table won't be dumped (and replicated) to the target. 2. Alter table or partition that changes the location. - For managed tables at source, the table location shouldn't be changed for the given non-partitioned table and partition location shouldn't be changed for partitioned table as alter event doesn't capture the new files list. So, it may cause data inconsistsency. So, if database is enabled for replication at source, then alter location on managed tables should be blocked. - For external partitioned tables, if location is changed at source, the the location should be changed for the table and any partitions which reside within the table location, but not for the partitions which are not within the table location. (may be we just need the test). > Handle alter events when replicate to cluster with hive.strict.managed.tables > enabled. > -- > > Key: HIVE-20967 > URL: https://issues.apache.org/jira/browse/HIVE-20967 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 4.0.0 >Reporter: mahesh kumar behera >Assignee: Ashutosh Bapat >Priority: Minor > Labels: DR > > Some of the events from Hive2 may cause conflicts in Hive3 > (hive.strict.managed.tables=true) when applied. So, need to handle them > properly. > 1. Alter table to convert non-acid to acid. > - This event should be no-op as the table in target might be already acid or > MM or external table. > 2. Alter table or partition that changes the location. > - For managed tables at source, the table location shouldn't be changed for > the given non-partitioned table and partition location shouldn't be changed > for partitioned table as alter event doesn't capture the new files list. So, > it may cause data inconsistsency. So, if database is enabled for replication > at source, then alter location on managed tables should be blocked. > - For external partitioned tables, if location is changed at source, the the > location should be changed for the table and any partitions which reside > within the table location, but not for the partitions which are not within > the table location. (may be we just need the test). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20967) Handle alter events when replicate to cluster with hive.strict.managed.tables enabled.
[ https://issues.apache.org/jira/browse/HIVE-20967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Bapat updated HIVE-20967: -- Description: Some of the events from Hive2 may cause conflicts in Hive3 (hive.strict.managed.tables=true) when applied. So, need to handle them properly. 1. Alter table to convert non-acid to acid. - This event should be no-op as the table in target might be already acid or MM or external table. 2. Alter table or partition that changes the location. - For managed tables at source, the table location shouldn't be changed for the given non-partitioned table and partition location shouldn't be changed for partitioned table as alter event doesn't capture the new files list. So, it may cause data inconsistsency. So, if database is enabled for replication at source, then alter location on managed tables should be blocked. - For external partitioned tables, if location is changed at source, the the location should be changed for the table and any partitions which reside within the table location, but not for the partitions which are not within the table location. (may be we just need the test). was: Some of the events from Hive2 may cause conflicts in Hive3 (hive.strict.managed.tables=true) when applied. So, need to handle them properly. 1. Alter table to convert non-acid to acid. - This event should be no-op as the table in target might be already acid or MM or external table. 2. Alter table or partition that changes the location. - For managed tables at source, the table location shouldn't be changed for the given non-partitioned table and partition location shouldn't be changed for partitioned table as alter event doesn't capture the new files list. So, it may cause data inconsistsency. So, if database is enabled for replication at source, then alter location on managed tables should be blocked. - For external partitioned tables, if location is changed at source, the the location should be changed for the table and any partitions which reside within the table location, but not for the partitions which are not within the table location. > Handle alter events when replicate to cluster with hive.strict.managed.tables > enabled. > -- > > Key: HIVE-20967 > URL: https://issues.apache.org/jira/browse/HIVE-20967 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 4.0.0 >Reporter: mahesh kumar behera >Assignee: Ashutosh Bapat >Priority: Minor > Labels: DR > > Some of the events from Hive2 may cause conflicts in Hive3 > (hive.strict.managed.tables=true) when applied. So, need to handle them > properly. > 1. Alter table to convert non-acid to acid. > - This event should be no-op as the table in target might be already acid or > MM or external table. > 2. Alter table or partition that changes the location. > - For managed tables at source, the table location shouldn't be changed for > the given non-partitioned table and partition location shouldn't be changed > for partitioned table as alter event doesn't capture the new files list. So, > it may cause data inconsistsency. So, if database is enabled for replication > at source, then alter location on managed tables should be blocked. > - For external partitioned tables, if location is changed at source, the the > location should be changed for the table and any partitions which reside > within the table location, but not for the partitions which are not within > the table location. (may be we just need the test). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
