[jira] [Commented] (HIVE-22149) Metastore: Unify codahale metrics.log json structure between hiveserver2 and metastore services
[
https://issues.apache.org/jira/browse/HIVE-22149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924754#comment-16924754
]
Hive QA commented on HIVE-22149:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
30s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
2s{color} | {color:blue} standalone-metastore/metastore-server in master has
181 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
35s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
20s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 2 new + 439 unchanged - 1 fixed = 441 total (was 440) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18490/dev-support/hive-personality.sh
|
| git revision | master / 0213afb |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18490/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| modules | C: standalone-metastore/metastore-server itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18490/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Metastore: Unify codahale metrics.log json structure between hiveserver2 and
> metastore services
> ---
>
> Key: HIVE-22149
> URL: https://issues.apache.org/jira/browse/HIVE-22149
> Project: Hive
> Issue Type: Bug
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-22149.01.patch, HIVE-22149.01.patch,
> HIVE-22149.01.patch, HIVE-22149.02.patch, HIVE-22149.03.patch,
> metrics_hiveserver2.log, metrics_metastore.log
>
>
> While fixing HIVE-22140 I found some really annoying differences between the
> codahale metric file structures between hiveserver2 and metastore, e.g.
> open_connections: can be found in "counters" for hs2, but in "gauges" for ms
> threads count: it's a proper "threads.count" for hs2
[jira] [Updated] (HIVE-22149) Metastore: Unify codahale metrics.log json structure between hiveserver2 and metastore services
[ https://issues.apache.org/jira/browse/HIVE-22149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-22149: Attachment: HIVE-22149.03.patch > Metastore: Unify codahale metrics.log json structure between hiveserver2 and > metastore services > --- > > Key: HIVE-22149 > URL: https://issues.apache.org/jira/browse/HIVE-22149 > Project: Hive > Issue Type: Bug >Reporter: Laszlo Bodor >Assignee: Laszlo Bodor >Priority: Major > Attachments: HIVE-22149.01.patch, HIVE-22149.01.patch, > HIVE-22149.01.patch, HIVE-22149.02.patch, HIVE-22149.03.patch, > metrics_hiveserver2.log, metrics_metastore.log > > > While fixing HIVE-22140 I found some really annoying differences between the > codahale metric file structures between hiveserver2 and metastore, e.g. > open_connections: can be found in "counters" for hs2, but in "gauges" for ms > threads count: it's a proper "threads.count" for hs2, but a really ambiguous > "count" for ms > so I realized that "memory." and "threads." prefix is completely absent in ms > metrics file, which is misleading -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22178) Parquet FilterPredicate throws CastException after SchemaEvolution.
[
https://issues.apache.org/jira/browse/HIVE-22178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924737#comment-16924737
]
Hive QA commented on HIVE-22178:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979736/HIVE-22178.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 16746 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestObjectStore.catalogs (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDatabaseOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDeprecatedConfigIsOverwritten
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropParitionsCleanup
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropPartitionsCacheCrossSession
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSqlErrorMetrics
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testEmptyTrustStoreProps
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testMasterKeyOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testMaxEventResponse
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testPartitionOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testQueryCloseOnError
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testRoleOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testTableOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testUseSSLProperty
(batchId=233)
org.apache.hadoop.hive.ql.io.parquet.read.TestParquetFilterPredicate.testFilterColumnsThatDoNoExistOnSchema
(batchId=320)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18489/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18489/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18489/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 15 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979736 - PreCommit-HIVE-Build
> Parquet FilterPredicate throws CastException after SchemaEvolution.
> ---
>
> Key: HIVE-22178
> URL: https://issues.apache.org/jira/browse/HIVE-22178
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: Naresh P R
>Assignee: Naresh P R
>Priority: Major
> Attachments: HIVE-22178.1.patch
>
>
> Below are the repro steps.
> {code:java}
> create table parq_test(age int, name string) stored as parquet;
> insert into parq_test values(1, '');
> alter table parq_test change age age string;
> insert into parq_test values('b', '');
> select * from parq_test where age='b';{code}
> Exception thrown after changing column datatype is below
> {code:java}
> Caused by: java.lang.IllegalArgumentException: FilterPredicate column: age's
> declared type (org.apache.parquet.io.api.Binary) does not match the schema
> found in file metadata. Column age is of type: INT32
> Valid types for this column are: [class java.lang.Integer]
> at
> org.apache.parquet.filter2.predicate.ValidTypeMap.assertTypeValid(ValidTypeMap.java:126)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validateColumn(SchemaCompatibilityValidator.java:181)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validateColumnFilterPredicate(SchemaCompatibilityValidator.java:151)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.visit(SchemaCompatibilityValidator.java:85)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.visit(SchemaCompatibilityValidator.java:58)
> at
> org.apache.parquet.filter2.predicate.Operators$Eq.accept(Operators.java:181)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validate(SchemaCompatibilityValidator.java:63)
> at
> org.apache.parquet.filter2.compat.RowGroupFilter.visit(RowGroupFilter.java:92)
> at
> org.apache.parquet.filter2.compat.RowGroupFilter.visit(RowGroupFilter.java:43)
> at
> org.apache.parquet.filter2.compat.FilterCompat$FilterPredicateCompat.accept(FilterCompat.java:137)
> at
> org.apache.parquet.filter2.compat.RowGroupFilter.filterRowGroups(RowGroupFilter.java:64)
> at
> org.apache.hadoop.hive.ql.io.parquet.ParquetRecordReaderBase.getSplit(ParquetRecordReaderBa
[jira] [Commented] (HIVE-22178) Parquet FilterPredicate throws CastException after SchemaEvolution.
[
https://issues.apache.org/jira/browse/HIVE-22178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924727#comment-16924727
]
Hive QA commented on HIVE-22178:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
40s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
36s{color} | {color:red} ql: The patch generated 20 new + 60 unchanged - 12
fixed = 80 total (was 72) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 49s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18489/dev-support/hive-personality.sh
|
| git revision | master / 0213afb |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18489/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18489/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Parquet FilterPredicate throws CastException after SchemaEvolution.
> ---
>
> Key: HIVE-22178
> URL: https://issues.apache.org/jira/browse/HIVE-22178
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: Naresh P R
>Assignee: Naresh P R
>Priority: Major
> Attachments: HIVE-22178.1.patch
>
>
> Below are the repro steps.
> {code:java}
> create table parq_test(age int, name string) stored as parquet;
> insert into parq_test values(1, '');
> alter table parq_test change age age string;
> insert into parq_test values('b', '');
> select * from parq_test where age='b';{code}
> Exception thrown after changing column datatype is below
> {code:java}
> Caused by: java.lang.IllegalArgumentException: FilterPredicate column: age's
> declared type (org.apache.parquet.io.api.Binary) does not match the schema
> found in file metadata. Column age is of type: INT32
> Valid types for this column are: [class java.lang.Integer]
> at
> org.apache.parquet.filter2.predicate.ValidTypeMap.assertTypeValid(ValidTypeMap.java:126)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validateColumn(SchemaCompatibilityValidator.java:181)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validateColumnFilterPredicate(SchemaCompatibilityValidator.java:151)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityVali
[jira] [Commented] (HIVE-22170) from_unixtime and unix_timestamp should use user session time zone
[
https://issues.apache.org/jira/browse/HIVE-22170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924719#comment-16924719
]
Hive QA commented on HIVE-22170:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979734/HIVE-22170.02.patch
{color:green}SUCCESS:{color} +1 due to 22 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 19 failed/errored test(s), 16746 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[udf_unix_timestamp]
(batchId=297)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[date_udf] (batchId=34)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamp_udf]
(batchId=86)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_folder_constants]
(batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_to_unix_timestamp]
(batchId=62)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_unix_timestamp]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_date_funcs]
(batchId=84)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[current_date_timestamp]
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_date_funcs]
(batchId=183)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[date_udf]
(batchId=127)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[timestamp_udf]
(batchId=149)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorized_timestamp_funcs]
(batchId=127)
org.apache.hadoop.hive.ql.exec.vector.expressions.TestVectorDateExpressions.testVectorUDFUnixTimeStamp
(batchId=336)
org.apache.hadoop.hive.ql.exec.vector.expressions.TestVectorTimestampExpressions.testVectorUDFUnixTimeStampString
(batchId=337)
org.apache.hadoop.hive.ql.exec.vector.expressions.TestVectorTimestampExpressions.testVectorUDFUnixTimeStampTimestamp
(batchId=337)
org.apache.hadoop.hive.ql.udf.generic.TestGenericUDFToUnixTimestamp.testDate
(batchId=312)
org.apache.hadoop.hive.ql.udf.generic.TestGenericUDFToUnixTimestamp.testString
(batchId=312)
org.apache.hadoop.hive.ql.udf.generic.TestGenericUDFToUnixTimestamp.testTimestamp
(batchId=312)
org.apache.hive.jdbc.TestSSL.testSSLFetch (batchId=282)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18488/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18488/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18488/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 19 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979734 - PreCommit-HIVE-Build
> from_unixtime and unix_timestamp should use user session time zone
> --
>
> Key: HIVE-22170
> URL: https://issues.apache.org/jira/browse/HIVE-22170
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0, 4.0.0, 3.2.0, 3.1.1, 3.1.2
>Reporter: Riju Trivedi
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-22170.01.patch, HIVE-22170.02.patch
>
>
> According to documentation, that is the expected behavior (since session time
> zone was not present, system time zone was being used previously). This was
> incorrectly changed by HIVE-12192 / HIVE-20007. This JIRA should fix this
> issue.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Work started] (HIVE-22178) Parquet FilterPredicate throws CastException after SchemaEvolution.
[
https://issues.apache.org/jira/browse/HIVE-22178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-22178 started by Naresh P R.
-
> Parquet FilterPredicate throws CastException after SchemaEvolution.
> ---
>
> Key: HIVE-22178
> URL: https://issues.apache.org/jira/browse/HIVE-22178
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: Naresh P R
>Assignee: Naresh P R
>Priority: Major
> Attachments: HIVE-22178.1.patch
>
>
> Below are the repro steps.
> {code:java}
> create table parq_test(age int, name string) stored as parquet;
> insert into parq_test values(1, '');
> alter table parq_test change age age string;
> insert into parq_test values('b', '');
> select * from parq_test where age='b';{code}
> Exception thrown after changing column datatype is below
> {code:java}
> Caused by: java.lang.IllegalArgumentException: FilterPredicate column: age's
> declared type (org.apache.parquet.io.api.Binary) does not match the schema
> found in file metadata. Column age is of type: INT32
> Valid types for this column are: [class java.lang.Integer]
> at
> org.apache.parquet.filter2.predicate.ValidTypeMap.assertTypeValid(ValidTypeMap.java:126)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validateColumn(SchemaCompatibilityValidator.java:181)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validateColumnFilterPredicate(SchemaCompatibilityValidator.java:151)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.visit(SchemaCompatibilityValidator.java:85)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.visit(SchemaCompatibilityValidator.java:58)
> at
> org.apache.parquet.filter2.predicate.Operators$Eq.accept(Operators.java:181)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validate(SchemaCompatibilityValidator.java:63)
> at
> org.apache.parquet.filter2.compat.RowGroupFilter.visit(RowGroupFilter.java:92)
> at
> org.apache.parquet.filter2.compat.RowGroupFilter.visit(RowGroupFilter.java:43)
> at
> org.apache.parquet.filter2.compat.FilterCompat$FilterPredicateCompat.accept(FilterCompat.java:137)
> at
> org.apache.parquet.filter2.compat.RowGroupFilter.filterRowGroups(RowGroupFilter.java:64)
> at
> org.apache.hadoop.hive.ql.io.parquet.ParquetRecordReaderBase.getSplit(ParquetRecordReaderBase.java:111)
> at
> org.apache.hadoop.hive.ql.io.parquet.vector.VectorizedParquetRecordReader.(VectorizedParquetRecordReader.java:147)
> ... 31 more{code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22178) Parquet FilterPredicate throws CastException after SchemaEvolution.
[
https://issues.apache.org/jira/browse/HIVE-22178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naresh P R updated HIVE-22178:
--
Attachment: HIVE-22178.1.patch
Status: Patch Available (was: In Progress)
> Parquet FilterPredicate throws CastException after SchemaEvolution.
> ---
>
> Key: HIVE-22178
> URL: https://issues.apache.org/jira/browse/HIVE-22178
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: Naresh P R
>Assignee: Naresh P R
>Priority: Major
> Attachments: HIVE-22178.1.patch
>
>
> Below are the repro steps.
> {code:java}
> create table parq_test(age int, name string) stored as parquet;
> insert into parq_test values(1, '');
> alter table parq_test change age age string;
> insert into parq_test values('b', '');
> select * from parq_test where age='b';{code}
> Exception thrown after changing column datatype is below
> {code:java}
> Caused by: java.lang.IllegalArgumentException: FilterPredicate column: age's
> declared type (org.apache.parquet.io.api.Binary) does not match the schema
> found in file metadata. Column age is of type: INT32
> Valid types for this column are: [class java.lang.Integer]
> at
> org.apache.parquet.filter2.predicate.ValidTypeMap.assertTypeValid(ValidTypeMap.java:126)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validateColumn(SchemaCompatibilityValidator.java:181)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validateColumnFilterPredicate(SchemaCompatibilityValidator.java:151)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.visit(SchemaCompatibilityValidator.java:85)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.visit(SchemaCompatibilityValidator.java:58)
> at
> org.apache.parquet.filter2.predicate.Operators$Eq.accept(Operators.java:181)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validate(SchemaCompatibilityValidator.java:63)
> at
> org.apache.parquet.filter2.compat.RowGroupFilter.visit(RowGroupFilter.java:92)
> at
> org.apache.parquet.filter2.compat.RowGroupFilter.visit(RowGroupFilter.java:43)
> at
> org.apache.parquet.filter2.compat.FilterCompat$FilterPredicateCompat.accept(FilterCompat.java:137)
> at
> org.apache.parquet.filter2.compat.RowGroupFilter.filterRowGroups(RowGroupFilter.java:64)
> at
> org.apache.hadoop.hive.ql.io.parquet.ParquetRecordReaderBase.getSplit(ParquetRecordReaderBase.java:111)
> at
> org.apache.hadoop.hive.ql.io.parquet.vector.VectorizedParquetRecordReader.(VectorizedParquetRecordReader.java:147)
> ... 31 more{code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Assigned] (HIVE-22178) Parquet FilterPredicate throws CastException after SchemaEvolution.
[
https://issues.apache.org/jira/browse/HIVE-22178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naresh P R reassigned HIVE-22178:
-
> Parquet FilterPredicate throws CastException after SchemaEvolution.
> ---
>
> Key: HIVE-22178
> URL: https://issues.apache.org/jira/browse/HIVE-22178
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: Naresh P R
>Assignee: Naresh P R
>Priority: Major
>
> Below are the repro steps.
> {code:java}
> create table parq_test(age int, name string) stored as parquet;
> insert into parq_test values(1, '');
> alter table parq_test change age age string;
> insert into parq_test values('b', '');
> select * from parq_test where age='b';{code}
> Exception thrown after changing column datatype is below
> {code:java}
> Caused by: java.lang.IllegalArgumentException: FilterPredicate column: age's
> declared type (org.apache.parquet.io.api.Binary) does not match the schema
> found in file metadata. Column age is of type: INT32
> Valid types for this column are: [class java.lang.Integer]
> at
> org.apache.parquet.filter2.predicate.ValidTypeMap.assertTypeValid(ValidTypeMap.java:126)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validateColumn(SchemaCompatibilityValidator.java:181)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validateColumnFilterPredicate(SchemaCompatibilityValidator.java:151)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.visit(SchemaCompatibilityValidator.java:85)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.visit(SchemaCompatibilityValidator.java:58)
> at
> org.apache.parquet.filter2.predicate.Operators$Eq.accept(Operators.java:181)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validate(SchemaCompatibilityValidator.java:63)
> at
> org.apache.parquet.filter2.compat.RowGroupFilter.visit(RowGroupFilter.java:92)
> at
> org.apache.parquet.filter2.compat.RowGroupFilter.visit(RowGroupFilter.java:43)
> at
> org.apache.parquet.filter2.compat.FilterCompat$FilterPredicateCompat.accept(FilterCompat.java:137)
> at
> org.apache.parquet.filter2.compat.RowGroupFilter.filterRowGroups(RowGroupFilter.java:64)
> at
> org.apache.hadoop.hive.ql.io.parquet.ParquetRecordReaderBase.getSplit(ParquetRecordReaderBase.java:111)
> at
> org.apache.hadoop.hive.ql.io.parquet.vector.VectorizedParquetRecordReader.(VectorizedParquetRecordReader.java:147)
> ... 31 more{code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22170) from_unixtime and unix_timestamp should use user session time zone
[
https://issues.apache.org/jira/browse/HIVE-22170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924710#comment-16924710
]
Hive QA commented on HIVE-22170:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
42s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
8s{color} | {color:red} ql: The patch generated 16 new + 4141 unchanged - 3
fixed = 4157 total (was 4144) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
1s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
50s{color} | {color:red} ql generated 2 new + 2243 unchanged - 3 fixed = 2245
total (was 2246) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 14s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Class
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorUDFUnixTimeStampDate
defines non-transient non-serializable instance field date In
VectorUDFUnixTimeStampDate.java:instance field date In
VectorUDFUnixTimeStampDate.java |
| | Class
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorUDFUnixTimeStampTimestamp
defines non-transient non-serializable instance field timestamp In
VectorUDFUnixTimeStampTimestamp.java:instance field timestamp In
VectorUDFUnixTimeStampTimestamp.java |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18488/dev-support/hive-personality.sh
|
| git revision | master / 0213afb |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18488/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18488/yetus/whitespace-eol.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18488/yetus/new-findbugs-ql.html
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18488/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> from_unixtime and unix_timestamp should use user session time zone
> --
>
> Key: HIVE-22170
> URL: https://issues.apache.org/jira/browse/HIVE-22170
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0, 4.0.0, 3.2.0, 3.1.1, 3.1.2
>Reporter: Riju Trivedi
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-22170.01.patch, HIVE-22170.02.patch
>
>
> According to documentation, that is the expected behavior (since session time
> zone was not present, system
[jira] [Commented] (HIVE-22174) Clean up Drop Partition
[
https://issues.apache.org/jira/browse/HIVE-22174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924707#comment-16924707
]
Hive QA commented on HIVE-22174:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979720/HIVE-22174.02.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16746 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18487/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18487/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18487/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979720 - PreCommit-HIVE-Build
> Clean up Drop Partition
> ---
>
> Key: HIVE-22174
> URL: https://issues.apache.org/jira/browse/HIVE-22174
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available, refactor-ddl
> Fix For: 4.0.0
>
> Attachments: HIVE-22174.01.patch, HIVE-22174.02.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> There are many complex drop partition and drop partitions operations in
> Hive.java, most of them are not used by anyone inside or outside of Hive. The
> unused codes should be removed, and the remaining code should be as simple as
> possible, calling dropPartition(s) operation at the metastore client. Also
> the AlterTableDropPartitionDesc class should not be exposed here.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Work logged] (HIVE-22143) Break up DDLSemanticAnalyzer - extract Database related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22143?focusedWorklogId=308264&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-308264 ] ASF GitHub Bot logged work on HIVE-22143: - Author: ASF GitHub Bot Created on: 07/Sep/19 02:27 Start Date: 07/Sep/19 02:27 Worklog Time Spent: 10m Work Description: jcamachor commented on pull request #757: HIVE-22143 Break up DDLSemanticAnalyzer - extract Database related analyzers URL: https://github.com/apache/hive/pull/757#discussion_r321951704 ## File path: hcatalog/core/src/main/java/org/apache/hive/hcatalog/cli/SemanticAnalysis/HCatSemanticAnalyzer.java ## @@ -21,10 +21,10 @@ import org.apache.hadoop.hive.metastore.api.Database; import org.apache.hadoop.hive.ql.ddl.DDLDesc; import org.apache.hadoop.hive.ql.ddl.DDLWork; -import org.apache.hadoop.hive.ql.ddl.database.DescDatabaseDesc; -import org.apache.hadoop.hive.ql.ddl.database.DropDatabaseDesc; -import org.apache.hadoop.hive.ql.ddl.database.ShowDatabasesDesc; -import org.apache.hadoop.hive.ql.ddl.database.SwitchDatabaseDesc; +import org.apache.hadoop.hive.ql.ddl.database.desc.DescDatabaseDesc; +import org.apache.hadoop.hive.ql.ddl.database.drop.DropDatabaseDesc; +import org.apache.hadoop.hive.ql.ddl.database.show.ShowDatabasesDesc; +import org.apache.hadoop.hive.ql.ddl.database.switchdb.SwitchDatabaseDesc; Review comment: I did not realize about `switch`... `use` is fine. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 308264) Time Spent: 1h 20m (was: 1h 10m) > Break up DDLSemanticAnalyzer - extract Database related analyzers > - > > Key: HIVE-22143 > URL: https://issues.apache.org/jira/browse/HIVE-22143 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22143.01.patch, HIVE-22143.02.patch, > HIVE-22143.04.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #1: extract all the database related analyzers from DDLSemanticAnalyzer, > and move them under the new package. Also create the new internal framework. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work logged] (HIVE-22143) Break up DDLSemanticAnalyzer - extract Database related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22143?focusedWorklogId=308265&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-308265 ] ASF GitHub Bot logged work on HIVE-22143: - Author: ASF GitHub Bot Created on: 07/Sep/19 02:27 Start Date: 07/Sep/19 02:27 Worklog Time Spent: 10m Work Description: jcamachor commented on pull request #757: HIVE-22143 Break up DDLSemanticAnalyzer - extract Database related analyzers URL: https://github.com/apache/hive/pull/757#discussion_r321951718 ## File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/bootstrap/load/LoadDatabase.java ## @@ -21,9 +21,9 @@ import org.apache.hadoop.hive.metastore.api.Database; import org.apache.hadoop.hive.metastore.api.InvalidOperationException; import org.apache.hadoop.hive.ql.ddl.DDLWork; -import org.apache.hadoop.hive.ql.ddl.database.AlterDatabaseSetOwnerDesc; -import org.apache.hadoop.hive.ql.ddl.database.AlterDatabaseSetPropertiesDesc; -import org.apache.hadoop.hive.ql.ddl.database.CreateDatabaseDesc; +import org.apache.hadoop.hive.ql.ddl.database.create.CreateDatabaseDesc; +import org.apache.hadoop.hive.ql.ddl.database.setowner.AlterDatabaseSetOwnerDesc; Review comment: Yes, your proposal makes sense, let's have 3 subpackages. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 308265) Time Spent: 1.5h (was: 1h 20m) > Break up DDLSemanticAnalyzer - extract Database related analyzers > - > > Key: HIVE-22143 > URL: https://issues.apache.org/jira/browse/HIVE-22143 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22143.01.patch, HIVE-22143.02.patch, > HIVE-22143.04.patch > > Time Spent: 1.5h > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #1: extract all the database related analyzers from DDLSemanticAnalyzer, > and move them under the new package. Also create the new internal framework. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22170) from_unixtime and unix_timestamp should use user session time zone
[ https://issues.apache.org/jira/browse/HIVE-22170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22170: --- Attachment: HIVE-22170.02.patch > from_unixtime and unix_timestamp should use user session time zone > -- > > Key: HIVE-22170 > URL: https://issues.apache.org/jira/browse/HIVE-22170 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0, 4.0.0, 3.2.0, 3.1.1, 3.1.2 >Reporter: Riju Trivedi >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-22170.01.patch, HIVE-22170.02.patch > > > According to documentation, that is the expected behavior (since session time > zone was not present, system time zone was being used previously). This was > incorrectly changed by HIVE-12192 / HIVE-20007. This JIRA should fix this > issue. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22174) Clean up Drop Partition
[
https://issues.apache.org/jira/browse/HIVE-22174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924700#comment-16924700
]
Hive QA commented on HIVE-22174:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
36s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} ql: The patch generated 0 new + 192 unchanged - 5
fixed = 192 total (was 197) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
49s{color} | {color:green} ql generated 0 new + 2244 unchanged - 2 fixed = 2244
total (was 2246) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18487/dev-support/hive-personality.sh
|
| git revision | master / 0213afb |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18487/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Clean up Drop Partition
> ---
>
> Key: HIVE-22174
> URL: https://issues.apache.org/jira/browse/HIVE-22174
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available, refactor-ddl
> Fix For: 4.0.0
>
> Attachments: HIVE-22174.01.patch, HIVE-22174.02.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> There are many complex drop partition and drop partitions operations in
> Hive.java, most of them are not used by anyone inside or outside of Hive. The
> unused codes should be removed, and the remaining code should be as simple as
> possible, calling dropPartition(s) operation at the metastore client. Also
> the AlterTableDropPartitionDesc class should not be exposed here.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-20113) Shuffle avoidance: Disable 1-1 edges for sorted shuffle
[
https://issues.apache.org/jira/browse/HIVE-20113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924698#comment-16924698
]
Hive QA commented on HIVE-20113:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979717/HIVE-20113.6.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18486/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18486/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18486/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12979717/HIVE-20113.6.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979717 - PreCommit-HIVE-Build
> Shuffle avoidance: Disable 1-1 edges for sorted shuffle
>
>
> Key: HIVE-20113
> URL: https://issues.apache.org/jira/browse/HIVE-20113
> Project: Hive
> Issue Type: Bug
> Components: Tez
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Labels: Branch3Candidate
> Attachments: HIVE-20113.1.patch, HIVE-20113.2.patch,
> HIVE-20113.3.patch, HIVE-20113.4.patch, HIVE-20113.4.patch,
> HIVE-20113.5.patch, HIVE-20113.6.patch
>
>
> The sorted shuffle avoidance can have some issues when the shuffle data gets
> broken up into multiple chunks on disk.
> The 1-1 edge cannot skip the tez final merge - there's no reason for 1-1 to
> have a final merge at all, it should open a single compressed file and write
> a single index entry.
> Until the shuffle issue is resolved & a lot more testing, it is prudent to
> disable the optimization for sorted shuffle edges and stop rewriting the
> RS(sorted) = = = RS(sorted) into RS(sorted) = = = RS(FORWARD).
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21510) Vectorization: add support for and/or for (constant,column) cases
[
https://issues.apache.org/jira/browse/HIVE-21510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924696#comment-16924696
]
Hive QA commented on HIVE-21510:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979716/HIVE-21510.04.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16747 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18485/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18485/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18485/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979716 - PreCommit-HIVE-Build
> Vectorization: add support for and/or for (constant,column) cases
> -
>
> Key: HIVE-21510
> URL: https://issues.apache.org/jira/browse/HIVE-21510
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-21510.01.patch, HIVE-21510.02.patch,
> HIVE-21510.03.patch, HIVE-21510.04.patch
>
>
> After HIVE-21001 some selectExpressions will start using VectorUDFAdaptor for
> "null and x" expressions. Because right now there are 2-3 places from which
> rewrite will be done to the form of "null and/or x" form; it would be better
> to support it.
> {code}
> [...]
> selectExpressions: VectorUDFAdaptor((null and dt1 is null))
> [...]
> usesVectorUDFAdaptor: true
> [...]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21510) Vectorization: add support for and/or for (constant,column) cases
[
https://issues.apache.org/jira/browse/HIVE-21510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924688#comment-16924688
]
Hive QA commented on HIVE-21510:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
39s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
22s{color} | {color:blue} storage-api in master has 48 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
40s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
40s{color} | {color:red} ql: The patch generated 6 new + 475 unchanged - 4
fixed = 481 total (was 479) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 18s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18485/dev-support/hive-personality.sh
|
| git revision | master / 0213afb |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18485/yetus/diff-checkstyle-ql.txt
|
| modules | C: storage-api ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18485/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Vectorization: add support for and/or for (constant,column) cases
> -
>
> Key: HIVE-21510
> URL: https://issues.apache.org/jira/browse/HIVE-21510
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-21510.01.patch, HIVE-21510.02.patch,
> HIVE-21510.03.patch, HIVE-21510.04.patch
>
>
> After HIVE-21001 some selectExpressions will start using VectorUDFAdaptor for
> "null and x" expressions. Because right now there are 2-3 places from which
> rewrite will be done to the form of "null and/or x" form; it would be better
> to support it.
> {code}
> [...]
> selectExpressions: VectorUDFAdaptor((null and dt1 is null))
> [...]
> usesVectorUDFAdaptor: true
> [...]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-20113) Shuffle avoidance: Disable 1-1 edges for sorted shuffle
[
https://issues.apache.org/jira/browse/HIVE-20113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924676#comment-16924676
]
Hive QA commented on HIVE-20113:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979717/HIVE-20113.6.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 16746 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropParitionsCleanup
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropPartitionsCacheCrossSession
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testPartitionOps (batchId=233)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18484/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18484/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18484/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979717 - PreCommit-HIVE-Build
> Shuffle avoidance: Disable 1-1 edges for sorted shuffle
>
>
> Key: HIVE-20113
> URL: https://issues.apache.org/jira/browse/HIVE-20113
> Project: Hive
> Issue Type: Bug
> Components: Tez
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Labels: Branch3Candidate
> Attachments: HIVE-20113.1.patch, HIVE-20113.2.patch,
> HIVE-20113.3.patch, HIVE-20113.4.patch, HIVE-20113.4.patch,
> HIVE-20113.5.patch, HIVE-20113.6.patch
>
>
> The sorted shuffle avoidance can have some issues when the shuffle data gets
> broken up into multiple chunks on disk.
> The 1-1 edge cannot skip the tez final merge - there's no reason for 1-1 to
> have a final merge at all, it should open a single compressed file and write
> a single index entry.
> Until the shuffle issue is resolved & a lot more testing, it is prudent to
> disable the optimization for sorted shuffle edges and stop rewriting the
> RS(sorted) = = = RS(sorted) into RS(sorted) = = = RS(FORWARD).
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-20113) Shuffle avoidance: Disable 1-1 edges for sorted shuffle
[
https://issues.apache.org/jira/browse/HIVE-20113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924658#comment-16924658
]
Hive QA commented on HIVE-20113:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
39s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 1 new + 347 unchanged - 1
fixed = 348 total (was 348) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 3s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18484/dev-support/hive-personality.sh
|
| git revision | master / 0213afb |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18484/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18484/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Shuffle avoidance: Disable 1-1 edges for sorted shuffle
>
>
> Key: HIVE-20113
> URL: https://issues.apache.org/jira/browse/HIVE-20113
> Project: Hive
> Issue Type: Bug
> Components: Tez
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Labels: Branch3Candidate
> Attachments: HIVE-20113.1.patch, HIVE-20113.2.patch,
> HIVE-20113.3.patch, HIVE-20113.4.patch, HIVE-20113.4.patch,
> HIVE-20113.5.patch, HIVE-20113.6.patch
>
>
> The sorted shuffle avoidance can have some issues when the shuffle data gets
> broken up into multiple chunks on disk.
> The 1-1 edge cannot skip the tez final merge - there's no reason for 1-1 to
> have a final merge at all, it should open a single compressed file and write
> a single index entry.
> Until the shuffle issue is resolved & a lot more testing, it is prudent to
> disable the optimization for sorted shuffle edges and stop rewriting the
> RS(sorted) = = = RS(sorted) into RS(sorted) = = = RS(FORWARD).
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22149) Metastore: Unify codahale metrics.log json structure between hiveserver2 and metastore services
[
https://issues.apache.org/jira/browse/HIVE-22149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924652#comment-16924652
]
Hive QA commented on HIVE-22149:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979714/HIVE-22149.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16747 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestMetaStoreMetrics.testConnections
(batchId=242)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18483/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18483/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18483/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979714 - PreCommit-HIVE-Build
> Metastore: Unify codahale metrics.log json structure between hiveserver2 and
> metastore services
> ---
>
> Key: HIVE-22149
> URL: https://issues.apache.org/jira/browse/HIVE-22149
> Project: Hive
> Issue Type: Bug
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-22149.01.patch, HIVE-22149.01.patch,
> HIVE-22149.01.patch, HIVE-22149.02.patch, metrics_hiveserver2.log,
> metrics_metastore.log
>
>
> While fixing HIVE-22140 I found some really annoying differences between the
> codahale metric file structures between hiveserver2 and metastore, e.g.
> open_connections: can be found in "counters" for hs2, but in "gauges" for ms
> threads count: it's a proper "threads.count" for hs2, but a really ambiguous
> "count" for ms
> so I realized that "memory." and "threads." prefix is completely absent in ms
> metrics file, which is misleading
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22149) Metastore: Unify codahale metrics.log json structure between hiveserver2 and metastore services
[
https://issues.apache.org/jira/browse/HIVE-22149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924631#comment-16924631
]
Hive QA commented on HIVE-22149:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
7s{color} | {color:blue} standalone-metastore/metastore-server in master has
181 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
20s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 2 new + 439 unchanged - 1 fixed = 441 total (was 440) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 29s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18483/dev-support/hive-personality.sh
|
| git revision | master / 0213afb |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18483/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18483/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Metastore: Unify codahale metrics.log json structure between hiveserver2 and
> metastore services
> ---
>
> Key: HIVE-22149
> URL: https://issues.apache.org/jira/browse/HIVE-22149
> Project: Hive
> Issue Type: Bug
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-22149.01.patch, HIVE-22149.01.patch,
> HIVE-22149.01.patch, HIVE-22149.02.patch, metrics_hiveserver2.log,
> metrics_metastore.log
>
>
> While fixing HIVE-22140 I found some really annoying differences between the
> codahale metric file structures between hiveserver2 and metastore, e.g.
> open_connections: can be found in "counters" for hs2, but in "gauges" for ms
> threads count: it's a proper "threads.count" for hs2, but a really ambiguous
> "count" for ms
> so I realized that "memory." and "threads." prefix is completely absent in ms
> metrics file, which is misleading
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22174) Clean up Drop Partition
[ https://issues.apache.org/jira/browse/HIVE-22174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22174: -- Attachment: HIVE-22174.02.patch > Clean up Drop Partition > --- > > Key: HIVE-22174 > URL: https://issues.apache.org/jira/browse/HIVE-22174 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Fix For: 4.0.0 > > Attachments: HIVE-22174.01.patch, HIVE-22174.02.patch > > Time Spent: 10m > Remaining Estimate: 0h > > There are many complex drop partition and drop partitions operations in > Hive.java, most of them are not used by anyone inside or outside of Hive. The > unused codes should be removed, and the remaining code should be as simple as > possible, calling dropPartition(s) operation at the metastore client. Also > the AlterTableDropPartitionDesc class should not be exposed here. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-21510) Vectorization: add support for and/or for (constant,column) cases
[
https://issues.apache.org/jira/browse/HIVE-21510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924628#comment-16924628
]
Hive QA commented on HIVE-21510:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979705/HIVE-21510.03.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16747 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats_part2] (batchId=22)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18482/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18482/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18482/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979705 - PreCommit-HIVE-Build
> Vectorization: add support for and/or for (constant,column) cases
> -
>
> Key: HIVE-21510
> URL: https://issues.apache.org/jira/browse/HIVE-21510
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-21510.01.patch, HIVE-21510.02.patch,
> HIVE-21510.03.patch, HIVE-21510.04.patch
>
>
> After HIVE-21001 some selectExpressions will start using VectorUDFAdaptor for
> "null and x" expressions. Because right now there are 2-3 places from which
> rewrite will be done to the form of "null and/or x" form; it would be better
> to support it.
> {code}
> [...]
> selectExpressions: VectorUDFAdaptor((null and dt1 is null))
> [...]
> usesVectorUDFAdaptor: true
> [...]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-20113) Shuffle avoidance: Disable 1-1 edges for sorted shuffle
[ https://issues.apache.org/jira/browse/HIVE-20113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20113: --- Attachment: (was: HIVE-20113.6.patch) > Shuffle avoidance: Disable 1-1 edges for sorted shuffle > > > Key: HIVE-20113 > URL: https://issues.apache.org/jira/browse/HIVE-20113 > Project: Hive > Issue Type: Bug > Components: Tez >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Labels: Branch3Candidate > Attachments: HIVE-20113.1.patch, HIVE-20113.2.patch, > HIVE-20113.3.patch, HIVE-20113.4.patch, HIVE-20113.4.patch, > HIVE-20113.5.patch, HIVE-20113.6.patch > > > The sorted shuffle avoidance can have some issues when the shuffle data gets > broken up into multiple chunks on disk. > The 1-1 edge cannot skip the tez final merge - there's no reason for 1-1 to > have a final merge at all, it should open a single compressed file and write > a single index entry. > Until the shuffle issue is resolved & a lot more testing, it is prudent to > disable the optimization for sorted shuffle edges and stop rewriting the > RS(sorted) = = = RS(sorted) into RS(sorted) = = = RS(FORWARD). -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-20113) Shuffle avoidance: Disable 1-1 edges for sorted shuffle
[ https://issues.apache.org/jira/browse/HIVE-20113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20113: --- Attachment: HIVE-20113.6.patch > Shuffle avoidance: Disable 1-1 edges for sorted shuffle > > > Key: HIVE-20113 > URL: https://issues.apache.org/jira/browse/HIVE-20113 > Project: Hive > Issue Type: Bug > Components: Tez >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Labels: Branch3Candidate > Attachments: HIVE-20113.1.patch, HIVE-20113.2.patch, > HIVE-20113.3.patch, HIVE-20113.4.patch, HIVE-20113.4.patch, > HIVE-20113.5.patch, HIVE-20113.6.patch > > > The sorted shuffle avoidance can have some issues when the shuffle data gets > broken up into multiple chunks on disk. > The 1-1 edge cannot skip the tez final merge - there's no reason for 1-1 to > have a final merge at all, it should open a single compressed file and write > a single index entry. > Until the shuffle issue is resolved & a lot more testing, it is prudent to > disable the optimization for sorted shuffle edges and stop rewriting the > RS(sorted) = = = RS(sorted) into RS(sorted) = = = RS(FORWARD). -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-21510) Vectorization: add support for and/or for (constant,column) cases
[
https://issues.apache.org/jira/browse/HIVE-21510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924602#comment-16924602
]
Laszlo Bodor commented on HIVE-21510:
-
need to fix asf license headers
> Vectorization: add support for and/or for (constant,column) cases
> -
>
> Key: HIVE-21510
> URL: https://issues.apache.org/jira/browse/HIVE-21510
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-21510.01.patch, HIVE-21510.02.patch,
> HIVE-21510.03.patch, HIVE-21510.04.patch
>
>
> After HIVE-21001 some selectExpressions will start using VectorUDFAdaptor for
> "null and x" expressions. Because right now there are 2-3 places from which
> rewrite will be done to the form of "null and/or x" form; it would be better
> to support it.
> {code}
> [...]
> selectExpressions: VectorUDFAdaptor((null and dt1 is null))
> [...]
> usesVectorUDFAdaptor: true
> [...]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22168) remove excessive logging by llap cache.
[ https://issues.apache.org/jira/browse/HIVE-22168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924601#comment-16924601 ] Jesus Camacho Rodriguez commented on HIVE-22168: +1 > remove excessive logging by llap cache. > --- > > Key: HIVE-22168 > URL: https://issues.apache.org/jira/browse/HIVE-22168 > Project: Hive > Issue Type: Improvement > Components: llap, Logging >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Attachments: HIVE-22168.patch, HIVE-22168.patch, HIVE-22168.patch > > > Llap cache logging is very expensive when it comes to log every request > buffers range. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-21510) Vectorization: add support for and/or for (constant,column) cases
[
https://issues.apache.org/jira/browse/HIVE-21510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Bodor updated HIVE-21510:
Attachment: HIVE-21510.04.patch
> Vectorization: add support for and/or for (constant,column) cases
> -
>
> Key: HIVE-21510
> URL: https://issues.apache.org/jira/browse/HIVE-21510
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-21510.01.patch, HIVE-21510.02.patch,
> HIVE-21510.03.patch, HIVE-21510.04.patch
>
>
> After HIVE-21001 some selectExpressions will start using VectorUDFAdaptor for
> "null and x" expressions. Because right now there are 2-3 places from which
> rewrite will be done to the form of "null and/or x" form; it would be better
> to support it.
> {code}
> [...]
> selectExpressions: VectorUDFAdaptor((null and dt1 is null))
> [...]
> usesVectorUDFAdaptor: true
> [...]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21510) Vectorization: add support for and/or for (constant,column) cases
[
https://issues.apache.org/jira/browse/HIVE-21510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924591#comment-16924591
]
Hive QA commented on HIVE-21510:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
19s{color} | {color:blue} storage-api in master has 48 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
43s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
39s{color} | {color:red} ql: The patch generated 6 new + 475 unchanged - 4
fixed = 481 total (was 479) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18482/dev-support/hive-personality.sh
|
| git revision | master / 0213afb |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18482/yetus/diff-checkstyle-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18482/yetus/patch-asflicense-problems.txt
|
| modules | C: storage-api ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18482/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Vectorization: add support for and/or for (constant,column) cases
> -
>
> Key: HIVE-21510
> URL: https://issues.apache.org/jira/browse/HIVE-21510
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-21510.01.patch, HIVE-21510.02.patch,
> HIVE-21510.03.patch
>
>
> After HIVE-21001 some selectExpressions will start using VectorUDFAdaptor for
> "null and x" expressions. Because right now there are 2-3 places from which
> rewrite will be done to the form of "null and/or x" form; it would be better
> to support it.
> {code}
> [...]
> selectExpressions: VectorUDFAdaptor((null and dt1 is null))
> [...]
> usesVectorUDFAdaptor: true
> [...]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-20113) Shuffle avoidance: Disable 1-1 edges for sorted shuffle
[ https://issues.apache.org/jira/browse/HIVE-20113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20113: --- Attachment: HIVE-20113.6.patch > Shuffle avoidance: Disable 1-1 edges for sorted shuffle > > > Key: HIVE-20113 > URL: https://issues.apache.org/jira/browse/HIVE-20113 > Project: Hive > Issue Type: Bug > Components: Tez >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Labels: Branch3Candidate > Attachments: HIVE-20113.1.patch, HIVE-20113.2.patch, > HIVE-20113.3.patch, HIVE-20113.4.patch, HIVE-20113.4.patch, > HIVE-20113.5.patch, HIVE-20113.6.patch > > > The sorted shuffle avoidance can have some issues when the shuffle data gets > broken up into multiple chunks on disk. > The 1-1 edge cannot skip the tez final merge - there's no reason for 1-1 to > have a final merge at all, it should open a single compressed file and write > a single index entry. > Until the shuffle issue is resolved & a lot more testing, it is prudent to > disable the optimization for sorted shuffle edges and stop rewriting the > RS(sorted) = = = RS(sorted) into RS(sorted) = = = RS(FORWARD). -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-20113) Shuffle avoidance: Disable 1-1 edges for sorted shuffle
[ https://issues.apache.org/jira/browse/HIVE-20113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20113: --- Status: Patch Available (was: Open) > Shuffle avoidance: Disable 1-1 edges for sorted shuffle > > > Key: HIVE-20113 > URL: https://issues.apache.org/jira/browse/HIVE-20113 > Project: Hive > Issue Type: Bug > Components: Tez >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Labels: Branch3Candidate > Attachments: HIVE-20113.1.patch, HIVE-20113.2.patch, > HIVE-20113.3.patch, HIVE-20113.4.patch, HIVE-20113.4.patch, > HIVE-20113.5.patch, HIVE-20113.6.patch > > > The sorted shuffle avoidance can have some issues when the shuffle data gets > broken up into multiple chunks on disk. > The 1-1 edge cannot skip the tez final merge - there's no reason for 1-1 to > have a final merge at all, it should open a single compressed file and write > a single index entry. > Until the shuffle issue is resolved & a lot more testing, it is prudent to > disable the optimization for sorted shuffle edges and stop rewriting the > RS(sorted) = = = RS(sorted) into RS(sorted) = = = RS(FORWARD). -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-20113) Shuffle avoidance: Disable 1-1 edges for sorted shuffle
[ https://issues.apache.org/jira/browse/HIVE-20113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20113: --- Status: Open (was: Patch Available) > Shuffle avoidance: Disable 1-1 edges for sorted shuffle > > > Key: HIVE-20113 > URL: https://issues.apache.org/jira/browse/HIVE-20113 > Project: Hive > Issue Type: Bug > Components: Tez >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Labels: Branch3Candidate > Attachments: HIVE-20113.1.patch, HIVE-20113.2.patch, > HIVE-20113.3.patch, HIVE-20113.4.patch, HIVE-20113.4.patch, > HIVE-20113.5.patch, HIVE-20113.6.patch > > > The sorted shuffle avoidance can have some issues when the shuffle data gets > broken up into multiple chunks on disk. > The 1-1 edge cannot skip the tez final merge - there's no reason for 1-1 to > have a final merge at all, it should open a single compressed file and write > a single index entry. > Until the shuffle issue is resolved & a lot more testing, it is prudent to > disable the optimization for sorted shuffle edges and stop rewriting the > RS(sorted) = = = RS(sorted) into RS(sorted) = = = RS(FORWARD). -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22149) Metastore: Unify codahale metrics.log json structure between hiveserver2 and metastore services
[ https://issues.apache.org/jira/browse/HIVE-22149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-22149: Attachment: HIVE-22149.02.patch > Metastore: Unify codahale metrics.log json structure between hiveserver2 and > metastore services > --- > > Key: HIVE-22149 > URL: https://issues.apache.org/jira/browse/HIVE-22149 > Project: Hive > Issue Type: Bug >Reporter: Laszlo Bodor >Assignee: Laszlo Bodor >Priority: Major > Attachments: HIVE-22149.01.patch, HIVE-22149.01.patch, > HIVE-22149.01.patch, HIVE-22149.02.patch, metrics_hiveserver2.log, > metrics_metastore.log > > > While fixing HIVE-22140 I found some really annoying differences between the > codahale metric file structures between hiveserver2 and metastore, e.g. > open_connections: can be found in "counters" for hs2, but in "gauges" for ms > threads count: it's a proper "threads.count" for hs2, but a really ambiguous > "count" for ms > so I realized that "memory." and "threads." prefix is completely absent in ms > metrics file, which is misleading -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22099) Several date related UDFs can't handle Julian dates properly since HIVE-20007
[
https://issues.apache.org/jira/browse/HIVE-22099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924576#comment-16924576
]
Hive QA commented on HIVE-22099:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979701/HIVE-22099.9.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18481/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18481/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18481/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12979701/HIVE-22099.9.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979701 - PreCommit-HIVE-Build
> Several date related UDFs can't handle Julian dates properly since HIVE-20007
> -
>
> Key: HIVE-22099
> URL: https://issues.apache.org/jira/browse/HIVE-22099
> Project: Hive
> Issue Type: Bug
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Attachments: HIVE-22099.0.patch, HIVE-22099.1.patch,
> HIVE-22099.2.patch, HIVE-22099.3.patch, HIVE-22099.4.patch,
> HIVE-22099.5.patch, HIVE-22099.6.patch, HIVE-22099.7.patch,
> HIVE-22099.8.patch, HIVE-22099.9.patch
>
>
> Currently dates that belong to Julian calendar (before Oct 15, 1582) are
> handled improperly by date/timestamp UDFs.
> E.g. DateFormat UDF:
> Although the dates are in Julian calendar, the formatter insists to print
> these according to Gregorian calendar causing multiple days of difference in
> some cases:
>
> {code:java}
> beeline> select date_format('1001-01-05','dd---MM--');
> ++
> | _c0 |
> ++
> | 30---12--1000 |
> ++{code}
> I've observed similar problems in the following UDFs:
> * add_months
> * date_format
> * day
> * month
> * months_between
> * weekofyear
> * year
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-19456) Beeline dfs command Error: Error while processing statement: (state=,code=1)
[
https://issues.apache.org/jira/browse/HIVE-19456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924574#comment-16924574
]
Hive QA commented on HIVE-19456:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12922419/HIVE-19456.0.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18480/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18480/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18480/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2019-09-06 20:23:30.254
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-18480/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2019-09-06 20:23:30.257
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 0213afb HIVE-22028 Clean up Add Partition (Miklos Gergely
reviewd by Jesus Camacho Rodriguez)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 0213afb HIVE-22028 Clean up Add Partition (Miklos Gergely
reviewd by Jesus Camacho Rodriguez)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2019-09-06 20:23:31.336
+ rm -rf ../yetus_PreCommit-HIVE-Build-18480
+ mkdir ../yetus_PreCommit-HIVE-Build-18480
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-18480
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-18480/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: a/ql/src/java/org/apache/hadoop/hive/ql/processors/DfsProcessor.java:
does not exist in index
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/processors/DfsProcessor.java:18
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/processors/DfsProcessor.java' with
conflicts.
Going to apply patch with: git apply -p1
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/processors/DfsProcessor.java:18
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/processors/DfsProcessor.java' with
conflicts.
U ql/src/java/org/apache/hadoop/hive/ql/processors/DfsProcessor.java
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-18480
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12922419 - PreCommit-HIVE-Build
> Beeline dfs command Error: Error while processing statement: (state=,code=1)
>
>
> Key: HIVE-19456
> URL: https://issues.apache.org/jira/browse/HIVE-19456
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 1.2.1
>Reporter: chuanjie.duan
>Assignee: chuanjie.duan
>Priority: Major
> Attachments: HIVE-19456.0.patch
>
>
> Reproduce steps:
> # hdfs dfs -chmod 750 /tmp/aaa (user:test)
> # beeline login with user test1
> # beeline "dfs -ls /tmp/aaa"
> # Error: Error while processing statement: (state=,code=1)
> Cause:
> beeline "dfs" command would call fsshell, fsshell just err.print(errmsg) and
> return "null" value when catching exception
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22099) Several date related UDFs can't handle Julian dates properly since HIVE-20007
[
https://issues.apache.org/jira/browse/HIVE-22099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924573#comment-16924573
]
Hive QA commented on HIVE-22099:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979701/HIVE-22099.9.patch
{color:green}SUCCESS:{color} +1 due to 7 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16747 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18479/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18479/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18479/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979701 - PreCommit-HIVE-Build
> Several date related UDFs can't handle Julian dates properly since HIVE-20007
> -
>
> Key: HIVE-22099
> URL: https://issues.apache.org/jira/browse/HIVE-22099
> Project: Hive
> Issue Type: Bug
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Attachments: HIVE-22099.0.patch, HIVE-22099.1.patch,
> HIVE-22099.2.patch, HIVE-22099.3.patch, HIVE-22099.4.patch,
> HIVE-22099.5.patch, HIVE-22099.6.patch, HIVE-22099.7.patch,
> HIVE-22099.8.patch, HIVE-22099.9.patch
>
>
> Currently dates that belong to Julian calendar (before Oct 15, 1582) are
> handled improperly by date/timestamp UDFs.
> E.g. DateFormat UDF:
> Although the dates are in Julian calendar, the formatter insists to print
> these according to Gregorian calendar causing multiple days of difference in
> some cases:
>
> {code:java}
> beeline> select date_format('1001-01-05','dd---MM--');
> ++
> | _c0 |
> ++
> | 30---12--1000 |
> ++{code}
> I've observed similar problems in the following UDFs:
> * add_months
> * date_format
> * day
> * month
> * months_between
> * weekofyear
> * year
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22099) Several date related UDFs can't handle Julian dates properly since HIVE-20007
[
https://issues.apache.org/jira/browse/HIVE-22099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924548#comment-16924548
]
Hive QA commented on HIVE-22099:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
41s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} ql: The patch generated 0 new + 206 unchanged - 3
fixed = 206 total (was 209) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 44s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18479/dev-support/hive-personality.sh
|
| git revision | master / 0213afb |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18479/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Several date related UDFs can't handle Julian dates properly since HIVE-20007
> -
>
> Key: HIVE-22099
> URL: https://issues.apache.org/jira/browse/HIVE-22099
> Project: Hive
> Issue Type: Bug
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Attachments: HIVE-22099.0.patch, HIVE-22099.1.patch,
> HIVE-22099.2.patch, HIVE-22099.3.patch, HIVE-22099.4.patch,
> HIVE-22099.5.patch, HIVE-22099.6.patch, HIVE-22099.7.patch,
> HIVE-22099.8.patch, HIVE-22099.9.patch
>
>
> Currently dates that belong to Julian calendar (before Oct 15, 1582) are
> handled improperly by date/timestamp UDFs.
> E.g. DateFormat UDF:
> Although the dates are in Julian calendar, the formatter insists to print
> these according to Gregorian calendar causing multiple days of difference in
> some cases:
>
> {code:java}
> beeline> select date_format('1001-01-05','dd---MM--');
> ++
> | _c0 |
> ++
> | 30---12--1000 |
> ++{code}
> I've observed similar problems in the following UDFs:
> * add_months
> * date_format
> * day
> * month
> * months_between
> * weekofyear
> * year
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22164) Vectorized Limit operator returns wrong number of results with offset
[ https://issues.apache.org/jira/browse/HIVE-22164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924536#comment-16924536 ] Ramesh Kumar Thangarajan commented on HIVE-22164: - Hi [~vgarg], Can you please help me review the pull request [https://github.com/apache/hive/pull/762/files]? The issue here is an off-by-one error in the vector limit operator when there is an offset value. > Vectorized Limit operator returns wrong number of results with offset > - > > Key: HIVE-22164 > URL: https://issues.apache.org/jira/browse/HIVE-22164 > Project: Hive > Issue Type: Bug > Components: Hive, llap, Vectorization >Affects Versions: 4.0.0 >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Attachments: HIVE-22164.1.patch > > > Vectorized Limit operator returns wrong number of results with offset -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22174) Clean up Drop Partition
[
https://issues.apache.org/jira/browse/HIVE-22174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924525#comment-16924525
]
Hive QA commented on HIVE-22174:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979652/HIVE-22174.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16746 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.security.TestAuthorizationPreEventListener.testListener
(batchId=276)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18478/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18478/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18478/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979652 - PreCommit-HIVE-Build
> Clean up Drop Partition
> ---
>
> Key: HIVE-22174
> URL: https://issues.apache.org/jira/browse/HIVE-22174
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available, refactor-ddl
> Fix For: 4.0.0
>
> Attachments: HIVE-22174.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> There are many complex drop partition and drop partitions operations in
> Hive.java, most of them are not used by anyone inside or outside of Hive. The
> unused codes should be removed, and the remaining code should be as simple as
> possible, calling dropPartition(s) operation at the metastore client. Also
> the AlterTableDropPartitionDesc class should not be exposed here.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21510) Vectorization: add support for and/or for (constant,column) cases
[
https://issues.apache.org/jira/browse/HIVE-21510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Bodor updated HIVE-21510:
Attachment: HIVE-21510.03.patch
> Vectorization: add support for and/or for (constant,column) cases
> -
>
> Key: HIVE-21510
> URL: https://issues.apache.org/jira/browse/HIVE-21510
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-21510.01.patch, HIVE-21510.02.patch,
> HIVE-21510.03.patch
>
>
> After HIVE-21001 some selectExpressions will start using VectorUDFAdaptor for
> "null and x" expressions. Because right now there are 2-3 places from which
> rewrite will be done to the form of "null and/or x" form; it would be better
> to support it.
> {code}
> [...]
> selectExpressions: VectorUDFAdaptor((null and dt1 is null))
> [...]
> usesVectorUDFAdaptor: true
> [...]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22150) HS2 allows setting system properties
[
https://issues.apache.org/jira/browse/HIVE-22150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924495#comment-16924495
]
Alan Gates commented on HIVE-22150:
---
Looks good to me. Any feedback [~thejas], [~vaibhgup] or others who have
worked more on HS2 than me?
> HS2 allows setting system properties
>
>
> Key: HIVE-22150
> URL: https://issues.apache.org/jira/browse/HIVE-22150
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 4.0.0, 3.1.1
>Reporter: Craig Condit
>Assignee: Hui An
>Priority: Major
> Attachments: HIVE-22150.patch.1, HIVE-22150.patch.2
>
>
> HiveServer2 currently allows setting system properties, which is a problem
> when used in a multi-user environment.
> Connecting via beeline and executing the following demonstrates the issue:
> {noformat}
> 0: jdbc:hive2://serv1000.example.com:2181,serv> SET system:java.io.tmpdir;
> +-+
> | set |
> +-+
> | system:java.io.tmpdir=/tmp |
> +-+
> 1 row selected (0.018 seconds)
> 0: jdbc:hive2://serv1000.example.com:2181,serv> SET
> system:java.io.tmpdir=/tmp/attacker-dir;
> No rows affected (0.013 seconds)
> 0: jdbc:hive2://serv1000.example.com:2181,serv> SET system:java.io.tmpdir;
> +--+
> | set|
> +--+
> | system:java.io.tmpdir=/tmp/attacker-dir |
> +--+
> 1 row selected (0.019 seconds)
> {noformat}
> Any changes persist until HS2 is restarted, and affect all connected users.
> At the very least, this is a denial-of-service vector (verified by setting
> line.separator to a random string).
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22099) Several date related UDFs can't handle Julian dates properly since HIVE-20007
[
https://issues.apache.org/jira/browse/HIVE-22099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Szita updated HIVE-22099:
--
Attachment: HIVE-22099.9.patch
> Several date related UDFs can't handle Julian dates properly since HIVE-20007
> -
>
> Key: HIVE-22099
> URL: https://issues.apache.org/jira/browse/HIVE-22099
> Project: Hive
> Issue Type: Bug
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Attachments: HIVE-22099.0.patch, HIVE-22099.1.patch,
> HIVE-22099.2.patch, HIVE-22099.3.patch, HIVE-22099.4.patch,
> HIVE-22099.5.patch, HIVE-22099.6.patch, HIVE-22099.7.patch,
> HIVE-22099.8.patch, HIVE-22099.9.patch
>
>
> Currently dates that belong to Julian calendar (before Oct 15, 1582) are
> handled improperly by date/timestamp UDFs.
> E.g. DateFormat UDF:
> Although the dates are in Julian calendar, the formatter insists to print
> these according to Gregorian calendar causing multiple days of difference in
> some cases:
>
> {code:java}
> beeline> select date_format('1001-01-05','dd---MM--');
> ++
> | _c0 |
> ++
> | 30---12--1000 |
> ++{code}
> I've observed similar problems in the following UDFs:
> * add_months
> * date_format
> * day
> * month
> * months_between
> * weekofyear
> * year
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22099) Several date related UDFs can't handle Julian dates properly since HIVE-20007
[
https://issues.apache.org/jira/browse/HIVE-22099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Szita updated HIVE-22099:
--
Status: Patch Available (was: In Progress)
> Several date related UDFs can't handle Julian dates properly since HIVE-20007
> -
>
> Key: HIVE-22099
> URL: https://issues.apache.org/jira/browse/HIVE-22099
> Project: Hive
> Issue Type: Bug
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Attachments: HIVE-22099.0.patch, HIVE-22099.1.patch,
> HIVE-22099.2.patch, HIVE-22099.3.patch, HIVE-22099.4.patch,
> HIVE-22099.5.patch, HIVE-22099.6.patch, HIVE-22099.7.patch,
> HIVE-22099.8.patch, HIVE-22099.9.patch
>
>
> Currently dates that belong to Julian calendar (before Oct 15, 1582) are
> handled improperly by date/timestamp UDFs.
> E.g. DateFormat UDF:
> Although the dates are in Julian calendar, the formatter insists to print
> these according to Gregorian calendar causing multiple days of difference in
> some cases:
>
> {code:java}
> beeline> select date_format('1001-01-05','dd---MM--');
> ++
> | _c0 |
> ++
> | 30---12--1000 |
> ++{code}
> I've observed similar problems in the following UDFs:
> * add_months
> * date_format
> * day
> * month
> * months_between
> * weekofyear
> * year
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22099) Several date related UDFs can't handle Julian dates properly since HIVE-20007
[
https://issues.apache.org/jira/browse/HIVE-22099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Szita updated HIVE-22099:
--
Status: In Progress (was: Patch Available)
> Several date related UDFs can't handle Julian dates properly since HIVE-20007
> -
>
> Key: HIVE-22099
> URL: https://issues.apache.org/jira/browse/HIVE-22099
> Project: Hive
> Issue Type: Bug
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Attachments: HIVE-22099.0.patch, HIVE-22099.1.patch,
> HIVE-22099.2.patch, HIVE-22099.3.patch, HIVE-22099.4.patch,
> HIVE-22099.5.patch, HIVE-22099.6.patch, HIVE-22099.7.patch,
> HIVE-22099.8.patch, HIVE-22099.9.patch
>
>
> Currently dates that belong to Julian calendar (before Oct 15, 1582) are
> handled improperly by date/timestamp UDFs.
> E.g. DateFormat UDF:
> Although the dates are in Julian calendar, the formatter insists to print
> these according to Gregorian calendar causing multiple days of difference in
> some cases:
>
> {code:java}
> beeline> select date_format('1001-01-05','dd---MM--');
> ++
> | _c0 |
> ++
> | 30---12--1000 |
> ++{code}
> I've observed similar problems in the following UDFs:
> * add_months
> * date_format
> * day
> * month
> * months_between
> * weekofyear
> * year
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22174) Clean up Drop Partition
[
https://issues.apache.org/jira/browse/HIVE-22174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924488#comment-16924488
]
Hive QA commented on HIVE-22174:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
46s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} ql: The patch generated 0 new + 192 unchanged - 5
fixed = 192 total (was 197) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
53s{color} | {color:green} ql generated 0 new + 2244 unchanged - 2 fixed = 2244
total (was 2246) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 38s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18478/dev-support/hive-personality.sh
|
| git revision | master / 0213afb |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18478/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Clean up Drop Partition
> ---
>
> Key: HIVE-22174
> URL: https://issues.apache.org/jira/browse/HIVE-22174
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available, refactor-ddl
> Fix For: 4.0.0
>
> Attachments: HIVE-22174.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> There are many complex drop partition and drop partitions operations in
> Hive.java, most of them are not used by anyone inside or outside of Hive. The
> unused codes should be removed, and the remaining code should be as simple as
> possible, calling dropPartition(s) operation at the metastore client. Also
> the AlterTableDropPartitionDesc class should not be exposed here.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21510) Vectorization: add support for and/or for (constant,column) cases
[
https://issues.apache.org/jira/browse/HIVE-21510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924466#comment-16924466
]
Hive QA commented on HIVE-21510:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979648/HIVE-21510.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 16747 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_date_1]
(batchId=166)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_interval_2]
(batchId=180)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18477/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18477/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18477/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979648 - PreCommit-HIVE-Build
> Vectorization: add support for and/or for (constant,column) cases
> -
>
> Key: HIVE-21510
> URL: https://issues.apache.org/jira/browse/HIVE-21510
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-21510.01.patch, HIVE-21510.02.patch
>
>
> After HIVE-21001 some selectExpressions will start using VectorUDFAdaptor for
> "null and x" expressions. Because right now there are 2-3 places from which
> rewrite will be done to the form of "null and/or x" form; it would be better
> to support it.
> {code}
> [...]
> selectExpressions: VectorUDFAdaptor((null and dt1 is null))
> [...]
> usesVectorUDFAdaptor: true
> [...]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21510) Vectorization: add support for and/or for (constant,column) cases
[
https://issues.apache.org/jira/browse/HIVE-21510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924409#comment-16924409
]
Hive QA commented on HIVE-21510:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
36s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
26s{color} | {color:blue} storage-api in master has 48 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
32s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
39s{color} | {color:red} ql: The patch generated 6 new + 475 unchanged - 4
fixed = 481 total (was 479) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
13s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 31s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18477/dev-support/hive-personality.sh
|
| git revision | master / 0213afb |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18477/yetus/diff-checkstyle-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18477/yetus/patch-asflicense-problems.txt
|
| modules | C: storage-api ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18477/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Vectorization: add support for and/or for (constant,column) cases
> -
>
> Key: HIVE-21510
> URL: https://issues.apache.org/jira/browse/HIVE-21510
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-21510.01.patch, HIVE-21510.02.patch
>
>
> After HIVE-21001 some selectExpressions will start using VectorUDFAdaptor for
> "null and x" expressions. Because right now there are 2-3 places from which
> rewrite will be done to the form of "null and/or x" form; it would be better
> to support it.
> {code}
> [...]
> selectExpressions: VectorUDFAdaptor((null and dt1 is null))
> [...]
> usesVectorUDFAdaptor: true
> [...]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22099) Several date related UDFs can't handle Julian dates properly since HIVE-20007
[
https://issues.apache.org/jira/browse/HIVE-22099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924356#comment-16924356
]
Hive QA commented on HIVE-22099:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979623/HIVE-22099.8.patch
{color:green}SUCCESS:{color} +1 due to 7 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16747 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.llap.cache.TestBuddyAllocator.testMTT[2] (batchId=361)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18476/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18476/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18476/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979623 - PreCommit-HIVE-Build
> Several date related UDFs can't handle Julian dates properly since HIVE-20007
> -
>
> Key: HIVE-22099
> URL: https://issues.apache.org/jira/browse/HIVE-22099
> Project: Hive
> Issue Type: Bug
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Attachments: HIVE-22099.0.patch, HIVE-22099.1.patch,
> HIVE-22099.2.patch, HIVE-22099.3.patch, HIVE-22099.4.patch,
> HIVE-22099.5.patch, HIVE-22099.6.patch, HIVE-22099.7.patch, HIVE-22099.8.patch
>
>
> Currently dates that belong to Julian calendar (before Oct 15, 1582) are
> handled improperly by date/timestamp UDFs.
> E.g. DateFormat UDF:
> Although the dates are in Julian calendar, the formatter insists to print
> these according to Gregorian calendar causing multiple days of difference in
> some cases:
>
> {code:java}
> beeline> select date_format('1001-01-05','dd---MM--');
> ++
> | _c0 |
> ++
> | 30---12--1000 |
> ++{code}
> I've observed similar problems in the following UDFs:
> * add_months
> * date_format
> * day
> * month
> * months_between
> * weekofyear
> * year
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22099) Several date related UDFs can't handle Julian dates properly since HIVE-20007
[
https://issues.apache.org/jira/browse/HIVE-22099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924312#comment-16924312
]
Hive QA commented on HIVE-22099:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
39s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} ql: The patch generated 0 new + 206 unchanged - 3
fixed = 206 total (was 209) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 36s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18476/dev-support/hive-personality.sh
|
| git revision | master / 0213afb |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18476/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Several date related UDFs can't handle Julian dates properly since HIVE-20007
> -
>
> Key: HIVE-22099
> URL: https://issues.apache.org/jira/browse/HIVE-22099
> Project: Hive
> Issue Type: Bug
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Attachments: HIVE-22099.0.patch, HIVE-22099.1.patch,
> HIVE-22099.2.patch, HIVE-22099.3.patch, HIVE-22099.4.patch,
> HIVE-22099.5.patch, HIVE-22099.6.patch, HIVE-22099.7.patch, HIVE-22099.8.patch
>
>
> Currently dates that belong to Julian calendar (before Oct 15, 1582) are
> handled improperly by date/timestamp UDFs.
> E.g. DateFormat UDF:
> Although the dates are in Julian calendar, the formatter insists to print
> these according to Gregorian calendar causing multiple days of difference in
> some cases:
>
> {code:java}
> beeline> select date_format('1001-01-05','dd---MM--');
> ++
> | _c0 |
> ++
> | 30---12--1000 |
> ++{code}
> I've observed similar problems in the following UDFs:
> * add_months
> * date_format
> * day
> * month
> * months_between
> * weekofyear
> * year
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22168) remove excessive logging by llap cache.
[
https://issues.apache.org/jira/browse/HIVE-22168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924262#comment-16924262
]
Hive QA commented on HIVE-22168:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979625/HIVE-22168.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16746 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18475/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18475/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18475/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979625 - PreCommit-HIVE-Build
> remove excessive logging by llap cache.
> ---
>
> Key: HIVE-22168
> URL: https://issues.apache.org/jira/browse/HIVE-22168
> Project: Hive
> Issue Type: Improvement
> Components: llap, Logging
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-22168.patch, HIVE-22168.patch, HIVE-22168.patch
>
>
> Llap cache logging is very expensive when it comes to log every request
> buffers range.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22176) Configure Kerberos for HMS Client via HS2 configuration
[ https://issues.apache.org/jira/browse/HIVE-22176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-22176: -- Attachment: HIVE-22176.1.patch > Configure Kerberos for HMS Client via HS2 configuration > --- > > Key: HIVE-22176 > URL: https://issues.apache.org/jira/browse/HIVE-22176 > Project: Hive > Issue Type: Improvement >Reporter: Denys Kuzmenko >Priority: Major > Attachments: HIVE-22176.1.patch > > > In Hive we would like to have possibility to enable Kerberos partially (i.e > only Ranger, Atlas and HMS). > However, since hadoop security is a global flag there are many places that > need to be commented out to avoid the UGI cluster wide configuration. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22176) Configure Kerberos for HMS Client via HS2 configuration
[ https://issues.apache.org/jira/browse/HIVE-22176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-22176: -- Description: In Hive we would like to have possibility to enable Kerberos partially (i.e only Ranger, Atlas and HMS). However, since hadoop security is a global flag there are many places that need to be commented out to avoid the UGI cluster wide configuration. > Configure Kerberos for HMS Client via HS2 configuration > --- > > Key: HIVE-22176 > URL: https://issues.apache.org/jira/browse/HIVE-22176 > Project: Hive > Issue Type: Improvement >Reporter: Denys Kuzmenko >Priority: Major > > In Hive we would like to have possibility to enable Kerberos partially (i.e > only Ranger, Atlas and HMS). > However, since hadoop security is a global flag there are many places that > need to be commented out to avoid the UGI cluster wide configuration. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22168) remove excessive logging by llap cache.
[
https://issues.apache.org/jira/browse/HIVE-22168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924228#comment-16924228
]
Hive QA commented on HIVE-22168:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
45s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18475/dev-support/hive-personality.sh
|
| git revision | master / 0213afb |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18475/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> remove excessive logging by llap cache.
> ---
>
> Key: HIVE-22168
> URL: https://issues.apache.org/jira/browse/HIVE-22168
> Project: Hive
> Issue Type: Improvement
> Components: llap, Logging
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-22168.patch, HIVE-22168.patch, HIVE-22168.patch
>
>
> Llap cache logging is very expensive when it comes to log every request
> buffers range.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22055) select count gives incorrect result after loading data from text file
[
https://issues.apache.org/jira/browse/HIVE-22055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Magyar updated HIVE-22055:
-
Attachment: (was: HIVE-22055.2.patch)
> select count gives incorrect result after loading data from text file
> -
>
> Key: HIVE-22055
> URL: https://issues.apache.org/jira/browse/HIVE-22055
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Attila Magyar
>Assignee: Attila Magyar
>Priority: Major
> Attachments: HIVE-22055.1.patch
>
>
> Add one more load to mm_loaddata.q:
> Load data 3 times (both kv1.txt and kv2.txt contains 500 records)
> {code:java}
> create table load0_mm (key string, value string) stored as textfile
> tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> load data local inpath '../../data/files/kv1.txt' into table load0_mm;
> select count(1) from load0_mm;
> load data local inpath '../../data/files/kv2.txt' into table load0_mm;
> select count(1) from load0_mm;
> load data local inpath '../../data/files/kv2.txt' into table load0_mm;
> select count(1) from load0_mm;{code}
> Expected output
> {code:java}
> PREHOOK: query: load data local inpath '../../data/files/kv2.txt' into table
> load0_mm
> PREHOOK: type: LOAD
> A masked pattern was here
> PREHOOK: Output: default@load0_mm
> POSTHOOK: query: load data local inpath '../../data/files/kv2.txt' into table
> load0_mm
> POSTHOOK: type: LOAD
> A masked pattern was here
> POSTHOOK: Output: default@load0_mm
> PREHOOK: query: select count(1) from load0_mm
> PREHOOK: type: QUERY
> PREHOOK: Input: default@load0_mm
> A masked pattern was here
> POSTHOOK: query: select count(1) from load0_mm
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@load0_mm
> A masked pattern was here
> 1500{code}
> Got:
> [ERROR] TestMiniLlapLocalCliDriver.testCliDriver:59 Client Execution
> succeeded but contained differences (error code = 1) after executing
> mm_loaddata.q
> 63c63
> < 1480
> —
> > 1500
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Assigned] (HIVE-19456) Beeline dfs command Error: Error while processing statement: (state=,code=1)
[ https://issues.apache.org/jira/browse/HIVE-19456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chuanjie.duan reassigned HIVE-19456: Assignee: chuanjie.duan (was: Bijoy Kumar Khandelwal) > Beeline dfs command Error: Error while processing statement: (state=,code=1) > > > Key: HIVE-19456 > URL: https://issues.apache.org/jira/browse/HIVE-19456 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 1.2.1 >Reporter: chuanjie.duan >Assignee: chuanjie.duan >Priority: Major > Attachments: HIVE-19456.0.patch > > > Reproduce steps: > # hdfs dfs -chmod 750 /tmp/aaa (user:test) > # beeline login with user test1 > # beeline "dfs -ls /tmp/aaa" > # Error: Error while processing statement: (state=,code=1) > Cause: > beeline "dfs" command would call fsshell, fsshell just err.print(errmsg) and > return "null" value when catching exception -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22055) select count gives incorrect result after loading data from text file
[
https://issues.apache.org/jira/browse/HIVE-22055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924209#comment-16924209
]
Hive QA commented on HIVE-22055:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979617/HIVE-22055.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18474/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18474/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18474/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2019-09-06 12:26:53.670
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-18474/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2019-09-06 12:26:53.673
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 0213afb HIVE-22028 Clean up Add Partition (Miklos Gergely
reviewd by Jesus Camacho Rodriguez)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 0213afb HIVE-22028 Clean up Add Partition (Miklos Gergely
reviewd by Jesus Camacho Rodriguez)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2019-09-06 12:26:54.754
+ rm -rf ../yetus_PreCommit-HIVE-Build-18474
+ mkdir ../yetus_PreCommit-HIVE-Build-18474
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-18474
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-18474/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
fatal: corrupt patch at line 26
fatal: corrupt patch at line 26
fatal: corrupt patch at line 26
The patch does not appear to apply with p0, p1, or p2
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-18474
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979617 - PreCommit-HIVE-Build
> select count gives incorrect result after loading data from text file
> -
>
> Key: HIVE-22055
> URL: https://issues.apache.org/jira/browse/HIVE-22055
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Attila Magyar
>Assignee: Attila Magyar
>Priority: Major
> Attachments: HIVE-22055.1.patch, HIVE-22055.2.patch
>
>
> Add one more load to mm_loaddata.q:
> Load data 3 times (both kv1.txt and kv2.txt contains 500 records)
> {code:java}
> create table load0_mm (key string, value string) stored as textfile
> tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> load data local inpath '../../data/files/kv1.txt' into table load0_mm;
> select count(1) from load0_mm;
> load data local inpath '../../data/files/kv2.txt' into table load0_mm;
> select count(1) from load0_mm;
> load data local inpath '../../data/files/kv2.txt' into table load0_mm;
> select count(1) from load0_mm;{code}
> Expected output
> {code:java}
> PREHOOK: query: load data local inpath '../../data/files/kv2.txt' into table
> load0_mm
> PREHOOK: type: LOAD
> A masked pattern was here
> PREHOOK: Output: default@load0_mm
> POSTHOOK: query: load data local inpath '../../data/files/kv2.txt' into table
> load0_mm
> POSTHOOK: type: LOAD
> A masked pattern was here
> POSTHOOK: Output: default@load0_mm
> PREHOOK: query: select count(1) from load0_mm
> PREHOOK: type: QUERY
> PREHOOK: Input: default@load0_mm
> #
[jira] [Commented] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924207#comment-16924207
]
Hive QA commented on HIVE-22163:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979610/HIVE-22163.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16747 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join_reordering_no_stats]
(batchId=179)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18473/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18473/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18473/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979610 - PreCommit-HIVE-Build
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567
> Basic stats: COMPLETE Column stats: NONE'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22175) TestBudyAllocator#testMTT test is flaky
[ https://issues.apache.org/jira/browse/HIVE-22175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924194#comment-16924194 ] Adam Szita commented on HIVE-22175: --- [~bslim] any ideas here? > TestBudyAllocator#testMTT test is flaky > --- > > Key: HIVE-22175 > URL: https://issues.apache.org/jira/browse/HIVE-22175 > Project: Hive > Issue Type: Bug >Reporter: Adam Szita >Priority: Major > > This test has a fail rate of about 20%-25% -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (HIVE-22098) Data loss occurs when multiple tables are join with different bucket_version
[
https://issues.apache.org/jira/browse/HIVE-22098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
LuGuangMing resolved HIVE-22098.
Target Version/s: 3.1.0 (was: 3.1.0, 3.2.0)
Resolution: Fixed
> Data loss occurs when multiple tables are join with different bucket_version
>
>
> Key: HIVE-22098
> URL: https://issues.apache.org/jira/browse/HIVE-22098
> Project: Hive
> Issue Type: Bug
> Components: Operators
>Affects Versions: 3.1.0
>Reporter: LuGuangMing
>Assignee: LuGuangMing
>Priority: Major
> Attachments: HIVE-22098.1.patch, image-2019-08-12-18-45-15-771.png,
> join_test.sql, table_a_data.orc, table_b_data.orc, table_c_data.orc
>
>
> When different bucketVersion of tables do join and reducers number greater
> than 2, result is easy to lose data.
> *Scenario 1*: Three tables join. The temporary result data of table_a in the
> first table and table_b in the second table joins result is recorded as
> tmp_a_b, When it joins with the third table, the bucket_version=2 of the
> table created by default after hive-3.0.0, temporary data tmp_a_b initialized
> the bucketVerison=-1, and then ReduceSinkOperator Verketison=-1 is joined. In
> the init method, the hash algorithm of selecting join column is selected
> according to bucketVersion. If bucketVersion = 2 and is not an acid
> operation, it will acquired the new algorithm of hash. Otherwise, the old
> algorithm of hash is acquired. Because of the inconsistency of the algorithm
> of hash, the partition of data allocation caused are different. At stage of
> Reducer, Data with the same key can not be paired resulting in data loss.
> *Scenario 2*: create two test tables, create table
> table_bucketversion_1(col_1 string, col_2 string) TBLPROPERTIES
> ('bucketing_version'='1'); table_bucketversion_2(col_1 string, col_2 string)
> TBLPROPERTIES ('bucketing_version'='2');
> when use table_bucketversion_1 to join table_bucketversion_2, partial result
> data will be loss due to bucketVerison is different.
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924164#comment-16924164
]
Hive QA commented on HIVE-22163:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
42s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 4s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18473/dev-support/hive-personality.sh
|
| git revision | master / 0213afb |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18473/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {cod
[jira] [Updated] (HIVE-22174) Clean up Drop Partition
[ https://issues.apache.org/jira/browse/HIVE-22174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-22174: -- Labels: pull-request-available refactor-ddl (was: refactor-ddl) > Clean up Drop Partition > --- > > Key: HIVE-22174 > URL: https://issues.apache.org/jira/browse/HIVE-22174 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Fix For: 4.0.0 > > Attachments: HIVE-22174.01.patch > > > There are many complex drop partition and drop partitions operations in > Hive.java, most of them are not used by anyone inside or outside of Hive. The > unused codes should be removed, and the remaining code should be as simple as > possible, calling dropPartition(s) operation at the metastore client. Also > the AlterTableDropPartitionDesc class should not be exposed here. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work logged] (HIVE-22174) Clean up Drop Partition
[ https://issues.apache.org/jira/browse/HIVE-22174?focusedWorklogId=307764&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-307764 ] ASF GitHub Bot logged work on HIVE-22174: - Author: ASF GitHub Bot Created on: 06/Sep/19 11:19 Start Date: 06/Sep/19 11:19 Worklog Time Spent: 10m Work Description: miklosgergely commented on pull request #761: HIVE-22174 Clean up Drop Partition (Miklos Gergely reviewd by Jesus Camacho Rodriguez) URL: https://github.com/apache/hive/pull/761 There are many complex drop partition and drop partitions operations in Hive.java, most of them are not used by anyone inside or outside of Hive. The unused codes should be removed, and the remaining code should be as simple as possible, calling dropPartition(s) operation at the metastore client. Also the AlterTableDropPartitionDesc class should not be exposed here. Also removed some other unused private methods from Hive.java, and an import with '*'. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 307764) Remaining Estimate: 0h Time Spent: 10m > Clean up Drop Partition > --- > > Key: HIVE-22174 > URL: https://issues.apache.org/jira/browse/HIVE-22174 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Fix For: 4.0.0 > > Attachments: HIVE-22174.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > > There are many complex drop partition and drop partitions operations in > Hive.java, most of them are not used by anyone inside or outside of Hive. The > unused codes should be removed, and the remaining code should be as simple as > possible, calling dropPartition(s) operation at the metastore client. Also > the AlterTableDropPartitionDesc class should not be exposed here. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22174) Clean up Drop Partition
[ https://issues.apache.org/jira/browse/HIVE-22174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22174: -- Status: Patch Available (was: Open) > Clean up Drop Partition > --- > > Key: HIVE-22174 > URL: https://issues.apache.org/jira/browse/HIVE-22174 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: refactor-ddl > Fix For: 4.0.0 > > Attachments: HIVE-22174.01.patch > > > There are many complex drop partition and drop partitions operations in > Hive.java, most of them are not used by anyone inside or outside of Hive. The > unused codes should be removed, and the remaining code should be as simple as > possible, calling dropPartition(s) operation at the metastore client. Also > the AlterTableDropPartitionDesc class should not be exposed here. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22174) Clean up Drop Partition
[ https://issues.apache.org/jira/browse/HIVE-22174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22174: -- Attachment: HIVE-22174.01.patch > Clean up Drop Partition > --- > > Key: HIVE-22174 > URL: https://issues.apache.org/jira/browse/HIVE-22174 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: refactor-ddl > Fix For: 4.0.0 > > Attachments: HIVE-22174.01.patch > > > There are many complex drop partition and drop partitions operations in > Hive.java, most of them are not used by anyone inside or outside of Hive. The > unused codes should be removed, and the remaining code should be as simple as > possible, calling dropPartition(s) operation at the metastore client. Also > the AlterTableDropPartitionDesc class should not be exposed here. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22174) Clean up Drop Partition
[ https://issues.apache.org/jira/browse/HIVE-22174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22174: -- Attachment: (was: HIVE-22174.01.patch) > Clean up Drop Partition > --- > > Key: HIVE-22174 > URL: https://issues.apache.org/jira/browse/HIVE-22174 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: refactor-ddl > Fix For: 4.0.0 > > Attachments: HIVE-22174.01.patch > > > There are many complex drop partition and drop partitions operations in > Hive.java, most of them are not used by anyone inside or outside of Hive. The > unused codes should be removed, and the remaining code should be as simple as > possible, calling dropPartition(s) operation at the metastore client. Also > the AlterTableDropPartitionDesc class should not be exposed here. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22174) Clean up Drop Partition
[ https://issues.apache.org/jira/browse/HIVE-22174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22174: -- Attachment: HIVE-22174.01.patch > Clean up Drop Partition > --- > > Key: HIVE-22174 > URL: https://issues.apache.org/jira/browse/HIVE-22174 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: refactor-ddl > Fix For: 4.0.0 > > Attachments: HIVE-22174.01.patch > > > There are many complex drop partition and drop partitions operations in > Hive.java, most of them are not used by anyone inside or outside of Hive. The > unused codes should be removed, and the remaining code should be as simple as > possible, calling dropPartition(s) operation at the metastore client. Also > the AlterTableDropPartitionDesc class should not be exposed here. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22174) Clean up Drop Partition
[ https://issues.apache.org/jira/browse/HIVE-22174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22174: -- Description: There are many complex drop partition and drop partitions operations in Hive.java, most of them are not used by anyone inside or outside of Hive. The unused codes should be removed, and the remaining code should be as simple as possible, calling dropPartition(s) operation at the metastore client. Also the AlterTableDropPartitionDesc class should not be exposed here. (was: AlterTableAddPartitionDesc should be immutable, like the rest of the desc classes. This can not be done 100% right now, as it requires the refactoring of the ImportSemanticAnalyzer, so the task will be finished only then. Add Partition logic should be moved from Hive.java to AlterTableAddPartitionOperation.java, only the metastore calls should remain in Hive.java.) > Clean up Drop Partition > --- > > Key: HIVE-22174 > URL: https://issues.apache.org/jira/browse/HIVE-22174 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Fix For: 4.0.0 > > Attachments: HIVE-22174.01.patch > > > There are many complex drop partition and drop partitions operations in > Hive.java, most of them are not used by anyone inside or outside of Hive. The > unused codes should be removed, and the remaining code should be as simple as > possible, calling dropPartition(s) operation at the metastore client. Also > the AlterTableDropPartitionDesc class should not be exposed here. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22174) Clean up Drop Partition
[ https://issues.apache.org/jira/browse/HIVE-22174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22174: -- Labels: refactor-ddl (was: pull-request-available refactor-ddl) > Clean up Drop Partition > --- > > Key: HIVE-22174 > URL: https://issues.apache.org/jira/browse/HIVE-22174 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: refactor-ddl > Fix For: 4.0.0 > > Attachments: HIVE-22174.01.patch > > > There are many complex drop partition and drop partitions operations in > Hive.java, most of them are not used by anyone inside or outside of Hive. The > unused codes should be removed, and the remaining code should be as simple as > possible, calling dropPartition(s) operation at the metastore client. Also > the AlterTableDropPartitionDesc class should not be exposed here. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (HIVE-22174) Clean up Drop Partition
[ https://issues.apache.org/jira/browse/HIVE-22174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely reassigned HIVE-22174: - > Clean up Drop Partition > --- > > Key: HIVE-22174 > URL: https://issues.apache.org/jira/browse/HIVE-22174 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Fix For: 4.0.0 > > > AlterTableAddPartitionDesc should be immutable, like the rest of the desc > classes. This can not be done 100% right now, as it requires the refactoring > of the ImportSemanticAnalyzer, so the task will be finished only then. > Add Partition logic should be moved from Hive.java to > AlterTableAddPartitionOperation.java, only the metastore calls should remain > in Hive.java. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-20113) Shuffle avoidance: Disable 1-1 edges for sorted shuffle
[
https://issues.apache.org/jira/browse/HIVE-20113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924143#comment-16924143
]
Hive QA commented on HIVE-20113:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979593/HIVE-20113.5.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 100 failed/errored test(s), 16746 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[dynamic_semijoin_user_level]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_smb_mapjoin_14]
(batchId=178)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_9]
(batchId=180)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez2]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_no_stats]
(batchId=182)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cluster]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[column_access_stats]
(batchId=177)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[correlationoptimizer2]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[correlationoptimizer3]
(batchId=185)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[correlationoptimizer4]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[correlationoptimizer6]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynamic_semijoin_reduction]
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynamic_semijoin_reduction_3]
(batchId=182)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynamic_semijoin_reduction_4]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynamic_semijoin_reduction_sw2]
(batchId=183)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynamic_semijoin_reduction_sw]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[explainuser_1]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[external_jdbc_table_perf]
(batchId=185)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join_is_not_distinct_from]
(batchId=186)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join_nullsafe]
(batchId=181)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[keep_uniform]
(batchId=181)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_smb_ptf]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_cluster]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_partition_cluster]
(batchId=183)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_rewrite_6]
(batchId=180)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_rewrite_7]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mrr]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[parallel]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[partialdhj]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[partition_shared_scan]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[semijoin]
(batchId=166)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[semijoin_hint]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[semijoin_reddedup]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sharedwork]
(batchId=181)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[skewjoin]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[smb_mapjoin_17]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[smb_mapjoin_4]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[smb_mapjoin_5]
(batchId=187)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sqlmerge]
(batchId=184)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sqlmerge_stats]
(batchId=183)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_ALL]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_ANY]
(batchId=177)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_in]
(batchId=179)
org.apache.hadoop.hive.cli.Te
[jira] [Updated] (HIVE-21510) Vectorization: add support for and/or for (constant,column) cases
[
https://issues.apache.org/jira/browse/HIVE-21510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Bodor updated HIVE-21510:
Attachment: HIVE-21510.02.patch
> Vectorization: add support for and/or for (constant,column) cases
> -
>
> Key: HIVE-21510
> URL: https://issues.apache.org/jira/browse/HIVE-21510
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-21510.01.patch, HIVE-21510.02.patch
>
>
> After HIVE-21001 some selectExpressions will start using VectorUDFAdaptor for
> "null and x" expressions. Because right now there are 2-3 places from which
> rewrite will be done to the form of "null and/or x" form; it would be better
> to support it.
> {code}
> [...]
> selectExpressions: VectorUDFAdaptor((null and dt1 is null))
> [...]
> usesVectorUDFAdaptor: true
> [...]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21510) Vectorization: add support for and/or for (constant,column) cases
[
https://issues.apache.org/jira/browse/HIVE-21510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Bodor updated HIVE-21510:
Attachment: (was: HIVE-21510.02.patch)
> Vectorization: add support for and/or for (constant,column) cases
> -
>
> Key: HIVE-21510
> URL: https://issues.apache.org/jira/browse/HIVE-21510
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-21510.01.patch
>
>
> After HIVE-21001 some selectExpressions will start using VectorUDFAdaptor for
> "null and x" expressions. Because right now there are 2-3 places from which
> rewrite will be done to the form of "null and/or x" form; it would be better
> to support it.
> {code}
> [...]
> selectExpressions: VectorUDFAdaptor((null and dt1 is null))
> [...]
> usesVectorUDFAdaptor: true
> [...]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21510) Vectorization: add support for and/or for (constant,column) cases
[
https://issues.apache.org/jira/browse/HIVE-21510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Bodor updated HIVE-21510:
Attachment: HIVE-21510.02.patch
> Vectorization: add support for and/or for (constant,column) cases
> -
>
> Key: HIVE-21510
> URL: https://issues.apache.org/jira/browse/HIVE-21510
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-21510.01.patch, HIVE-21510.02.patch
>
>
> After HIVE-21001 some selectExpressions will start using VectorUDFAdaptor for
> "null and x" expressions. Because right now there are 2-3 places from which
> rewrite will be done to the form of "null and/or x" form; it would be better
> to support it.
> {code}
> [...]
> selectExpressions: VectorUDFAdaptor((null and dt1 is null))
> [...]
> usesVectorUDFAdaptor: true
> [...]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-20113) Shuffle avoidance: Disable 1-1 edges for sorted shuffle
[
https://issues.apache.org/jira/browse/HIVE-20113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924106#comment-16924106
]
Hive QA commented on HIVE-20113:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
32s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
34s{color} | {color:red} ql: The patch generated 1 new + 20 unchanged - 1 fixed
= 21 total (was 21) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 16s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18472/dev-support/hive-personality.sh
|
| git revision | master / 0213afb |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18472/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18472/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Shuffle avoidance: Disable 1-1 edges for sorted shuffle
>
>
> Key: HIVE-20113
> URL: https://issues.apache.org/jira/browse/HIVE-20113
> Project: Hive
> Issue Type: Bug
> Components: Tez
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Labels: Branch3Candidate
> Attachments: HIVE-20113.1.patch, HIVE-20113.2.patch,
> HIVE-20113.3.patch, HIVE-20113.4.patch, HIVE-20113.4.patch, HIVE-20113.5.patch
>
>
> The sorted shuffle avoidance can have some issues when the shuffle data gets
> broken up into multiple chunks on disk.
> The 1-1 edge cannot skip the tez final merge - there's no reason for 1-1 to
> have a final merge at all, it should open a single compressed file and write
> a single index entry.
> Until the shuffle issue is resolved & a lot more testing, it is prudent to
> disable the optimization for sorted shuffle edges and stop rewriting the
> RS(sorted) = = = RS(sorted) into RS(sorted) = = = RS(FORWARD).
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22149) Metastore: Unify codahale metrics.log json structure between hiveserver2 and metastore services
[
https://issues.apache.org/jira/browse/HIVE-22149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924091#comment-16924091
]
Hive QA commented on HIVE-22149:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979588/HIVE-22149.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 192 failed/errored test(s), 15052 tests
executed
*Failed tests:*
{noformat}
TestAddPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=223)
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=274)
TestAuthorizationPreEventListener - did not produce a TEST-*.xml file (likely
timed out) (batchId=276)
TestAuthzApiEmbedAuthorizerInRemote - did not produce a TEST-*.xml file (likely
timed out) (batchId=246)
TestBeelineConnectionUsingHiveSite - did not produce a TEST-*.xml file (likely
timed out) (batchId=277)
TestBeelineWithUserHs2ConnectionFile - did not produce a TEST-*.xml file
(likely timed out) (batchId=277)
TestCachedStore - did not produce a TEST-*.xml file (likely timed out)
(batchId=233)
TestCatalogCaching - did not produce a TEST-*.xml file (likely timed out)
(batchId=233)
TestCatalogNonDefaultClient - did not produce a TEST-*.xml file (likely timed
out) (batchId=223)
TestCatalogOldClient - did not produce a TEST-*.xml file (likely timed out)
(batchId=223)
TestCheckConstraint - did not produce a TEST-*.xml file (likely timed out)
(batchId=223)
TestClientSideAuthorizationProvider - did not produce a TEST-*.xml file (likely
timed out) (batchId=277)
TestCommands - did not produce a TEST-*.xml file (likely timed out)
(batchId=207)
TestCopyUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=274)
TestCreateUdfEntities - did not produce a TEST-*.xml file (likely timed out)
(batchId=246)
TestDDLWithRemoteMetastoreSecondNamenode - did not produce a TEST-*.xml file
(likely timed out) (batchId=246)
TestDFSErrorHandling - did not produce a TEST-*.xml file (likely timed out)
(batchId=281)
TestDataSourceProviderFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=233)
TestDeadline - did not produce a TEST-*.xml file (likely timed out)
(batchId=233)
TestDropPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=223)
TestEximReplicationTasks - did not produce a TEST-*.xml file (likely timed out)
(batchId=207)
TestExportImport - did not produce a TEST-*.xml file (likely timed out)
(batchId=274)
TestFilterHooks - did not produce a TEST-*.xml file (likely timed out)
(batchId=223)
TestFunctions - did not produce a TEST-*.xml file (likely timed out)
(batchId=223)
TestGetPartitionsUsingProjectionAndFilterSpecs - did not produce a TEST-*.xml
file (likely timed out) (batchId=225)
TestGetTableMeta - did not produce a TEST-*.xml file (likely timed out)
(batchId=223)
TestHCatClient - did not produce a TEST-*.xml file (likely timed out)
(batchId=207)
TestHDFSPermissionPolicyProvider - did not produce a TEST-*.xml file (likely
timed out) (batchId=277)
TestHS2ImpersonationWithRemoteMS - did not produce a TEST-*.xml file (likely
timed out) (batchId=281)
TestHiveAlterHandler - did not produce a TEST-*.xml file (likely timed out)
(batchId=223)
TestHiveAuthFactory - did not produce a TEST-*.xml file (likely timed out)
(batchId=298)
TestHiveAuthorizerCheckInvocation - did not produce a TEST-*.xml file (likely
timed out) (batchId=277)
TestHiveAuthorizerShowFilters - did not produce a TEST-*.xml file (likely timed
out) (batchId=277)
TestHiveHistory - did not produce a TEST-*.xml file (likely timed out)
(batchId=277)
TestHiveMetaStoreAlterColumnPar - did not produce a TEST-*.xml file (likely
timed out) (batchId=246)
TestHiveMetaStoreGetMetaConf - did not produce a TEST-*.xml file (likely timed
out) (batchId=233)
TestHiveMetaStorePartitionSpecs - did not produce a TEST-*.xml file (likely
timed out) (batchId=225)
TestHiveMetaStoreSchemaMethods - did not produce a TEST-*.xml file (likely
timed out) (batchId=233)
TestHiveMetaStoreTimeout - did not produce a TEST-*.xml file (likely timed out)
(batchId=233)
TestHiveMetaStoreTxns - did not produce a TEST-*.xml file (likely timed out)
(batchId=233)
TestHiveMetastoreCli - did not produce a TEST-*.xml file (likely timed out)
(batchId=223)
TestHiveProtoEventsCleanerTask - did not produce a TEST-*.xml file (likely
timed out) (batchId=246)
TestHiveRemote - did not produce a TEST-*.xml file (likely timed out)
(batchId=331)
TestHs2Hooks - did not produce a TEST-*.xml file (likely timed out)
(batchId=246)
TestHs2HooksWithMiniKdc - did not produce a TEST-*.xml file (likely timed out)
(batchId=298)
TestJdbcGenericUDTFGetSplits - did not produce a TEST-*.xml file (likely timed
out) (batchId=281)
TestJdbcNonKrbSASLWithMiniKdc - did not produce a TEST-*.xml file (likely t
[jira] [Commented] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924082#comment-16924082
]
Ana Jalba commented on HIVE-21508:
--
Is there anything I should do about the builds? I don't really understand
what's happening with these failing builds.
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 3.2.0, 2.3.7
>
> Attachments: HIVE-21508.branch-2.3.patch, HIVE-21508.branch-2.patch,
> HIVE-21508.branch-3.1.patch, HIVE-21508.branch-3.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22168) remove excessive logging by llap cache.
[ https://issues.apache.org/jira/browse/HIVE-22168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-22168: -- Attachment: HIVE-22168.patch > remove excessive logging by llap cache. > --- > > Key: HIVE-22168 > URL: https://issues.apache.org/jira/browse/HIVE-22168 > Project: Hive > Issue Type: Improvement > Components: llap, Logging >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Attachments: HIVE-22168.patch, HIVE-22168.patch, HIVE-22168.patch > > > Llap cache logging is very expensive when it comes to log every request > buffers range. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22099) Several date related UDFs can't handle Julian dates properly since HIVE-20007
[
https://issues.apache.org/jira/browse/HIVE-22099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Szita updated HIVE-22099:
--
Attachment: HIVE-22099.8.patch
> Several date related UDFs can't handle Julian dates properly since HIVE-20007
> -
>
> Key: HIVE-22099
> URL: https://issues.apache.org/jira/browse/HIVE-22099
> Project: Hive
> Issue Type: Bug
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Attachments: HIVE-22099.0.patch, HIVE-22099.1.patch,
> HIVE-22099.2.patch, HIVE-22099.3.patch, HIVE-22099.4.patch,
> HIVE-22099.5.patch, HIVE-22099.6.patch, HIVE-22099.7.patch, HIVE-22099.8.patch
>
>
> Currently dates that belong to Julian calendar (before Oct 15, 1582) are
> handled improperly by date/timestamp UDFs.
> E.g. DateFormat UDF:
> Although the dates are in Julian calendar, the formatter insists to print
> these according to Gregorian calendar causing multiple days of difference in
> some cases:
>
> {code:java}
> beeline> select date_format('1001-01-05','dd---MM--');
> ++
> | _c0 |
> ++
> | 30---12--1000 |
> ++{code}
> I've observed similar problems in the following UDFs:
> * add_months
> * date_format
> * day
> * month
> * months_between
> * weekofyear
> * year
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22099) Several date related UDFs can't handle Julian dates properly since HIVE-20007
[
https://issues.apache.org/jira/browse/HIVE-22099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Szita updated HIVE-22099:
--
Status: Patch Available (was: In Progress)
> Several date related UDFs can't handle Julian dates properly since HIVE-20007
> -
>
> Key: HIVE-22099
> URL: https://issues.apache.org/jira/browse/HIVE-22099
> Project: Hive
> Issue Type: Bug
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Attachments: HIVE-22099.0.patch, HIVE-22099.1.patch,
> HIVE-22099.2.patch, HIVE-22099.3.patch, HIVE-22099.4.patch,
> HIVE-22099.5.patch, HIVE-22099.6.patch, HIVE-22099.7.patch, HIVE-22099.8.patch
>
>
> Currently dates that belong to Julian calendar (before Oct 15, 1582) are
> handled improperly by date/timestamp UDFs.
> E.g. DateFormat UDF:
> Although the dates are in Julian calendar, the formatter insists to print
> these according to Gregorian calendar causing multiple days of difference in
> some cases:
>
> {code:java}
> beeline> select date_format('1001-01-05','dd---MM--');
> ++
> | _c0 |
> ++
> | 30---12--1000 |
> ++{code}
> I've observed similar problems in the following UDFs:
> * add_months
> * date_format
> * day
> * month
> * months_between
> * weekofyear
> * year
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22099) Several date related UDFs can't handle Julian dates properly since HIVE-20007
[
https://issues.apache.org/jira/browse/HIVE-22099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Szita updated HIVE-22099:
--
Status: In Progress (was: Patch Available)
> Several date related UDFs can't handle Julian dates properly since HIVE-20007
> -
>
> Key: HIVE-22099
> URL: https://issues.apache.org/jira/browse/HIVE-22099
> Project: Hive
> Issue Type: Bug
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Attachments: HIVE-22099.0.patch, HIVE-22099.1.patch,
> HIVE-22099.2.patch, HIVE-22099.3.patch, HIVE-22099.4.patch,
> HIVE-22099.5.patch, HIVE-22099.6.patch, HIVE-22099.7.patch
>
>
> Currently dates that belong to Julian calendar (before Oct 15, 1582) are
> handled improperly by date/timestamp UDFs.
> E.g. DateFormat UDF:
> Although the dates are in Julian calendar, the formatter insists to print
> these according to Gregorian calendar causing multiple days of difference in
> some cases:
>
> {code:java}
> beeline> select date_format('1001-01-05','dd---MM--');
> ++
> | _c0 |
> ++
> | 30---12--1000 |
> ++{code}
> I've observed similar problems in the following UDFs:
> * add_months
> * date_format
> * day
> * month
> * months_between
> * weekofyear
> * year
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21942) Remove useless MetastoreType enum from AbstractCliConfig
[ https://issues.apache.org/jira/browse/HIVE-21942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924008#comment-16924008 ] Laszlo Bodor commented on HIVE-21942: - +1 > Remove useless MetastoreType enum from AbstractCliConfig > > > Key: HIVE-21942 > URL: https://issues.apache.org/jira/browse/HIVE-21942 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Laszlo Bodor >Assignee: Imre Molnar >Priority: Major > Labels: newbie > Fix For: 4.0.0 > > Attachments: HIVE-21942.1.patch > > > I'm not 100% aware of historical reasons, but this single-value enum seems > totally useless at the moment. > https://github.com/apache/hive/blob/86a15600ae610e1c6017019883ef9344585572ef/itests/util/src/main/java/org/apache/hadoop/hive/cli/control/AbstractCliConfig.java#L50-L52 > According to HIVE-14854, it may have made sense when there was an initiative > of hbase metastore, but it's not in the scope anymore AFAIK. > https://github.com/apache/hive/commit/474425aa62e3f25b119419439373aa684c6c2121 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22149) Metastore: Unify codahale metrics.log json structure between hiveserver2 and metastore services
[
https://issues.apache.org/jira/browse/HIVE-22149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16923999#comment-16923999
]
Hive QA commented on HIVE-22149:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
59s{color} | {color:blue} standalone-metastore/metastore-server in master has
181 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 9s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18471/dev-support/hive-personality.sh
|
| git revision | master / 0213afb |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18471/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Metastore: Unify codahale metrics.log json structure between hiveserver2 and
> metastore services
> ---
>
> Key: HIVE-22149
> URL: https://issues.apache.org/jira/browse/HIVE-22149
> Project: Hive
> Issue Type: Bug
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-22149.01.patch, HIVE-22149.01.patch,
> HIVE-22149.01.patch, metrics_hiveserver2.log, metrics_metastore.log
>
>
> While fixing HIVE-22140 I found some really annoying differences between the
> codahale metric file structures between hiveserver2 and metastore, e.g.
> open_connections: can be found in "counters" for hs2, but in "gauges" for ms
> threads count: it's a proper "threads.count" for hs2, but a really ambiguous
> "count" for ms
> so I realized that "memory." and "threads." prefix is completely absent in ms
> metrics file, which is misleading
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22169) Tez: SplitGenerator tries to look for plan files which won't exist for Tez
[
https://issues.apache.org/jira/browse/HIVE-22169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16923981#comment-16923981
]
Hive QA commented on HIVE-22169:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979587/HIVE-22169.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 16746 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestObjectStore.catalogs (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDatabaseOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDeprecatedConfigIsOverwritten
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropParitionsCleanup
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropPartitionsCacheCrossSession
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSqlErrorMetrics
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testEmptyTrustStoreProps
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testMasterKeyOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testMaxEventResponse
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testPartitionOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testQueryCloseOnError
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testRoleOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testTableOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testUseSSLProperty
(batchId=233)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18470/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18470/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18470/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979587 - PreCommit-HIVE-Build
> Tez: SplitGenerator tries to look for plan files which won't exist for Tez
> --
>
> Key: HIVE-22169
> URL: https://issues.apache.org/jira/browse/HIVE-22169
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-22169.1.patch, HIVE-22169.1.patch
>
>
> {code}
> at
> org.apache.hadoop.hive.ql.exec.Utilities.clearWork(Utilities.java:310)
> at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:318)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269)
> at java.security.AccessController.doPrivileged(Native Method
> {code}
> The split generator tries to clear out the work items from HDFS, which will
> never exist for Tez plans.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22055) select count gives incorrect result after loading data from text file
[
https://issues.apache.org/jira/browse/HIVE-22055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Magyar updated HIVE-22055:
-
Attachment: HIVE-22055.2.patch
> select count gives incorrect result after loading data from text file
> -
>
> Key: HIVE-22055
> URL: https://issues.apache.org/jira/browse/HIVE-22055
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Attila Magyar
>Assignee: Attila Magyar
>Priority: Major
> Attachments: HIVE-22055.1.patch, HIVE-22055.2.patch
>
>
> Add one more load to mm_loaddata.q:
> Load data 3 times (both kv1.txt and kv2.txt contains 500 records)
> {code:java}
> create table load0_mm (key string, value string) stored as textfile
> tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> load data local inpath '../../data/files/kv1.txt' into table load0_mm;
> select count(1) from load0_mm;
> load data local inpath '../../data/files/kv2.txt' into table load0_mm;
> select count(1) from load0_mm;
> load data local inpath '../../data/files/kv2.txt' into table load0_mm;
> select count(1) from load0_mm;{code}
> Expected output
> {code:java}
> PREHOOK: query: load data local inpath '../../data/files/kv2.txt' into table
> load0_mm
> PREHOOK: type: LOAD
> A masked pattern was here
> PREHOOK: Output: default@load0_mm
> POSTHOOK: query: load data local inpath '../../data/files/kv2.txt' into table
> load0_mm
> POSTHOOK: type: LOAD
> A masked pattern was here
> POSTHOOK: Output: default@load0_mm
> PREHOOK: query: select count(1) from load0_mm
> PREHOOK: type: QUERY
> PREHOOK: Input: default@load0_mm
> A masked pattern was here
> POSTHOOK: query: select count(1) from load0_mm
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@load0_mm
> A masked pattern was here
> 1500{code}
> Got:
> [ERROR] TestMiniLlapLocalCliDriver.testCliDriver:59 Client Execution
> succeeded but contained differences (error code = 1) after executing
> mm_loaddata.q
> 63c63
> < 1480
> —
> > 1500
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22055) select count gives incorrect result after loading data from text file
[
https://issues.apache.org/jira/browse/HIVE-22055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Magyar updated HIVE-22055:
-
Status: Open (was: Patch Available)
> select count gives incorrect result after loading data from text file
> -
>
> Key: HIVE-22055
> URL: https://issues.apache.org/jira/browse/HIVE-22055
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Attila Magyar
>Assignee: Attila Magyar
>Priority: Major
> Attachments: HIVE-22055.1.patch, HIVE-22055.2.patch
>
>
> Add one more load to mm_loaddata.q:
> Load data 3 times (both kv1.txt and kv2.txt contains 500 records)
> {code:java}
> create table load0_mm (key string, value string) stored as textfile
> tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> load data local inpath '../../data/files/kv1.txt' into table load0_mm;
> select count(1) from load0_mm;
> load data local inpath '../../data/files/kv2.txt' into table load0_mm;
> select count(1) from load0_mm;
> load data local inpath '../../data/files/kv2.txt' into table load0_mm;
> select count(1) from load0_mm;{code}
> Expected output
> {code:java}
> PREHOOK: query: load data local inpath '../../data/files/kv2.txt' into table
> load0_mm
> PREHOOK: type: LOAD
> A masked pattern was here
> PREHOOK: Output: default@load0_mm
> POSTHOOK: query: load data local inpath '../../data/files/kv2.txt' into table
> load0_mm
> POSTHOOK: type: LOAD
> A masked pattern was here
> POSTHOOK: Output: default@load0_mm
> PREHOOK: query: select count(1) from load0_mm
> PREHOOK: type: QUERY
> PREHOOK: Input: default@load0_mm
> A masked pattern was here
> POSTHOOK: query: select count(1) from load0_mm
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@load0_mm
> A masked pattern was here
> 1500{code}
> Got:
> [ERROR] TestMiniLlapLocalCliDriver.testCliDriver:59 Client Execution
> succeeded but contained differences (error code = 1) after executing
> mm_loaddata.q
> 63c63
> < 1480
> —
> > 1500
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22055) select count gives incorrect result after loading data from text file
[
https://issues.apache.org/jira/browse/HIVE-22055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Magyar updated HIVE-22055:
-
Status: Patch Available (was: Open)
> select count gives incorrect result after loading data from text file
> -
>
> Key: HIVE-22055
> URL: https://issues.apache.org/jira/browse/HIVE-22055
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Attila Magyar
>Assignee: Attila Magyar
>Priority: Major
> Attachments: HIVE-22055.1.patch, HIVE-22055.2.patch
>
>
> Add one more load to mm_loaddata.q:
> Load data 3 times (both kv1.txt and kv2.txt contains 500 records)
> {code:java}
> create table load0_mm (key string, value string) stored as textfile
> tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> load data local inpath '../../data/files/kv1.txt' into table load0_mm;
> select count(1) from load0_mm;
> load data local inpath '../../data/files/kv2.txt' into table load0_mm;
> select count(1) from load0_mm;
> load data local inpath '../../data/files/kv2.txt' into table load0_mm;
> select count(1) from load0_mm;{code}
> Expected output
> {code:java}
> PREHOOK: query: load data local inpath '../../data/files/kv2.txt' into table
> load0_mm
> PREHOOK: type: LOAD
> A masked pattern was here
> PREHOOK: Output: default@load0_mm
> POSTHOOK: query: load data local inpath '../../data/files/kv2.txt' into table
> load0_mm
> POSTHOOK: type: LOAD
> A masked pattern was here
> POSTHOOK: Output: default@load0_mm
> PREHOOK: query: select count(1) from load0_mm
> PREHOOK: type: QUERY
> PREHOOK: Input: default@load0_mm
> A masked pattern was here
> POSTHOOK: query: select count(1) from load0_mm
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@load0_mm
> A masked pattern was here
> 1500{code}
> Got:
> [ERROR] TestMiniLlapLocalCliDriver.testCliDriver:59 Client Execution
> succeeded but contained differences (error code = 1) after executing
> mm_loaddata.q
> 63c63
> < 1480
> —
> > 1500
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
