[jira] [Resolved] (HIVE-22792) Fix NPE in VectorColumnOutputMapping.finalize
[
https://issues.apache.org/jira/browse/HIVE-22792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Balamohan resolved HIVE-22792.
-
Resolution: Duplicate
Looks like HIVE-22252 fixed this. Closing this ticket.
> Fix NPE in VectorColumnOutputMapping.finalize
> -
>
> Key: HIVE-22792
> URL: https://issues.apache.org/jira/browse/HIVE-22792
> Project: Hive
> Issue Type: Improvement
> Components: Vectorization
>Reporter: Rajesh Balamohan
>Priority: Minor
> Attachments: Screenshot 2020-01-30 at 1.09.51 PM.png
>
>
> !Screenshot 2020-01-30 at 1.09.51 PM.png!
> Vectorizer already invokes finalize() explicitly making
> {{vectorColumnMapping}} null.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22792) Fix NPE in VectorColumnOutputMapping.finalize
[
https://issues.apache.org/jira/browse/HIVE-22792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Balamohan updated HIVE-22792:
Description:
!Screenshot 2020-01-30 at 1.09.51 PM.png!
Vectorizer already invokes finalize() explicitly making {{vectorColumnMapping}}
null.
was:
Vectorizer already invokes finalize() explicitly making {{vectorColumnMapping}}
null.
> Fix NPE in VectorColumnOutputMapping.finalize
> -
>
> Key: HIVE-22792
> URL: https://issues.apache.org/jira/browse/HIVE-22792
> Project: Hive
> Issue Type: Improvement
> Components: Vectorization
>Reporter: Rajesh Balamohan
>Priority: Minor
> Attachments: Screenshot 2020-01-30 at 1.09.51 PM.png
>
>
> !Screenshot 2020-01-30 at 1.09.51 PM.png!
> Vectorizer already invokes finalize() explicitly making
> {{vectorColumnMapping}} null.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22792) Fix NPE in VectorColumnOutputMapping.finalize
[
https://issues.apache.org/jira/browse/HIVE-22792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Balamohan updated HIVE-22792:
Attachment: (was: Screenshot 2020-01-30 at 1.06.37 PM.png)
> Fix NPE in VectorColumnOutputMapping.finalize
> -
>
> Key: HIVE-22792
> URL: https://issues.apache.org/jira/browse/HIVE-22792
> Project: Hive
> Issue Type: Improvement
> Components: Vectorization
>Reporter: Rajesh Balamohan
>Priority: Minor
> Attachments: Screenshot 2020-01-30 at 1.09.51 PM.png
>
>
> Vectorizer already invokes finalize() explicitly making
> {{vectorColumnMapping}} null.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22792) Fix NPE in VectorColumnOutputMapping.finalize
[
https://issues.apache.org/jira/browse/HIVE-22792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Balamohan updated HIVE-22792:
Attachment: Screenshot 2020-01-30 at 1.09.51 PM.png
> Fix NPE in VectorColumnOutputMapping.finalize
> -
>
> Key: HIVE-22792
> URL: https://issues.apache.org/jira/browse/HIVE-22792
> Project: Hive
> Issue Type: Improvement
> Components: Vectorization
>Reporter: Rajesh Balamohan
>Priority: Minor
> Attachments: Screenshot 2020-01-30 at 1.09.51 PM.png
>
>
> !Screenshot 2020-01-30 at 1.06.37 PM.png!
> Vectorizer already invokes finalize() explicitly making
> {{vectorColumnMapping}} null.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22792) Fix NPE in VectorColumnOutputMapping.finalize
[
https://issues.apache.org/jira/browse/HIVE-22792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Balamohan updated HIVE-22792:
Description:
Vectorizer already invokes finalize() explicitly making {{vectorColumnMapping}}
null.
was:
!Screenshot 2020-01-30 at 1.06.37 PM.png!
Vectorizer already invokes finalize() explicitly making {{vectorColumnMapping}}
null.
> Fix NPE in VectorColumnOutputMapping.finalize
> -
>
> Key: HIVE-22792
> URL: https://issues.apache.org/jira/browse/HIVE-22792
> Project: Hive
> Issue Type: Improvement
> Components: Vectorization
>Reporter: Rajesh Balamohan
>Priority: Minor
> Attachments: Screenshot 2020-01-30 at 1.09.51 PM.png
>
>
> Vectorizer already invokes finalize() explicitly making
> {{vectorColumnMapping}} null.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17026417#comment-17026417
]
Shivam commented on HIVE-22771:
---
Thanks you [~jcamachorodriguez] and [~mithun] for you comments.
I have added the test cases to write to static partitions in _TestHCatStorer_.
Can you please review the changes and let me know if they look okay?
Thank you!
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Assignee: Shivam
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-22771.2.patch, HIVE-22771.3.patch,
> HIVE-22771.3.patch, HIVE-22771.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate _idHash_ using below statement:
> *+In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+*
> *line 116: idHash = String.valueOf(Math.random());*
> The output of idHash can be similar to values like this : 7.145347157239135E-4
>
> And, in class _FileOutputCommitterContainer_ in package
> _org.apache.hive.hcatalog.mapreduce;_
> Uses below statement to compute final partition path:
> +*In org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java*+
> *line 366: String finalLocn = jobLocation.replaceAll(Path.SEPARATOR +
> SCRATCH_DIR_NAME + "{color:#ff}\\d\\.?
> d+"{color},"");*
> *line 367: partPath = new Path(finalLocn);*
>
> Regex used here is incorrect, since it will only remove integers after the
> *SCRATCH_DIR_NAME,* and hence will append 'E-4' (for the above example) in
> the final partition location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22777) Sorted dynamic partition optimization doesn't work if plan require implicit cast
[
https://issues.apache.org/jira/browse/HIVE-22777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17026363#comment-17026363
]
Hive QA commented on HIVE-22777:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992173/HIVE-22777.4.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 17958 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_num_reducers]
(batchId=184)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[enforce_constraint_notnull]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[bucket4]
(batchId=199)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[bucket5]
(batchId=200)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[disable_merge_for_bucketing]
(batchId=200)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[bucket2]
(batchId=139)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[bucket3]
(batchId=123)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[bucket4]
(batchId=151)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[disable_merge_for_bucketing]
(batchId=152)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[smb_mapjoin_20]
(batchId=150)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[stats10]
(batchId=126)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20408/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20408/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20408/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12992173 - PreCommit-HIVE-Build
> Sorted dynamic partition optimization doesn't work if plan require implicit
> cast
>
>
> Key: HIVE-22777
> URL: https://issues.apache.org/jira/browse/HIVE-22777
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22777.1.patch, HIVE-22777.2.patch,
> HIVE-22777.3.patch, HIVE-22777.4.patch
>
>
> *Repro*
> set hive.stats.autogather=false;
> set hive.optimize.sort.dynamic.partition.threshold=1;
> set hive.optimize.bucketingsorting = true;
> {code:sql}
> drop table if exists t1_staging;
> create table t1_staging(
>a string,
>b int,
>c int,
>d string)
> partitioned by (e decimal(18,0))
> clustered by(a)
> into 256 buckets STORED AS TEXTFILE;
> load data local inpath '../../data/files/sortdp/00_0' overwrite into
> table t1_staging partition (e=100);
> drop table t1_n147;
> create table t1_n147(
> a string,
> b decimal(6,0),
> c int,
> d string)
> partitioned by (e decimal(3,0))
> clustered by(a,b)
> into 10 buckets STORED AS ORC TBLPROPERTIES ('transactional'='true');
> set hive.stats.autogather=false;
> set hive.optimize.bucketingsorting = true;
> explain insert overwrite table t1_n147 partition(e) select a,b,c,d,e from
> t1_staging;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22777) Sorted dynamic partition optimization doesn't work if plan require implicit cast
[
https://issues.apache.org/jira/browse/HIVE-22777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17026340#comment-17026340
]
Hive QA commented on HIVE-22777:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
23s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
36s{color} | {color:blue} ql in master has 1537 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
45s{color} | {color:red} ql: The patch generated 3 new + 362 unchanged - 0
fixed = 365 total (was 362) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 10s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20408/dev-support/hive-personality.sh
|
| git revision | master / 2412f73 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20408/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20408/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Sorted dynamic partition optimization doesn't work if plan require implicit
> cast
>
>
> Key: HIVE-22777
> URL: https://issues.apache.org/jira/browse/HIVE-22777
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22777.1.patch, HIVE-22777.2.patch,
> HIVE-22777.3.patch, HIVE-22777.4.patch
>
>
> *Repro*
> set hive.stats.autogather=false;
> set hive.optimize.sort.dynamic.partition.threshold=1;
> set hive.optimize.bucketingsorting = true;
> {code:sql}
> drop table if exists t1_staging;
> create table t1_staging(
>a string,
>b int,
>c int,
>d string)
> partitioned by (e decimal(18,0))
> clustered by(a)
> into 256 buckets STORED AS TEXTFILE;
> load data local inpath '../../data/files/sortdp/00_0' overwrite into
> table t1_staging partition (e=100);
> drop table t1_n147;
> create table t1_n147(
> a string,
>
[jira] [Resolved] (HIVE-22789) Table details are retrieved twice during query compilation phase
[
https://issues.apache.org/jira/browse/HIVE-22789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Balamohan resolved HIVE-22789.
-
Resolution: Duplicate
Appears that HIVE-22366 takes care of this in master branch. Marking this as
dup.
> Table details are retrieved twice during query compilation phase
>
>
> Key: HIVE-22789
> URL: https://issues.apache.org/jira/browse/HIVE-22789
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Reporter: Rajesh Balamohan
>Priority: Minor
>
> https://issues.apache.org/jira/browse/HIVE-22366 takes care of normalizing
> table details so that {{tabNameToTabObject}} is effectively used in
> CalcitePlanner.
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L513
> effectively invalidates entire {{tabNameToTabObject}} cache and forces to
> recompute the details. It would be good to check whether this can be avoided.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22777) Sorted dynamic partition optimization doesn't work if plan require implicit cast
[
https://issues.apache.org/jira/browse/HIVE-22777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-22777:
---
Status: Patch Available (was: Open)
> Sorted dynamic partition optimization doesn't work if plan require implicit
> cast
>
>
> Key: HIVE-22777
> URL: https://issues.apache.org/jira/browse/HIVE-22777
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22777.1.patch, HIVE-22777.2.patch,
> HIVE-22777.3.patch, HIVE-22777.4.patch
>
>
> *Repro*
> set hive.stats.autogather=false;
> set hive.optimize.sort.dynamic.partition.threshold=1;
> set hive.optimize.bucketingsorting = true;
> {code:sql}
> drop table if exists t1_staging;
> create table t1_staging(
>a string,
>b int,
>c int,
>d string)
> partitioned by (e decimal(18,0))
> clustered by(a)
> into 256 buckets STORED AS TEXTFILE;
> load data local inpath '../../data/files/sortdp/00_0' overwrite into
> table t1_staging partition (e=100);
> drop table t1_n147;
> create table t1_n147(
> a string,
> b decimal(6,0),
> c int,
> d string)
> partitioned by (e decimal(3,0))
> clustered by(a,b)
> into 10 buckets STORED AS ORC TBLPROPERTIES ('transactional'='true');
> set hive.stats.autogather=false;
> set hive.optimize.bucketingsorting = true;
> explain insert overwrite table t1_n147 partition(e) select a,b,c,d,e from
> t1_staging;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22777) Sorted dynamic partition optimization doesn't work if plan require implicit cast
[
https://issues.apache.org/jira/browse/HIVE-22777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-22777:
---
Attachment: HIVE-22777.4.patch
> Sorted dynamic partition optimization doesn't work if plan require implicit
> cast
>
>
> Key: HIVE-22777
> URL: https://issues.apache.org/jira/browse/HIVE-22777
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22777.1.patch, HIVE-22777.2.patch,
> HIVE-22777.3.patch, HIVE-22777.4.patch
>
>
> *Repro*
> set hive.stats.autogather=false;
> set hive.optimize.sort.dynamic.partition.threshold=1;
> set hive.optimize.bucketingsorting = true;
> {code:sql}
> drop table if exists t1_staging;
> create table t1_staging(
>a string,
>b int,
>c int,
>d string)
> partitioned by (e decimal(18,0))
> clustered by(a)
> into 256 buckets STORED AS TEXTFILE;
> load data local inpath '../../data/files/sortdp/00_0' overwrite into
> table t1_staging partition (e=100);
> drop table t1_n147;
> create table t1_n147(
> a string,
> b decimal(6,0),
> c int,
> d string)
> partitioned by (e decimal(3,0))
> clustered by(a,b)
> into 10 buckets STORED AS ORC TBLPROPERTIES ('transactional'='true');
> set hive.stats.autogather=false;
> set hive.optimize.bucketingsorting = true;
> explain insert overwrite table t1_n147 partition(e) select a,b,c,d,e from
> t1_staging;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22777) Sorted dynamic partition optimization doesn't work if plan require implicit cast
[
https://issues.apache.org/jira/browse/HIVE-22777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-22777:
---
Status: Open (was: Patch Available)
> Sorted dynamic partition optimization doesn't work if plan require implicit
> cast
>
>
> Key: HIVE-22777
> URL: https://issues.apache.org/jira/browse/HIVE-22777
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22777.1.patch, HIVE-22777.2.patch,
> HIVE-22777.3.patch, HIVE-22777.4.patch
>
>
> *Repro*
> set hive.stats.autogather=false;
> set hive.optimize.sort.dynamic.partition.threshold=1;
> set hive.optimize.bucketingsorting = true;
> {code:sql}
> drop table if exists t1_staging;
> create table t1_staging(
>a string,
>b int,
>c int,
>d string)
> partitioned by (e decimal(18,0))
> clustered by(a)
> into 256 buckets STORED AS TEXTFILE;
> load data local inpath '../../data/files/sortdp/00_0' overwrite into
> table t1_staging partition (e=100);
> drop table t1_n147;
> create table t1_n147(
> a string,
> b decimal(6,0),
> c int,
> d string)
> partitioned by (e decimal(3,0))
> clustered by(a,b)
> into 10 buckets STORED AS ORC TBLPROPERTIES ('transactional'='true');
> set hive.stats.autogather=false;
> set hive.optimize.bucketingsorting = true;
> explain insert overwrite table t1_n147 partition(e) select a,b,c,d,e from
> t1_staging;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22515) Support cast to decimal64 in Vectorization
[ https://issues.apache.org/jira/browse/HIVE-22515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-22515: Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master. Thanks, Ramesh! > Support cast to decimal64 in Vectorization > -- > > Key: HIVE-22515 > URL: https://issues.apache.org/jira/browse/HIVE-22515 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22515.11.patch, HIVE-22515.5.patch, > HIVE-22515.8.patch, HIVE-22515.9.patch > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Support cast to decimal64 in Vectorization -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22726) TopN Key optimizer should use array instead of priority queue
[ https://issues.apache.org/jira/browse/HIVE-22726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-22726: Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master. Thanks, Attila! > TopN Key optimizer should use array instead of priority queue > - > > Key: HIVE-22726 > URL: https://issues.apache.org/jira/browse/HIVE-22726 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Attila Magyar >Assignee: Attila Magyar >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22726.1.patch, HIVE-22726.2.patch, > HIVE-22726.3.patch, HIVE-22726.4.patch, HIVE-22726.5.patch > > > The TopN key optimizer currently uses a priority queue for keeping track of > the largest/smallest rows. Its max size is the same as the user specified > limit. This should be replaced a more cache line friendly array with a small > (128) maximum size and see how much performance is gained. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22777) Sorted dynamic partition optimization doesn't work if plan require implicit cast
[
https://issues.apache.org/jira/browse/HIVE-22777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17026302#comment-17026302
]
Hive QA commented on HIVE-22777:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992054/HIVE-22777.3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20407/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20407/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20407/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12992054/HIVE-22777.3.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12992054 - PreCommit-HIVE-Build
> Sorted dynamic partition optimization doesn't work if plan require implicit
> cast
>
>
> Key: HIVE-22777
> URL: https://issues.apache.org/jira/browse/HIVE-22777
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22777.1.patch, HIVE-22777.2.patch,
> HIVE-22777.3.patch
>
>
> *Repro*
> set hive.stats.autogather=false;
> set hive.optimize.sort.dynamic.partition.threshold=1;
> set hive.optimize.bucketingsorting = true;
> {code:sql}
> drop table if exists t1_staging;
> create table t1_staging(
>a string,
>b int,
>c int,
>d string)
> partitioned by (e decimal(18,0))
> clustered by(a)
> into 256 buckets STORED AS TEXTFILE;
> load data local inpath '../../data/files/sortdp/00_0' overwrite into
> table t1_staging partition (e=100);
> drop table t1_n147;
> create table t1_n147(
> a string,
> b decimal(6,0),
> c int,
> d string)
> partitioned by (e decimal(3,0))
> clustered by(a,b)
> into 10 buckets STORED AS ORC TBLPROPERTIES ('transactional'='true');
> set hive.stats.autogather=false;
> set hive.optimize.bucketingsorting = true;
> explain insert overwrite table t1_n147 partition(e) select a,b,c,d,e from
> t1_staging;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22726) TopN Key optimizer should use array instead of priority queue
[ https://issues.apache.org/jira/browse/HIVE-22726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17026299#comment-17026299 ] Gopal Vijayaraghavan commented on HIVE-22726: - LGTM - +1 > TopN Key optimizer should use array instead of priority queue > - > > Key: HIVE-22726 > URL: https://issues.apache.org/jira/browse/HIVE-22726 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Attila Magyar >Assignee: Attila Magyar >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22726.1.patch, HIVE-22726.2.patch, > HIVE-22726.3.patch, HIVE-22726.4.patch, HIVE-22726.5.patch > > > The TopN key optimizer currently uses a priority queue for keeping track of > the largest/smallest rows. Its max size is the same as the user specified > limit. This should be replaced a more cache line friendly array with a small > (128) maximum size and see how much performance is gained. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17026298#comment-17026298
]
Hive QA commented on HIVE-22771:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992150/HIVE-22771.3.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17969 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20406/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20406/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20406/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12992150 - PreCommit-HIVE-Build
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Assignee: Shivam
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-22771.2.patch, HIVE-22771.3.patch,
> HIVE-22771.3.patch, HIVE-22771.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate _idHash_ using below statement:
> *+In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+*
> *line 116: idHash = String.valueOf(Math.random());*
> The output of idHash can be similar to values like this : 7.145347157239135E-4
>
> And, in class _FileOutputCommitterContainer_ in package
> _org.apache.hive.hcatalog.mapreduce;_
> Uses below statement to compute final partition path:
> +*In org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java*+
> *line 366: String finalLocn = jobLocation.replaceAll(Path.SEPARATOR +
> SCRATCH_DIR_NAME + "{color:#ff}\\d\\.?
> d+"{color},"");*
> *line 367: partPath = new Path(finalLocn);*
>
> Regex used here is incorrect, since it will only remove integers after the
> *SCRATCH_DIR_NAME,* and hence will append 'E-4' (for the above example) in
> the final partition location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Reopened] (HIVE-20370) Vectorization: Add Native Vector MapJoin hash table optimization for Left/Right Outer Joins when there are no Small Table values
[ https://issues.apache.org/jira/browse/HIVE-20370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan reopened HIVE-20370: - Assignee: (was: Matt McCline) > Vectorization: Add Native Vector MapJoin hash table optimization for > Left/Right Outer Joins when there are no Small Table values > > > Key: HIVE-20370 > URL: https://issues.apache.org/jira/browse/HIVE-20370 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Priority: Critical > Attachments: HIVE-20370.01.patch, HIVE-20370.02.patch, > HIVE-20370.03.patch, HIVE-20370.04.patch > > > Similar to Native Vector MapJoin's InnerBigOnly optimization that uses an > efficient Hash Multi-Set with a counter instead of a Hash Map with an empty > value, do the same for Outer joins. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22788) Query cause NPE due to implicit cast on ROW__ID
[
https://issues.apache.org/jira/browse/HIVE-22788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-22788:
---
Fix Version/s: 4.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to master. Thanks [~kkasa]
> Query cause NPE due to implicit cast on ROW__ID
> ---
>
> Key: HIVE-22788
> URL: https://issues.apache.org/jira/browse/HIVE-22788
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Krisztian Kasa
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22788.1.patch, HIVE-22788.2.patch,
> HIVE-22788.3.patch
>
>
> *Repro*
> {code:sql}
> CREATE TABLE table_16 (
> timestamp_col_19timestamp,
> timestamp_col_29timestamp,
> int_col_27 int,
> int_col_39 int,
> boolean_col_18 boolean,
> varchar0045_col_23 varchar(45)
> );
> CREATE TABLE table_7 (
> int_col_10 int,
> bigint_col_3bigint
> );
> CREATE TABLE table_10 (
> boolean_col_8 boolean,
> boolean_col_16 boolean,
> timestamp_col_5 timestamp,
> timestamp_col_15timestamp,
> timestamp_col_30timestamp,
> decimal3825_col_26 decimal(38, 25),
> smallint_col_9 smallint,
> int_col_18 int
> );
> explain cbo
> SELECT
> DISTINCT COALESCE(a4.timestamp_col_15, IF(a4.boolean_col_16,
> a4.timestamp_col_30, a4.timestamp_col_5)) AS timestamp_col
> FROM table_7 a3
> RIGHT JOIN table_10 a4
> WHERE (a3.bigint_col_3) >= (a4.int_col_18)
> INTERSECT ALL
> SELECT
> COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> ) AS timestamp_col
> FROM table_16 a1
> GROUP BY COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(
> COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> );
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17026271#comment-17026271
]
Hive QA commented on HIVE-22771:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
40s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} hcatalog/core in master has 36 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
29s{color} | {color:blue} hcatalog/hcatalog-pig-adapter in master has 2 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} hcatalog/core: The patch generated 0 new + 39
unchanged - 1 fixed = 39 total (was 40) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} The patch hcatalog-pig-adapter passed checkstyle
{color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 4s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20406/dev-support/hive-personality.sh
|
| git revision | master / 03346cf |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: hcatalog/core hcatalog/hcatalog-pig-adapter U: hcatalog |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20406/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Assignee: Shivam
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-22771.2.patch, HIVE-22771.3.patch,
> HIVE-22771.3.patch, HIVE-22771.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate _idHash_ using below statement:
> *+In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+*
> *line 116: idHash = String.valueOf(Math.random());*
[jira] [Commented] (HIVE-22358) Add schedule shorthands for convinience
[
https://issues.apache.org/jira/browse/HIVE-22358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17026246#comment-17026246
]
Hive QA commented on HIVE-22358:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992129/HIVE-22358.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17962 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestRemoteUGIHiveMetaStoreIpAddress.testIpAddress
(batchId=235)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20405/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20405/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20405/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12992129 - PreCommit-HIVE-Build
> Add schedule shorthands for convinience
> ---
>
> Key: HIVE-22358
> URL: https://issues.apache.org/jira/browse/HIVE-22358
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22358.01.patch
>
>
> Add shorthands for schedules
> {code}
> every minute
> every 10 minutes
> every 10 seconds
> every 4 hours
> every 4 hours offset by '2:03:04'
> every day offset by '2:03:04'
> every day at '2:03:04'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22358) Add schedule shorthands for convinience
[
https://issues.apache.org/jira/browse/HIVE-22358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17026193#comment-17026193
]
Hive QA commented on HIVE-22358:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
44s{color} | {color:blue} ql in master has 1537 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
43s{color} | {color:red} ql: The patch generated 2 new + 8 unchanged - 0 fixed
= 10 total (was 8) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20405/dev-support/hive-personality.sh
|
| git revision | master / 03346cf |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20405/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20405/yetus/whitespace-eol.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20405/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add schedule shorthands for convinience
> ---
>
> Key: HIVE-22358
> URL: https://issues.apache.org/jira/browse/HIVE-22358
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22358.01.patch
>
>
> Add shorthands for schedules
> {code}
> every minute
> every 10 minutes
> every 10 seconds
> every 4 hours
> every 4 hours offset by '2:03:04'
> every day offset by '2:03:04'
> every day at '2:03:04'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22788) Query cause NPE due to implicit cast on ROW__ID
[
https://issues.apache.org/jira/browse/HIVE-22788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17026175#comment-17026175
]
Hive QA commented on HIVE-22788:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992121/HIVE-22788.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17956 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20404/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20404/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20404/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12992121 - PreCommit-HIVE-Build
> Query cause NPE due to implicit cast on ROW__ID
> ---
>
> Key: HIVE-22788
> URL: https://issues.apache.org/jira/browse/HIVE-22788
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22788.1.patch, HIVE-22788.2.patch,
> HIVE-22788.3.patch
>
>
> *Repro*
> {code:sql}
> CREATE TABLE table_16 (
> timestamp_col_19timestamp,
> timestamp_col_29timestamp,
> int_col_27 int,
> int_col_39 int,
> boolean_col_18 boolean,
> varchar0045_col_23 varchar(45)
> );
> CREATE TABLE table_7 (
> int_col_10 int,
> bigint_col_3bigint
> );
> CREATE TABLE table_10 (
> boolean_col_8 boolean,
> boolean_col_16 boolean,
> timestamp_col_5 timestamp,
> timestamp_col_15timestamp,
> timestamp_col_30timestamp,
> decimal3825_col_26 decimal(38, 25),
> smallint_col_9 smallint,
> int_col_18 int
> );
> explain cbo
> SELECT
> DISTINCT COALESCE(a4.timestamp_col_15, IF(a4.boolean_col_16,
> a4.timestamp_col_30, a4.timestamp_col_5)) AS timestamp_col
> FROM table_7 a3
> RIGHT JOIN table_10 a4
> WHERE (a3.bigint_col_3) >= (a4.int_col_18)
> INTERSECT ALL
> SELECT
> COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> ) AS timestamp_col
> FROM table_16 a1
> GROUP BY COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(
> COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> );
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22589) Add storage support for ProlepticCalendar
[
https://issues.apache.org/jira/browse/HIVE-22589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-22589:
---
Attachment: HIVE-22589.04.patch
> Add storage support for ProlepticCalendar
> -
>
> Key: HIVE-22589
> URL: https://issues.apache.org/jira/browse/HIVE-22589
> Project: Hive
> Issue Type: Bug
> Components: Avro, ORC, Parquet
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Fix For: 4.0.0, 3.2.0, 3.1.3
>
> Attachments: HIVE-22589.01.patch, HIVE-22589.02.patch,
> HIVE-22589.03.patch, HIVE-22589.04.patch, HIVE-22589.patch, HIVE-22589.patch
>
>
> Hive recently moved its processing to the proleptic calendar, which has
> created some issues for users who have dates before 1580 AD.
> I'd propose extending the column vectors for times & dates to encode which
> calendar they are using.
> * create DateColumnVector that extends LongColumnVector
> * add a method to change calendars to both DateColumnVector and
> TimestampColumnVector.
> {code}
> /**
>* Change the calendar to or from proleptic. If the new and old values of
> the flag are the
>* same, nothing is done.
>* useProleptic - set the flag for the proleptic calendar
>* updateData - change the data to match the new value of the flag.
>*/
> void changeCalendar(useProleptic: boolean, updateData: boolean);
> /**
>* Detect whether this data is using the proleptic calendar.
>*/
> boolean usingProlepticCalendar();
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22589) Add storage support for ProlepticCalendar
[
https://issues.apache.org/jira/browse/HIVE-22589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-22589:
---
Attachment: HIVE-22589.03.patch
> Add storage support for ProlepticCalendar
> -
>
> Key: HIVE-22589
> URL: https://issues.apache.org/jira/browse/HIVE-22589
> Project: Hive
> Issue Type: Bug
> Components: Avro, ORC, Parquet
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Fix For: 4.0.0, 3.2.0, 3.1.3
>
> Attachments: HIVE-22589.01.patch, HIVE-22589.02.patch,
> HIVE-22589.03.patch, HIVE-22589.patch, HIVE-22589.patch
>
>
> Hive recently moved its processing to the proleptic calendar, which has
> created some issues for users who have dates before 1580 AD.
> I'd propose extending the column vectors for times & dates to encode which
> calendar they are using.
> * create DateColumnVector that extends LongColumnVector
> * add a method to change calendars to both DateColumnVector and
> TimestampColumnVector.
> {code}
> /**
>* Change the calendar to or from proleptic. If the new and old values of
> the flag are the
>* same, nothing is done.
>* useProleptic - set the flag for the proleptic calendar
>* updateData - change the data to match the new value of the flag.
>*/
> void changeCalendar(useProleptic: boolean, updateData: boolean);
> /**
>* Detect whether this data is using the proleptic calendar.
>*/
> boolean usingProlepticCalendar();
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22784) Boundary condition to check if there is nothing to truncate in StringExpr functions
[ https://issues.apache.org/jira/browse/HIVE-22784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17026128#comment-17026128 ] Gopal Vijayaraghavan commented on HIVE-22784: - LGTM - +1 > Boundary condition to check if there is nothing to truncate in StringExpr > functions > --- > > Key: HIVE-22784 > URL: https://issues.apache.org/jira/browse/HIVE-22784 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Attachments: HIVE-22784.1.patch, HIVE-22784.2.patch, > HIVE-22784.3.patch > > > Add a boundary condition to check if there is nothing to truncate in > StringExpr functions > truncate() > rightTrimAndTruncate() -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22784) Boundary condition to check if there is nothing to truncate in StringExpr functions
[ https://issues.apache.org/jira/browse/HIVE-22784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-22784: Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master. Thanks, Ramesh! > Boundary condition to check if there is nothing to truncate in StringExpr > functions > --- > > Key: HIVE-22784 > URL: https://issues.apache.org/jira/browse/HIVE-22784 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22784.1.patch, HIVE-22784.2.patch, > HIVE-22784.3.patch > > > Add a boundary condition to check if there is nothing to truncate in > StringExpr functions > truncate() > rightTrimAndTruncate() -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22788) Query cause NPE due to implicit cast on ROW__ID
[
https://issues.apache.org/jira/browse/HIVE-22788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17026125#comment-17026125
]
Hive QA commented on HIVE-22788:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
26s{color} | {color:blue} ql in master has 1537 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20404/dev-support/hive-personality.sh
|
| git revision | master / 6eca44e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20404/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Query cause NPE due to implicit cast on ROW__ID
> ---
>
> Key: HIVE-22788
> URL: https://issues.apache.org/jira/browse/HIVE-22788
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22788.1.patch, HIVE-22788.2.patch,
> HIVE-22788.3.patch
>
>
> *Repro*
> {code:sql}
> CREATE TABLE table_16 (
> timestamp_col_19timestamp,
> timestamp_col_29timestamp,
> int_col_27 int,
> int_col_39 int,
> boolean_col_18 boolean,
> varchar0045_col_23 varchar(45)
> );
> CREATE TABLE table_7 (
> int_col_10 int,
> bigint_col_3bigint
> );
> CREATE TABLE table_10 (
> boolean_col_8 boolean,
> boolean_col_16 boolean,
> timestamp_col_5 timestamp,
> timestamp_col_15timestamp,
> timestamp_col_30timestamp,
> decimal3825_col_26 decimal(38, 25),
> smallint_col_9 smallint,
> int_col_18 int
> );
> explain cbo
> SELECT
> DISTINCT COALESCE(a4.timestamp_col_15, IF(a4.boolean_col_16,
> a4.timestamp_col_30, a4.timestamp_col_5)) AS timestamp_col
> FROM table_7 a3
> RIGHT JOIN table_10 a4
> WHERE (a3.bigint_col_3) >= (a4.int_col_18)
> INTERSECT ALL
> SELECT
> COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CA
[jira] [Commented] (HIVE-22358) Add schedule shorthands for convinience
[
https://issues.apache.org/jira/browse/HIVE-22358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17026124#comment-17026124
]
Jesus Camacho Rodriguez commented on HIVE-22358:
[~kgyrtkirk], LGTM. For more complicated schedules, users should defer to
Quartz which gives you all the expressive power you may need. Please, make sure
once this goes in, the shorthands are added to the scheduled queries
documentation.
Concerning the patch, a couple of comments:
- Make sure you add new keywords to {{IdentifiersParser.g}} if they are
non-reserved SQL keywords (if there are any keywords related to scheduled
queries that we missed in previous patch, please do the same).
- Expand the import .* into the individual classes, since that is the
convention that we use.
> Add schedule shorthands for convinience
> ---
>
> Key: HIVE-22358
> URL: https://issues.apache.org/jira/browse/HIVE-22358
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22358.01.patch
>
>
> Add shorthands for schedules
> {code}
> every minute
> every 10 minutes
> every 10 seconds
> every 4 hours
> every 4 hours offset by '2:03:04'
> every day offset by '2:03:04'
> every day at '2:03:04'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22736) Support replication across multiple encryption zones
[
https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17026097#comment-17026097
]
Hive QA commented on HIVE-22736:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992117/HIVE-22736.patch
{color:green}SUCCESS:{color} +1 due to 5 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17956 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestMetaStoreMultipleEncryptionZones.org.apache.hadoop.hive.metastore.TestMetaStoreMultipleEncryptionZones
(batchId=247)
org.apache.hive.hcatalog.templeton.TestConcurrentJobRequestsThreadsAndTimeout.ConcurrentListJobsVerifyExceptions
(batchId=212)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20403/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20403/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20403/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12992117 - PreCommit-HIVE-Build
> Support replication across multiple encryption zones
>
>
> Key: HIVE-22736
> URL: https://issues.apache.org/jira/browse/HIVE-22736
> Project: Hive
> Issue Type: Task
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22358) Add schedule shorthands for convinience
[
https://issues.apache.org/jira/browse/HIVE-22358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17026088#comment-17026088
]
Zoltan Haindrich commented on HIVE-22358:
-
[~jcamachorodriguez] what do you think about the above shorthands? will we miss
any common usecases?
I can get into more complicated forms of "every day"; but that will most
certainly need thinks like MONDAY/.../SUNDAY/FIRST/LAST/?...
it could enable phrases like {code}every monday at '2:00:00'{code} or
{code}every month first monday at '2:01:22'{code}
but...about this last few; I think people will (already) reach out to the full
power of quartz expressions...
> Add schedule shorthands for convinience
> ---
>
> Key: HIVE-22358
> URL: https://issues.apache.org/jira/browse/HIVE-22358
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22358.01.patch
>
>
> Add shorthands for schedules
> {code}
> every minute
> every 10 minutes
> every 10 seconds
> every 4 hours
> every 4 hours offset by '2:03:04'
> every day offset by '2:03:04'
> every day at '2:03:04'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22736) Support replication across multiple encryption zones
[
https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17026080#comment-17026080
]
Hive QA commented on HIVE-22736:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
42s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
9s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
18s{color} | {color:blue} shims/common in master has 6 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
21s{color} | {color:blue} shims/0.23 in master has 7 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
36s{color} | {color:blue} standalone-metastore/metastore-common in master has
35 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
33s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
15s{color} | {color:blue} standalone-metastore/metastore-server in master has
181 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
9s{color} | {color:blue} ql in master has 1537 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
41s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
22s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} shims/common: The patch generated 0 new + 93
unchanged - 1 fixed = 93 total (was 94) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} The patch 0.23 passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch metastore-common passed checkstyle {color}
|
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} The patch common passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} The patch metastore-server passed checkstyle {color}
|
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} The patch ql passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} The patch hive-unit passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 10m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{c
[jira] [Resolved] (HIVE-22787) NPE when compiling query contains intersect all
[
https://issues.apache.org/jira/browse/HIVE-22787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa resolved HIVE-22787.
---
Release Note:
Fixing CBO makes this unnecessary.
Reopen if CBO=off path should be supported.
Resolution: Won't Fix
> NPE when compiling query contains intersect all
> ---
>
> Key: HIVE-22787
> URL: https://issues.apache.org/jira/browse/HIVE-22787
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
>
> The query contains INTERSECT ALL operator and one of its operands has an
> OUTER JOIN like:
> {code}
> SELECT ... FROM t1 RIGHT OUTER JOIN t2 ON ...
> INTERSECT ALL
> SELECT ...
> {code}
> In this case both AST trees (before and after calcite) has a TOK_INTERSECTALL
> node and it is not handled
> when generating the plan in
> {code}
> org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
> private Operator genPlan(QB parent, QBExpr qbexpr) throws SemanticException {
> if (qbexpr.getOpcode() == QBExpr.Opcode.NULLOP) {
> boolean skipAmbiguityCheck = viewSelect == null &&
> parent.isTopLevelSelectStarQuery();
> return genPlan(qbexpr.getQB(), skipAmbiguityCheck);
> }
> if (qbexpr.getOpcode() == QBExpr.Opcode.UNION) {
> Operator qbexpr1Ops = genPlan(parent, qbexpr.getQBExpr1());
> Operator qbexpr2Ops = genPlan(parent, qbexpr.getQBExpr2());
> return genUnionPlan(qbexpr.getAlias(), qbexpr.getQBExpr1().getAlias(),
> qbexpr1Ops, qbexpr.getQBExpr2().getAlias(), qbexpr2Ops);
> }
> return null;
> }{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22787) NPE when compiling query contains intersect all
[
https://issues.apache.org/jira/browse/HIVE-22787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17026067#comment-17026067
]
Vineet Garg commented on HIVE-22787:
[~kkasa] This will be repro only with cbo turned off right?
> NPE when compiling query contains intersect all
> ---
>
> Key: HIVE-22787
> URL: https://issues.apache.org/jira/browse/HIVE-22787
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
>
> The query contains INTERSECT ALL operator and one of its operands has an
> OUTER JOIN like:
> {code}
> SELECT ... FROM t1 RIGHT OUTER JOIN t2 ON ...
> INTERSECT ALL
> SELECT ...
> {code}
> In this case both AST trees (before and after calcite) has a TOK_INTERSECTALL
> node and it is not handled
> when generating the plan in
> {code}
> org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
> private Operator genPlan(QB parent, QBExpr qbexpr) throws SemanticException {
> if (qbexpr.getOpcode() == QBExpr.Opcode.NULLOP) {
> boolean skipAmbiguityCheck = viewSelect == null &&
> parent.isTopLevelSelectStarQuery();
> return genPlan(qbexpr.getQB(), skipAmbiguityCheck);
> }
> if (qbexpr.getOpcode() == QBExpr.Opcode.UNION) {
> Operator qbexpr1Ops = genPlan(parent, qbexpr.getQBExpr1());
> Operator qbexpr2Ops = genPlan(parent, qbexpr.getQBExpr2());
> return genUnionPlan(qbexpr.getAlias(), qbexpr.getQBExpr1().getAlias(),
> qbexpr1Ops, qbexpr.getQBExpr2().getAlias(), qbexpr2Ops);
> }
> return null;
> }{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22663) Quote all table and column names or do not quote any
[
https://issues.apache.org/jira/browse/HIVE-22663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17026061#comment-17026061
]
Zoltan Haindrich commented on HIVE-22663:
-
this patch seem to have broke some metastore db support (at least mysql) which
was working before...
> Quote all table and column names or do not quote any
>
>
> Key: HIVE-22663
> URL: https://issues.apache.org/jira/browse/HIVE-22663
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, Standalone Metastore
>Affects Versions: 4.0.0
>Reporter: Ashutosh Bapat
>Assignee: Zoltan Chovan
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22663.2.patch, HIVE-22663.3.patch,
> HIVE-22663.4.patch, HIVE-22663.5.patch, HIVE-22663.6.patch, HIVE-22663.patch
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> The change in HIVE-22546 is causing following stack trace when I run Hive
> with PostgreSQL as backend db for the metastore.
> 0: jdbc:hive2://localhost:1> create database dumpdb with
> ('repl.source.for'='1,2,3');0: jdbc:hive2://localhost:1> create database
> dumpdb with ('repl.source.for'='1,2,3');Error: Error while compiling
> statement: FAILED: ParseException line 1:28 missing KW_DBPROPERTIES at '('
> near '' (state=42000,code=4)0: jdbc:hive2://localhost:1> create
> database dumpdb with dbproperties ('repl.source.for'='1,2,3');ERROR : FAILED:
> Hive Internal Error: org.apache.hadoop.hive.ql.lockmgr.LockException(Error
> communicating with the
> metastore)org.apache.hadoop.hive.ql.lockmgr.LockException: Error
> communicating with the metastore at
> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.commitTxn(DbTxnManager.java:541)
> at
> org.apache.hadoop.hive.ql.Driver.releaseLocksAndCommitOrRollback(Driver.java:687)
> at
> org.apache.hadoop.hive.ql.Driver.releaseLocksAndCommitOrRollback(Driver.java:653)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:969)
> ... stack trace clipped
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)Caused by:
> MetaException(message:Unable to update transaction database
> org.postgresql.util.PSQLException: ERROR: relation
> "materialization_rebuild_locks" does not exist Position: 13 at
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2440)
> at
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2183)
> at
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:308)
> at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441) at
> org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365) at
> This happens because the table names in all the queries in TxnHandler.java
> (including the one at 1312, which causes this stack trace) are not quoting
> the table names. All the tablenames and column names should be quoted there.
> Just the change in HIVE-22546 won't suffice.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22788) Query cause NPE due to implicit cast on ROW__ID
[
https://issues.apache.org/jira/browse/HIVE-22788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17026059#comment-17026059
]
Jesus Camacho Rodriguez commented on HIVE-22788:
+1
> Query cause NPE due to implicit cast on ROW__ID
> ---
>
> Key: HIVE-22788
> URL: https://issues.apache.org/jira/browse/HIVE-22788
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22788.1.patch, HIVE-22788.2.patch,
> HIVE-22788.3.patch
>
>
> *Repro*
> {code:sql}
> CREATE TABLE table_16 (
> timestamp_col_19timestamp,
> timestamp_col_29timestamp,
> int_col_27 int,
> int_col_39 int,
> boolean_col_18 boolean,
> varchar0045_col_23 varchar(45)
> );
> CREATE TABLE table_7 (
> int_col_10 int,
> bigint_col_3bigint
> );
> CREATE TABLE table_10 (
> boolean_col_8 boolean,
> boolean_col_16 boolean,
> timestamp_col_5 timestamp,
> timestamp_col_15timestamp,
> timestamp_col_30timestamp,
> decimal3825_col_26 decimal(38, 25),
> smallint_col_9 smallint,
> int_col_18 int
> );
> explain cbo
> SELECT
> DISTINCT COALESCE(a4.timestamp_col_15, IF(a4.boolean_col_16,
> a4.timestamp_col_30, a4.timestamp_col_5)) AS timestamp_col
> FROM table_7 a3
> RIGHT JOIN table_10 a4
> WHERE (a3.bigint_col_3) >= (a4.int_col_18)
> INTERSECT ALL
> SELECT
> COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> ) AS timestamp_col
> FROM table_16 a1
> GROUP BY COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(
> COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> );
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shivam updated HIVE-22771:
--
Attachment: HIVE-22771.3.patch
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Assignee: Shivam
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-22771.2.patch, HIVE-22771.3.patch,
> HIVE-22771.3.patch, HIVE-22771.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate _idHash_ using below statement:
> *+In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+*
> *line 116: idHash = String.valueOf(Math.random());*
> The output of idHash can be similar to values like this : 7.145347157239135E-4
>
> And, in class _FileOutputCommitterContainer_ in package
> _org.apache.hive.hcatalog.mapreduce;_
> Uses below statement to compute final partition path:
> +*In org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java*+
> *line 366: String finalLocn = jobLocation.replaceAll(Path.SEPARATOR +
> SCRATCH_DIR_NAME + "{color:#ff}\\d\\.?
> d+"{color},"");*
> *line 367: partPath = new Path(finalLocn);*
>
> Regex used here is incorrect, since it will only remove integers after the
> *SCRATCH_DIR_NAME,* and hence will append 'E-4' (for the above example) in
> the final partition location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17026035#comment-17026035
]
Hive QA commented on HIVE-22771:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992119/HIVE-22771.3.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17969 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testShowLocksAgentInfo
(batchId=358)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20402/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20402/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20402/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12992119 - PreCommit-HIVE-Build
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Assignee: Shivam
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-22771.2.patch, HIVE-22771.3.patch, HIVE-22771.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate _idHash_ using below statement:
> *+In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+*
> *line 116: idHash = String.valueOf(Math.random());*
> The output of idHash can be similar to values like this : 7.145347157239135E-4
>
> And, in class _FileOutputCommitterContainer_ in package
> _org.apache.hive.hcatalog.mapreduce;_
> Uses below statement to compute final partition path:
> +*In org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java*+
> *line 366: String finalLocn = jobLocation.replaceAll(Path.SEPARATOR +
> SCRATCH_DIR_NAME + "{color:#ff}\\d\\.?
> d+"{color},"");*
> *line 367: partPath = new Path(finalLocn);*
>
> Regex used here is incorrect, since it will only remove integers after the
> *SCRATCH_DIR_NAME,* and hence will append 'E-4' (for the above example) in
> the final partition location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17025984#comment-17025984
]
Hive QA commented on HIVE-22771:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
46s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} hcatalog/core in master has 36 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
27s{color} | {color:blue} hcatalog/hcatalog-pig-adapter in master has 2 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} hcatalog/core: The patch generated 0 new + 39
unchanged - 1 fixed = 39 total (was 40) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch hcatalog-pig-adapter passed checkstyle
{color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 59s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20402/dev-support/hive-personality.sh
|
| git revision | master / 6eca44e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: hcatalog/core hcatalog/hcatalog-pig-adapter U: hcatalog |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20402/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Assignee: Shivam
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-22771.2.patch, HIVE-22771.3.patch, HIVE-22771.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate _idHash_ using below statement:
> *+In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+*
> *line 116: idHash = String.valueOf(Math.random());*
> The output of idHash
[jira] [Commented] (HIVE-10362) Support Type check/conversion in dynamic partition column
[
https://issues.apache.org/jira/browse/HIVE-10362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17025967#comment-17025967
]
Hive QA commented on HIVE-10362:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992115/HIVE-10362.08.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 23 failed/errored test(s), 17955 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_6]
(batchId=75)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dynpart_sort_optimization_acid2]
(batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[infer_bucket_sort_num_buckets]
(batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge10] (batchId=73)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge1] (batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge2] (batchId=103)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge_diff_fs]
(batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_join_partition_key]
(batchId=15)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_merge10]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_merge1]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_merge2]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_merge_diff_fs]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[rcfile_merge2]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_16]
(batchId=180)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynpart_sort_optimization_acid]
(batchId=177)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_smb]
(batchId=191)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_dml]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[auto_sortmerge_join_16]
(batchId=198)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[infer_bucket_sort_num_buckets]
(batchId=198)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[orc_merge1]
(batchId=197)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[orc_merge2]
(batchId=200)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[orc_merge_diff_fs]
(batchId=197)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_sortmerge_join_16]
(batchId=141)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20401/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20401/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20401/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 23 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12992115 - PreCommit-HIVE-Build
> Support Type check/conversion in dynamic partition column
> -
>
> Key: HIVE-10362
> URL: https://issues.apache.org/jira/browse/HIVE-10362
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor, Types
>Affects Versions: 1.0.0, 3.0.0, 4.0.0
>Reporter: Chaoyu Tang
>Assignee: Karen Coppage
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-10362.01.patch, HIVE-10362.02.patch,
> HIVE-10362.03.patch, HIVE-10362.04.patch, HIVE-10362.04.patch,
> HIVE-10362.06.patch, HIVE-10362.07.patch, HIVE-10362.07.patch,
> HIVE-10362.08.patch
>
>
> There are quite a lot of issues associated with the non-noramlized or
> type-mismatched values for partition column. Hive has many ways to introduce
> such problematic data. HIVE-10307 mainly provides the support to type
> check/convert/normalize the partition column value in static partition
> specification. This JIRA tries to deal with the partition column type in
> dynamic partition insert. Currently any data can be inserted as a partition
> column value as long as it is quoted as a string. For example,
> create table dynparttypechecknum (key int, value string) partitioned by (part
> int);
> insert into dynparttypechecknum partition (part) select key, value, '1'
> from src limit 1;
> show partitions dynparttypechecknum;
> --
> part=1
> The partition column value i
[jira] [Resolved] (HIVE-17388) Spark Stats for the WebUI Query Plan
[ https://issues.apache.org/jira/browse/HIVE-17388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage resolved HIVE-17388. -- Resolution: Later > Spark Stats for the WebUI Query Plan > > > Key: HIVE-17388 > URL: https://issues.apache.org/jira/browse/HIVE-17388 > Project: Hive > Issue Type: Improvement > Components: Web UI >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Minor > Labels: features, patch > Attachments: HIVE-17388.patch, running_1.png, running_2.png, > success_1.png, success_2.png, success_3.png > > > Click on a Spark stage in the WebUI/Drilldown/Query Plan graph, and Spark > task progress as well as the log file path will be displayed if > hive.server2.webui.show.stats=true. If the task is successful, > SparkStatistics will also be shown. > Screenshots attached are from a run on a CDH cluster. > Issues: > * SparkStatistics aren't shown if task fails or is running. > * Will need rebasing after HIVE-17300 is committed (current patch includes > HIVE-17300 changes) > * Will need testing upstream. > Suggestion > * It would be really easy to incorporate a progress bar to follow Spark > progress, with only a few tweaks to the JavaScript in: > service/src/resources/hive-webapps/static/js/query-plan-graph.js -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-17388) Spark Stats for the WebUI Query Plan
[ https://issues.apache.org/jira/browse/HIVE-17388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-17388: - Status: Open (was: Patch Available) > Spark Stats for the WebUI Query Plan > > > Key: HIVE-17388 > URL: https://issues.apache.org/jira/browse/HIVE-17388 > Project: Hive > Issue Type: Improvement > Components: Web UI >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Minor > Labels: features, patch > Attachments: HIVE-17388.patch, running_1.png, running_2.png, > success_1.png, success_2.png, success_3.png > > > Click on a Spark stage in the WebUI/Drilldown/Query Plan graph, and Spark > task progress as well as the log file path will be displayed if > hive.server2.webui.show.stats=true. If the task is successful, > SparkStatistics will also be shown. > Screenshots attached are from a run on a CDH cluster. > Issues: > * SparkStatistics aren't shown if task fails or is running. > * Will need rebasing after HIVE-17300 is committed (current patch includes > HIVE-17300 changes) > * Will need testing upstream. > Suggestion > * It would be really easy to incorporate a progress bar to follow Spark > progress, with only a few tweaks to the JavaScript in: > service/src/resources/hive-webapps/static/js/query-plan-graph.js -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-10362) Support Type check/conversion in dynamic partition column
[
https://issues.apache.org/jira/browse/HIVE-10362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17025923#comment-17025923
]
Hive QA commented on HIVE-10362:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
19s{color} | {color:blue} ql in master has 1537 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
40s{color} | {color:red} ql generated 1 new + 1537 unchanged - 0 fixed = 1538
total (was 1537) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 42s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Redundant nullcheck of destinationTable which is known to be null in
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genFileSinkPlan(String, QB,
Operator) Redundant null check at SemanticAnalyzer.java:is known to be null in
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genFileSinkPlan(String, QB,
Operator) Redundant null check at SemanticAnalyzer.java:[line 7677] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20401/dev-support/hive-personality.sh
|
| git revision | master / 6eca44e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20401/yetus/new-findbugs-ql.html
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20401/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support Type check/conversion in dynamic partition column
> -
>
> Key: HIVE-10362
> URL: https://issues.apache.org/jira/browse/HIVE-10362
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor, Types
>Affects Versions: 1.0.0, 3.0.0, 4.0.0
>Reporter: Chaoyu Tang
>Assignee: Karen Coppage
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-10362.01.patch, HIVE-10362.02.patch,
> HIVE-10362.03.patch, HIVE-10362.04.patch, HIVE-10362.04.patch,
> HIVE-10362.06.patch, HIVE-10362.07.patch, HIVE-10362.07.patch,
> HIVE-10362.08.patch
>
>
> There are quite a lot of issues associated with the non-noramlized or
> type-mismatched values for partition column. Hive has many ways to introduce
> such problematic data. HIVE-10307 mainly provides the support to type
> check/convert/normalize the partition column value in static partition
> specification. This JIRA tries to deal with the partition column type in
> dynamic part
[jira] [Updated] (HIVE-22770) Skip interning of MapWork fields during deserialization
[ https://issues.apache.org/jira/browse/HIVE-22770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-22770: Resolution: Fixed Status: Resolved (was: Patch Available) > Skip interning of MapWork fields during deserialization > --- > > Key: HIVE-22770 > URL: https://issues.apache.org/jira/browse/HIVE-22770 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22770.01.patch, HIVE-22770.01.patch, > HIVE-22770.02.patch, HIVE-22770.02.patch, HIVE-22770.03.patch, > HIVE-22770.04.patch, HIVE-22770.05.patch, HIVE-22770.06.patch > > > HIVE-19937 introduced some interning logic into mapwork deserialization > process, but it's only related to spark, maybe we should skip this for tez, > reducing the cpu pressure in tez tasks. > UPDATE: Hive on spark is not supported anymore, the MapWorkSerializer can be > completely removed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22770) Skip interning of MapWork fields during deserialization
[ https://issues.apache.org/jira/browse/HIVE-22770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17025897#comment-17025897 ] László Bodor commented on HIVE-22770: - pushed to master, thanks [~ashutoshc] for the review > Skip interning of MapWork fields during deserialization > --- > > Key: HIVE-22770 > URL: https://issues.apache.org/jira/browse/HIVE-22770 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Attachments: HIVE-22770.01.patch, HIVE-22770.01.patch, > HIVE-22770.02.patch, HIVE-22770.02.patch, HIVE-22770.03.patch, > HIVE-22770.04.patch, HIVE-22770.05.patch, HIVE-22770.06.patch > > > HIVE-19937 introduced some interning logic into mapwork deserialization > process, but it's only related to spark, maybe we should skip this for tez, > reducing the cpu pressure in tez tasks. > UPDATE: Hive on spark is not supported anymore, the MapWorkSerializer can be > completely removed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22770) Skip interning of MapWork fields during deserialization
[ https://issues.apache.org/jira/browse/HIVE-22770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-22770: Fix Version/s: 4.0.0 > Skip interning of MapWork fields during deserialization > --- > > Key: HIVE-22770 > URL: https://issues.apache.org/jira/browse/HIVE-22770 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22770.01.patch, HIVE-22770.01.patch, > HIVE-22770.02.patch, HIVE-22770.02.patch, HIVE-22770.03.patch, > HIVE-22770.04.patch, HIVE-22770.05.patch, HIVE-22770.06.patch > > > HIVE-19937 introduced some interning logic into mapwork deserialization > process, but it's only related to spark, maybe we should skip this for tez, > reducing the cpu pressure in tez tasks. > UPDATE: Hive on spark is not supported anymore, the MapWorkSerializer can be > completely removed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22788) Query cause NPE due to implicit cast on ROW__ID
[
https://issues.apache.org/jira/browse/HIVE-22788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17025891#comment-17025891
]
Hive QA commented on HIVE-22788:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992109/HIVE-22788.2.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17956 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20400/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20400/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20400/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12992109 - PreCommit-HIVE-Build
> Query cause NPE due to implicit cast on ROW__ID
> ---
>
> Key: HIVE-22788
> URL: https://issues.apache.org/jira/browse/HIVE-22788
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22788.1.patch, HIVE-22788.2.patch,
> HIVE-22788.3.patch
>
>
> *Repro*
> {code:sql}
> CREATE TABLE table_16 (
> timestamp_col_19timestamp,
> timestamp_col_29timestamp,
> int_col_27 int,
> int_col_39 int,
> boolean_col_18 boolean,
> varchar0045_col_23 varchar(45)
> );
> CREATE TABLE table_7 (
> int_col_10 int,
> bigint_col_3bigint
> );
> CREATE TABLE table_10 (
> boolean_col_8 boolean,
> boolean_col_16 boolean,
> timestamp_col_5 timestamp,
> timestamp_col_15timestamp,
> timestamp_col_30timestamp,
> decimal3825_col_26 decimal(38, 25),
> smallint_col_9 smallint,
> int_col_18 int
> );
> explain cbo
> SELECT
> DISTINCT COALESCE(a4.timestamp_col_15, IF(a4.boolean_col_16,
> a4.timestamp_col_30, a4.timestamp_col_5)) AS timestamp_col
> FROM table_7 a3
> RIGHT JOIN table_10 a4
> WHERE (a3.bigint_col_3) >= (a4.int_col_18)
> INTERSECT ALL
> SELECT
> COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> ) AS timestamp_col
> FROM table_16 a1
> GROUP BY COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(
> COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> );
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22358) Add schedule shorthands for convinience
[
https://issues.apache.org/jira/browse/HIVE-22358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-22358:
Description:
Add shorthands for schedules
{code}
every minute
every 10 minutes
every 10 seconds
every 4 hours
every 4 hours offset by '2:03:04'
every day offset by '2:03:04'
every day at '2:03:04'
{code}
was:
{code}
DAILY AT 12.00
{code}
> Add schedule shorthands for convinience
> ---
>
> Key: HIVE-22358
> URL: https://issues.apache.org/jira/browse/HIVE-22358
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22358.01.patch
>
>
> Add shorthands for schedules
> {code}
> every minute
> every 10 minutes
> every 10 seconds
> every 4 hours
> every 4 hours offset by '2:03:04'
> every day offset by '2:03:04'
> every day at '2:03:04'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-22358) Add schedule shorthands for convinience
[
https://issues.apache.org/jira/browse/HIVE-22358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich reassigned HIVE-22358:
---
Assignee: Zoltan Haindrich
> Add schedule shorthands for convinience
> ---
>
> Key: HIVE-22358
> URL: https://issues.apache.org/jira/browse/HIVE-22358
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22358.01.patch
>
>
> {code}
> DAILY AT 12.00
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22358) Add schedule shorthands for convinience
[
https://issues.apache.org/jira/browse/HIVE-22358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-22358:
Attachment: HIVE-22358.01.patch
> Add schedule shorthands for convinience
> ---
>
> Key: HIVE-22358
> URL: https://issues.apache.org/jira/browse/HIVE-22358
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22358.01.patch
>
>
> {code}
> DAILY AT 12.00
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22358) Add schedule shorthands for convinience
[
https://issues.apache.org/jira/browse/HIVE-22358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-22358:
Status: Patch Available (was: Open)
> Add schedule shorthands for convinience
> ---
>
> Key: HIVE-22358
> URL: https://issues.apache.org/jira/browse/HIVE-22358
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22358.01.patch
>
>
> {code}
> DAILY AT 12.00
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22788) Query cause NPE due to implicit cast on ROW__ID
[
https://issues.apache.org/jira/browse/HIVE-22788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17025880#comment-17025880
]
Hive QA commented on HIVE-22788:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
26s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
56s{color} | {color:blue} ql in master has 1538 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} ql: The patch generated 0 new + 22 unchanged - 1
fixed = 22 total (was 23) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
51s{color} | {color:green} root: The patch generated 0 new + 22 unchanged - 1
fixed = 22 total (was 23) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 6 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
43s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
13s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 61m 23s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20400/dev-support/hive-personality.sh
|
| git revision | master / 964f08a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20400/yetus/whitespace-eol.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20400/yetus/patch-asflicense-problems.txt
|
| modules | C: ql . U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20400/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Query cause NPE due to implicit cast on ROW__ID
> ---
>
> Key: HIVE-22788
> URL: https://issues.apache.org/jira/browse/HIVE-22788
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22788.1.patch, HIVE-22788.2.patch,
> HIVE-22788.3.patch
>
>
> *Repro*
> {code:sql}
> CREATE TABLE table_16 (
> timestamp_col_19timestamp,
> timestamp_col_29timestamp,
> int_col_27 int,
> int_col_39 int,
> boolean_col_18 boolean,
> varchar0045_col_23 varchar(45)
> );
> CREATE TABLE table_7 (
> int_col_10 int,
> bigint_col_3bigint
> );
> CREATE TABLE table_10 (
> boolean_col_8 boolean,
> boolean_col_16
[jira] [Issue Comment Deleted] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shivam updated HIVE-22771:
--
Comment: was deleted
(was: Thank you [~jcamachorodriguez] and [~mithun] for your comments, I have
added the test cases to write to static partition.
Can you please review and let me know if the changes look fine?
Test failure do not seem to be related.)
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Assignee: Shivam
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-22771.2.patch, HIVE-22771.3.patch, HIVE-22771.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate _idHash_ using below statement:
> *+In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+*
> *line 116: idHash = String.valueOf(Math.random());*
> The output of idHash can be similar to values like this : 7.145347157239135E-4
>
> And, in class _FileOutputCommitterContainer_ in package
> _org.apache.hive.hcatalog.mapreduce;_
> Uses below statement to compute final partition path:
> +*In org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java*+
> *line 366: String finalLocn = jobLocation.replaceAll(Path.SEPARATOR +
> SCRATCH_DIR_NAME + "{color:#ff}\\d\\.?
> d+"{color},"");*
> *line 367: partPath = new Path(finalLocn);*
>
> Regex used here is incorrect, since it will only remove integers after the
> *SCRATCH_DIR_NAME,* and hence will append 'E-4' (for the above example) in
> the final partition location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22788) Query cause NPE due to implicit cast on ROW__ID
[
https://issues.apache.org/jira/browse/HIVE-22788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22788:
--
Status: Open (was: Patch Available)
> Query cause NPE due to implicit cast on ROW__ID
> ---
>
> Key: HIVE-22788
> URL: https://issues.apache.org/jira/browse/HIVE-22788
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22788.1.patch, HIVE-22788.2.patch,
> HIVE-22788.3.patch
>
>
> *Repro*
> {code:sql}
> CREATE TABLE table_16 (
> timestamp_col_19timestamp,
> timestamp_col_29timestamp,
> int_col_27 int,
> int_col_39 int,
> boolean_col_18 boolean,
> varchar0045_col_23 varchar(45)
> );
> CREATE TABLE table_7 (
> int_col_10 int,
> bigint_col_3bigint
> );
> CREATE TABLE table_10 (
> boolean_col_8 boolean,
> boolean_col_16 boolean,
> timestamp_col_5 timestamp,
> timestamp_col_15timestamp,
> timestamp_col_30timestamp,
> decimal3825_col_26 decimal(38, 25),
> smallint_col_9 smallint,
> int_col_18 int
> );
> explain cbo
> SELECT
> DISTINCT COALESCE(a4.timestamp_col_15, IF(a4.boolean_col_16,
> a4.timestamp_col_30, a4.timestamp_col_5)) AS timestamp_col
> FROM table_7 a3
> RIGHT JOIN table_10 a4
> WHERE (a3.bigint_col_3) >= (a4.int_col_18)
> INTERSECT ALL
> SELECT
> COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> ) AS timestamp_col
> FROM table_16 a1
> GROUP BY COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(
> COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> );
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22788) Query cause NPE due to implicit cast on ROW__ID
[
https://issues.apache.org/jira/browse/HIVE-22788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22788:
--
Status: Patch Available (was: Open)
> Query cause NPE due to implicit cast on ROW__ID
> ---
>
> Key: HIVE-22788
> URL: https://issues.apache.org/jira/browse/HIVE-22788
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22788.1.patch, HIVE-22788.2.patch,
> HIVE-22788.3.patch
>
>
> *Repro*
> {code:sql}
> CREATE TABLE table_16 (
> timestamp_col_19timestamp,
> timestamp_col_29timestamp,
> int_col_27 int,
> int_col_39 int,
> boolean_col_18 boolean,
> varchar0045_col_23 varchar(45)
> );
> CREATE TABLE table_7 (
> int_col_10 int,
> bigint_col_3bigint
> );
> CREATE TABLE table_10 (
> boolean_col_8 boolean,
> boolean_col_16 boolean,
> timestamp_col_5 timestamp,
> timestamp_col_15timestamp,
> timestamp_col_30timestamp,
> decimal3825_col_26 decimal(38, 25),
> smallint_col_9 smallint,
> int_col_18 int
> );
> explain cbo
> SELECT
> DISTINCT COALESCE(a4.timestamp_col_15, IF(a4.boolean_col_16,
> a4.timestamp_col_30, a4.timestamp_col_5)) AS timestamp_col
> FROM table_7 a3
> RIGHT JOIN table_10 a4
> WHERE (a3.bigint_col_3) >= (a4.int_col_18)
> INTERSECT ALL
> SELECT
> COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> ) AS timestamp_col
> FROM table_16 a1
> GROUP BY COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(
> COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> );
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22788) Query cause NPE due to implicit cast on ROW__ID
[
https://issues.apache.org/jira/browse/HIVE-22788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22788:
--
Attachment: HIVE-22788.3.patch
> Query cause NPE due to implicit cast on ROW__ID
> ---
>
> Key: HIVE-22788
> URL: https://issues.apache.org/jira/browse/HIVE-22788
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22788.1.patch, HIVE-22788.2.patch,
> HIVE-22788.3.patch
>
>
> *Repro*
> {code:sql}
> CREATE TABLE table_16 (
> timestamp_col_19timestamp,
> timestamp_col_29timestamp,
> int_col_27 int,
> int_col_39 int,
> boolean_col_18 boolean,
> varchar0045_col_23 varchar(45)
> );
> CREATE TABLE table_7 (
> int_col_10 int,
> bigint_col_3bigint
> );
> CREATE TABLE table_10 (
> boolean_col_8 boolean,
> boolean_col_16 boolean,
> timestamp_col_5 timestamp,
> timestamp_col_15timestamp,
> timestamp_col_30timestamp,
> decimal3825_col_26 decimal(38, 25),
> smallint_col_9 smallint,
> int_col_18 int
> );
> explain cbo
> SELECT
> DISTINCT COALESCE(a4.timestamp_col_15, IF(a4.boolean_col_16,
> a4.timestamp_col_30, a4.timestamp_col_5)) AS timestamp_col
> FROM table_7 a3
> RIGHT JOIN table_10 a4
> WHERE (a3.bigint_col_3) >= (a4.int_col_18)
> INTERSECT ALL
> SELECT
> COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> ) AS timestamp_col
> FROM table_16 a1
> GROUP BY COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(
> COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> );
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shivam updated HIVE-22771:
--
Attachment: (was: HIVE-22771.3.patch)
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Assignee: Shivam
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-22771.2.patch, HIVE-22771.3.patch, HIVE-22771.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate _idHash_ using below statement:
> *+In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+*
> *line 116: idHash = String.valueOf(Math.random());*
> The output of idHash can be similar to values like this : 7.145347157239135E-4
>
> And, in class _FileOutputCommitterContainer_ in package
> _org.apache.hive.hcatalog.mapreduce;_
> Uses below statement to compute final partition path:
> +*In org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java*+
> *line 366: String finalLocn = jobLocation.replaceAll(Path.SEPARATOR +
> SCRATCH_DIR_NAME + "{color:#ff}\\d\\.?
> d+"{color},"");*
> *line 367: partPath = new Path(finalLocn);*
>
> Regex used here is incorrect, since it will only remove integers after the
> *SCRATCH_DIR_NAME,* and hence will append 'E-4' (for the above example) in
> the final partition location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shivam updated HIVE-22771:
--
Attachment: (was: HIVE-22771.2.patch)
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Assignee: Shivam
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-22771.2.patch, HIVE-22771.3.patch,
> HIVE-22771.3.patch, HIVE-22771.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate _idHash_ using below statement:
> *+In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+*
> *line 116: idHash = String.valueOf(Math.random());*
> The output of idHash can be similar to values like this : 7.145347157239135E-4
>
> And, in class _FileOutputCommitterContainer_ in package
> _org.apache.hive.hcatalog.mapreduce;_
> Uses below statement to compute final partition path:
> +*In org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java*+
> *line 366: String finalLocn = jobLocation.replaceAll(Path.SEPARATOR +
> SCRATCH_DIR_NAME + "{color:#ff}\\d\\.?
> d+"{color},"");*
> *line 367: partPath = new Path(finalLocn);*
>
> Regex used here is incorrect, since it will only remove integers after the
> *SCRATCH_DIR_NAME,* and hence will append 'E-4' (for the above example) in
> the final partition location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22788) Query cause NPE due to implicit cast on ROW__ID
[
https://issues.apache.org/jira/browse/HIVE-22788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22788:
--
Attachment: (was: HIVE-22788.3.patch)
> Query cause NPE due to implicit cast on ROW__ID
> ---
>
> Key: HIVE-22788
> URL: https://issues.apache.org/jira/browse/HIVE-22788
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22788.1.patch, HIVE-22788.2.patch
>
>
> *Repro*
> {code:sql}
> CREATE TABLE table_16 (
> timestamp_col_19timestamp,
> timestamp_col_29timestamp,
> int_col_27 int,
> int_col_39 int,
> boolean_col_18 boolean,
> varchar0045_col_23 varchar(45)
> );
> CREATE TABLE table_7 (
> int_col_10 int,
> bigint_col_3bigint
> );
> CREATE TABLE table_10 (
> boolean_col_8 boolean,
> boolean_col_16 boolean,
> timestamp_col_5 timestamp,
> timestamp_col_15timestamp,
> timestamp_col_30timestamp,
> decimal3825_col_26 decimal(38, 25),
> smallint_col_9 smallint,
> int_col_18 int
> );
> explain cbo
> SELECT
> DISTINCT COALESCE(a4.timestamp_col_15, IF(a4.boolean_col_16,
> a4.timestamp_col_30, a4.timestamp_col_5)) AS timestamp_col
> FROM table_7 a3
> RIGHT JOIN table_10 a4
> WHERE (a3.bigint_col_3) >= (a4.int_col_18)
> INTERSECT ALL
> SELECT
> COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> ) AS timestamp_col
> FROM table_16 a1
> GROUP BY COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(
> COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> );
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shivam updated HIVE-22771:
--
Attachment: HIVE-22771.3.patch
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Assignee: Shivam
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-22771.2.patch, HIVE-22771.2.patch,
> HIVE-22771.3.patch, HIVE-22771.3.patch, HIVE-22771.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate _idHash_ using below statement:
> *+In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+*
> *line 116: idHash = String.valueOf(Math.random());*
> The output of idHash can be similar to values like this : 7.145347157239135E-4
>
> And, in class _FileOutputCommitterContainer_ in package
> _org.apache.hive.hcatalog.mapreduce;_
> Uses below statement to compute final partition path:
> +*In org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java*+
> *line 366: String finalLocn = jobLocation.replaceAll(Path.SEPARATOR +
> SCRATCH_DIR_NAME + "{color:#ff}\\d\\.?
> d+"{color},"");*
> *line 367: partPath = new Path(finalLocn);*
>
> Regex used here is incorrect, since it will only remove integers after the
> *SCRATCH_DIR_NAME,* and hence will append 'E-4' (for the above example) in
> the final partition location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17025837#comment-17025837
]
Shivam commented on HIVE-22771:
---
Thank you [~jcamachorodriguez] and [~mithun] for your comments, I have added
the test cases to write to static partition.
Can you please review and let me know if the changes look fine?
Test failure do not seem to be related.
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Assignee: Shivam
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-22771.2.patch, HIVE-22771.2.patch,
> HIVE-22771.3.patch, HIVE-22771.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate _idHash_ using below statement:
> *+In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+*
> *line 116: idHash = String.valueOf(Math.random());*
> The output of idHash can be similar to values like this : 7.145347157239135E-4
>
> And, in class _FileOutputCommitterContainer_ in package
> _org.apache.hive.hcatalog.mapreduce;_
> Uses below statement to compute final partition path:
> +*In org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java*+
> *line 366: String finalLocn = jobLocation.replaceAll(Path.SEPARATOR +
> SCRATCH_DIR_NAME + "{color:#ff}\\d\\.?
> d+"{color},"");*
> *line 367: partPath = new Path(finalLocn);*
>
> Regex used here is incorrect, since it will only remove integers after the
> *SCRATCH_DIR_NAME,* and hence will append 'E-4' (for the above example) in
> the final partition location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22788) Query cause NPE due to implicit cast on ROW__ID
[
https://issues.apache.org/jira/browse/HIVE-22788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22788:
--
Attachment: HIVE-22788.3.patch
> Query cause NPE due to implicit cast on ROW__ID
> ---
>
> Key: HIVE-22788
> URL: https://issues.apache.org/jira/browse/HIVE-22788
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22788.1.patch, HIVE-22788.2.patch,
> HIVE-22788.3.patch
>
>
> *Repro*
> {code:sql}
> CREATE TABLE table_16 (
> timestamp_col_19timestamp,
> timestamp_col_29timestamp,
> int_col_27 int,
> int_col_39 int,
> boolean_col_18 boolean,
> varchar0045_col_23 varchar(45)
> );
> CREATE TABLE table_7 (
> int_col_10 int,
> bigint_col_3bigint
> );
> CREATE TABLE table_10 (
> boolean_col_8 boolean,
> boolean_col_16 boolean,
> timestamp_col_5 timestamp,
> timestamp_col_15timestamp,
> timestamp_col_30timestamp,
> decimal3825_col_26 decimal(38, 25),
> smallint_col_9 smallint,
> int_col_18 int
> );
> explain cbo
> SELECT
> DISTINCT COALESCE(a4.timestamp_col_15, IF(a4.boolean_col_16,
> a4.timestamp_col_30, a4.timestamp_col_5)) AS timestamp_col
> FROM table_7 a3
> RIGHT JOIN table_10 a4
> WHERE (a3.bigint_col_3) >= (a4.int_col_18)
> INTERSECT ALL
> SELECT
> COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> ) AS timestamp_col
> FROM table_16 a1
> GROUP BY COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(
> COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> );
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22736) Support replication across multiple encryption zones
[ https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22736: --- Attachment: HIVE-22736.patch Status: Patch Available (was: In Progress) > Support replication across multiple encryption zones > > > Key: HIVE-22736 > URL: https://issues.apache.org/jira/browse/HIVE-22736 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch > > Time Spent: 2h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17025836#comment-17025836
]
Hive QA commented on HIVE-22771:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992092/HIVE-22771.3.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 17951 tests
executed
*Failed tests:*
{noformat}
TestMiniSparkOnYarnCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=200)
[scriptfile1.q,vector_outer_join5.q,file_with_header_footer.q,input16_cc.q,bucket5.q,orc_merge2.q,reduce_deduplicate.q,schemeAuthority2.q,spark_dynamic_partition_pruning_5.q,orc_merge8.q,orc_merge_incompat2.q,infer_bucket_sort_bucketed_table.q,vector_outer_join4.q,disable_merge_for_bucketing.q,orc_merge7.q]
org.apache.hadoop.hive.metastore.TestHiveMetaStoreGetMetaConf.org.apache.hadoop.hive.metastore.TestHiveMetaStoreGetMetaConf
(batchId=238)
org.apache.hadoop.hive.ql.parse.TestTableLevelReplicationScenarios.testBasicBootstrapWithIncludeAndExcludeList
(batchId=261)
org.apache.hadoop.hive.ql.parse.TestTableLevelReplicationScenarios.testBasicReplaceReplPolicy
(batchId=261)
org.apache.hadoop.hive.ql.parse.TestTableLevelReplicationScenarios.testIncorrectTablePolicyInReplDump
(batchId=261)
org.apache.hadoop.hive.ql.parse.TestTableLevelReplicationScenarios.testRenameTableScenariosWithReplacePolicyDMLOperattion
(batchId=261)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20399/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20399/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20399/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12992092 - PreCommit-HIVE-Build
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Assignee: Shivam
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-22771.2.patch, HIVE-22771.2.patch,
> HIVE-22771.3.patch, HIVE-22771.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate _idHash_ using below statement:
> *+In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+*
> *line 116: idHash = String.valueOf(Math.random());*
> The output of idHash can be similar to values like this : 7.145347157239135E-4
>
> And, in class _FileOutputCommitterContainer_ in package
> _org.apache.hive.hcatalog.mapreduce;_
> Uses below statement to compute final partition path:
> +*In org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java*+
> *line 366: String finalLocn = jobLocation.replaceAll(Path.SEPARATOR +
> SCRATCH_DIR_NAME + "{color:#ff}\\d\\.?
> d+"{color},"");*
> *line 367: partPath = new Path(finalLocn);*
>
> Regex used here is incorrect, since it will only remove integers after the
> *SCRATCH_DIR_NAME,* and hence will append 'E-4' (for the above example) in
> the final partition location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22736) Support replication across multiple encryption zones
[ https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22736: --- Status: In Progress (was: Patch Available) > Support replication across multiple encryption zones > > > Key: HIVE-22736 > URL: https://issues.apache.org/jira/browse/HIVE-22736 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch > > Time Spent: 2h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-10362) Support Type check/conversion in dynamic partition column
[ https://issues.apache.org/jira/browse/HIVE-10362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-10362: - Status: Open (was: Patch Available) > Support Type check/conversion in dynamic partition column > - > > Key: HIVE-10362 > URL: https://issues.apache.org/jira/browse/HIVE-10362 > Project: Hive > Issue Type: Improvement > Components: Query Processor, Types >Affects Versions: 3.0.0, 1.0.0, 4.0.0 >Reporter: Chaoyu Tang >Assignee: Karen Coppage >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-10362.01.patch, HIVE-10362.02.patch, > HIVE-10362.03.patch, HIVE-10362.04.patch, HIVE-10362.04.patch, > HIVE-10362.06.patch, HIVE-10362.07.patch, HIVE-10362.07.patch, > HIVE-10362.08.patch > > > There are quite a lot of issues associated with the non-noramlized or > type-mismatched values for partition column. Hive has many ways to introduce > such problematic data. HIVE-10307 mainly provides the support to type > check/convert/normalize the partition column value in static partition > specification. This JIRA tries to deal with the partition column type in > dynamic partition insert. Currently any data can be inserted as a partition > column value as long as it is quoted as a string. For example, > create table dynparttypechecknum (key int, value string) partitioned by (part > int); > insert into dynparttypechecknum partition (part) select key, value, '1' > from src limit 1; > show partitions dynparttypechecknum; > -- > part=1 > The partition column value is non-normalized int 1. It causes some > unnecessary problems such as integer partition column JDO filter pushdown > (see HIVE-6052) and others like HIVE-10210. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-10362) Support Type check/conversion in dynamic partition column
[ https://issues.apache.org/jira/browse/HIVE-10362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-10362: - Attachment: HIVE-10362.08.patch Status: Patch Available (was: Open) > Support Type check/conversion in dynamic partition column > - > > Key: HIVE-10362 > URL: https://issues.apache.org/jira/browse/HIVE-10362 > Project: Hive > Issue Type: Improvement > Components: Query Processor, Types >Affects Versions: 3.0.0, 1.0.0, 4.0.0 >Reporter: Chaoyu Tang >Assignee: Karen Coppage >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-10362.01.patch, HIVE-10362.02.patch, > HIVE-10362.03.patch, HIVE-10362.04.patch, HIVE-10362.04.patch, > HIVE-10362.06.patch, HIVE-10362.07.patch, HIVE-10362.07.patch, > HIVE-10362.08.patch > > > There are quite a lot of issues associated with the non-noramlized or > type-mismatched values for partition column. Hive has many ways to introduce > such problematic data. HIVE-10307 mainly provides the support to type > check/convert/normalize the partition column value in static partition > specification. This JIRA tries to deal with the partition column type in > dynamic partition insert. Currently any data can be inserted as a partition > column value as long as it is quoted as a string. For example, > create table dynparttypechecknum (key int, value string) partitioned by (part > int); > insert into dynparttypechecknum partition (part) select key, value, '1' > from src limit 1; > show partitions dynparttypechecknum; > -- > part=1 > The partition column value is non-normalized int 1. It causes some > unnecessary problems such as integer partition column JDO filter pushdown > (see HIVE-6052) and others like HIVE-10210. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17025818#comment-17025818
]
Hive QA commented on HIVE-22771:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
50s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
38s{color} | {color:blue} hcatalog/core in master has 36 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
27s{color} | {color:blue} hcatalog/hcatalog-pig-adapter in master has 2 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} hcatalog/core: The patch generated 0 new + 39
unchanged - 1 fixed = 39 total (was 40) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} The patch hcatalog-pig-adapter passed checkstyle
{color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20399/dev-support/hive-personality.sh
|
| git revision | master / 964f08a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: hcatalog/core hcatalog/hcatalog-pig-adapter U: hcatalog |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20399/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Assignee: Shivam
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-22771.2.patch, HIVE-22771.2.patch,
> HIVE-22771.3.patch, HIVE-22771.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate _idHash_ using below statement:
> *+In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+*
> *line 116: idHash = String.valueOf(Math.random());*
[jira] [Updated] (HIVE-22788) Query cause NPE due to implicit cast on ROW__ID
[
https://issues.apache.org/jira/browse/HIVE-22788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22788:
--
Status: Patch Available (was: Open)
> Query cause NPE due to implicit cast on ROW__ID
> ---
>
> Key: HIVE-22788
> URL: https://issues.apache.org/jira/browse/HIVE-22788
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22788.1.patch, HIVE-22788.2.patch
>
>
> *Repro*
> {code:sql}
> CREATE TABLE table_16 (
> timestamp_col_19timestamp,
> timestamp_col_29timestamp,
> int_col_27 int,
> int_col_39 int,
> boolean_col_18 boolean,
> varchar0045_col_23 varchar(45)
> );
> CREATE TABLE table_7 (
> int_col_10 int,
> bigint_col_3bigint
> );
> CREATE TABLE table_10 (
> boolean_col_8 boolean,
> boolean_col_16 boolean,
> timestamp_col_5 timestamp,
> timestamp_col_15timestamp,
> timestamp_col_30timestamp,
> decimal3825_col_26 decimal(38, 25),
> smallint_col_9 smallint,
> int_col_18 int
> );
> explain cbo
> SELECT
> DISTINCT COALESCE(a4.timestamp_col_15, IF(a4.boolean_col_16,
> a4.timestamp_col_30, a4.timestamp_col_5)) AS timestamp_col
> FROM table_7 a3
> RIGHT JOIN table_10 a4
> WHERE (a3.bigint_col_3) >= (a4.int_col_18)
> INTERSECT ALL
> SELECT
> COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> ) AS timestamp_col
> FROM table_16 a1
> GROUP BY COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(
> COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> );
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22788) Query cause NPE due to implicit cast on ROW__ID
[
https://issues.apache.org/jira/browse/HIVE-22788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22788:
--
Attachment: HIVE-22788.2.patch
> Query cause NPE due to implicit cast on ROW__ID
> ---
>
> Key: HIVE-22788
> URL: https://issues.apache.org/jira/browse/HIVE-22788
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22788.1.patch, HIVE-22788.2.patch
>
>
> *Repro*
> {code:sql}
> CREATE TABLE table_16 (
> timestamp_col_19timestamp,
> timestamp_col_29timestamp,
> int_col_27 int,
> int_col_39 int,
> boolean_col_18 boolean,
> varchar0045_col_23 varchar(45)
> );
> CREATE TABLE table_7 (
> int_col_10 int,
> bigint_col_3bigint
> );
> CREATE TABLE table_10 (
> boolean_col_8 boolean,
> boolean_col_16 boolean,
> timestamp_col_5 timestamp,
> timestamp_col_15timestamp,
> timestamp_col_30timestamp,
> decimal3825_col_26 decimal(38, 25),
> smallint_col_9 smallint,
> int_col_18 int
> );
> explain cbo
> SELECT
> DISTINCT COALESCE(a4.timestamp_col_15, IF(a4.boolean_col_16,
> a4.timestamp_col_30, a4.timestamp_col_5)) AS timestamp_col
> FROM table_7 a3
> RIGHT JOIN table_10 a4
> WHERE (a3.bigint_col_3) >= (a4.int_col_18)
> INTERSECT ALL
> SELECT
> COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> ) AS timestamp_col
> FROM table_16 a1
> GROUP BY COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(
> COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> );
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-22788) Query cause NPE due to implicit cast on ROW__ID
[
https://issues.apache.org/jira/browse/HIVE-22788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa reassigned HIVE-22788:
-
Assignee: Krisztian Kasa (was: Vineet Garg)
> Query cause NPE due to implicit cast on ROW__ID
> ---
>
> Key: HIVE-22788
> URL: https://issues.apache.org/jira/browse/HIVE-22788
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22788.1.patch, HIVE-22788.2.patch
>
>
> *Repro*
> {code:sql}
> CREATE TABLE table_16 (
> timestamp_col_19timestamp,
> timestamp_col_29timestamp,
> int_col_27 int,
> int_col_39 int,
> boolean_col_18 boolean,
> varchar0045_col_23 varchar(45)
> );
> CREATE TABLE table_7 (
> int_col_10 int,
> bigint_col_3bigint
> );
> CREATE TABLE table_10 (
> boolean_col_8 boolean,
> boolean_col_16 boolean,
> timestamp_col_5 timestamp,
> timestamp_col_15timestamp,
> timestamp_col_30timestamp,
> decimal3825_col_26 decimal(38, 25),
> smallint_col_9 smallint,
> int_col_18 int
> );
> explain cbo
> SELECT
> DISTINCT COALESCE(a4.timestamp_col_15, IF(a4.boolean_col_16,
> a4.timestamp_col_30, a4.timestamp_col_5)) AS timestamp_col
> FROM table_7 a3
> RIGHT JOIN table_10 a4
> WHERE (a3.bigint_col_3) >= (a4.int_col_18)
> INTERSECT ALL
> SELECT
> COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> ) AS timestamp_col
> FROM table_16 a1
> GROUP BY COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(
> COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> );
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22788) Query cause NPE due to implicit cast on ROW__ID
[
https://issues.apache.org/jira/browse/HIVE-22788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22788:
--
Status: Open (was: Patch Available)
> Query cause NPE due to implicit cast on ROW__ID
> ---
>
> Key: HIVE-22788
> URL: https://issues.apache.org/jira/browse/HIVE-22788
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22788.1.patch
>
>
> *Repro*
> {code:sql}
> CREATE TABLE table_16 (
> timestamp_col_19timestamp,
> timestamp_col_29timestamp,
> int_col_27 int,
> int_col_39 int,
> boolean_col_18 boolean,
> varchar0045_col_23 varchar(45)
> );
> CREATE TABLE table_7 (
> int_col_10 int,
> bigint_col_3bigint
> );
> CREATE TABLE table_10 (
> boolean_col_8 boolean,
> boolean_col_16 boolean,
> timestamp_col_5 timestamp,
> timestamp_col_15timestamp,
> timestamp_col_30timestamp,
> decimal3825_col_26 decimal(38, 25),
> smallint_col_9 smallint,
> int_col_18 int
> );
> explain cbo
> SELECT
> DISTINCT COALESCE(a4.timestamp_col_15, IF(a4.boolean_col_16,
> a4.timestamp_col_30, a4.timestamp_col_5)) AS timestamp_col
> FROM table_7 a3
> RIGHT JOIN table_10 a4
> WHERE (a3.bigint_col_3) >= (a4.int_col_18)
> INTERSECT ALL
> SELECT
> COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> ) AS timestamp_col
> FROM table_16 a1
> GROUP BY COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(
> COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> );
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22736) Support replication across multiple encryption zones
[
https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17025809#comment-17025809
]
Hive QA commented on HIVE-22736:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992090/HIVE-22736.patch
{color:green}SUCCESS:{color} +1 due to 5 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17968 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20398/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20398/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20398/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12992090 - PreCommit-HIVE-Build
> Support replication across multiple encryption zones
>
>
> Key: HIVE-22736
> URL: https://issues.apache.org/jira/browse/HIVE-22736
> Project: Hive
> Issue Type: Task
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22736) Support replication across multiple encryption zones
[
https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17025803#comment-17025803
]
Hive QA commented on HIVE-22736:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
41s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
14s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
19s{color} | {color:blue} shims/common in master has 6 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
22s{color} | {color:blue} shims/0.23 in master has 7 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
37s{color} | {color:blue} standalone-metastore/metastore-common in master has
35 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
35s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
12s{color} | {color:blue} standalone-metastore/metastore-server in master has
181 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
10s{color} | {color:blue} ql in master has 1538 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
42s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
17s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} shims/common: The patch generated 0 new + 93
unchanged - 1 fixed = 93 total (was 94) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} The patch 0.23 passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} The patch metastore-common passed checkstyle {color}
|
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} The patch common passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} The patch metastore-server passed checkstyle {color}
|
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} The patch ql passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} The patch hive-unit passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 11m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{c
[jira] [Commented] (HIVE-22729) Provide a failure reason for failed compactions
[
https://issues.apache.org/jira/browse/HIVE-22729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17025744#comment-17025744
]
Denys Kuzmenko commented on HIVE-22729:
---
+1
> Provide a failure reason for failed compactions
> ---
>
> Key: HIVE-22729
> URL: https://issues.apache.org/jira/browse/HIVE-22729
> Project: Hive
> Issue Type: Improvement
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-22729.01.patch, HIVE-22729.02.patch,
> HIVE-22729.03.patch, HIVE-22729.04.patch, HIVE-22729.05.patch,
> HIVE-22729.06.patch, HIVE-22729.07.patch, HIVE-22729.08.patch,
> HIVE-22729.09.patch
>
>
> We should provide a compaction failure reason as easily accessible as
> possible. Like in the result of the {{SHOW COMPACTIONS}} command.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17025742#comment-17025742
]
Hive QA commented on HIVE-22771:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992085/HIVE-22771.2.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17969 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20397/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20397/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20397/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12992085 - PreCommit-HIVE-Build
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Assignee: Shivam
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-22771.2.patch, HIVE-22771.2.patch,
> HIVE-22771.3.patch, HIVE-22771.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate _idHash_ using below statement:
> *+In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+*
> *line 116: idHash = String.valueOf(Math.random());*
> The output of idHash can be similar to values like this : 7.145347157239135E-4
>
> And, in class _FileOutputCommitterContainer_ in package
> _org.apache.hive.hcatalog.mapreduce;_
> Uses below statement to compute final partition path:
> +*In org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java*+
> *line 366: String finalLocn = jobLocation.replaceAll(Path.SEPARATOR +
> SCRATCH_DIR_NAME + "{color:#ff}\\d\\.?
> d+"{color},"");*
> *line 367: partPath = new Path(finalLocn);*
>
> Regex used here is incorrect, since it will only remove integers after the
> *SCRATCH_DIR_NAME,* and hence will append 'E-4' (for the above example) in
> the final partition location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22729) Provide a failure reason for failed compactions
[
https://issues.apache.org/jira/browse/HIVE-22729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17025736#comment-17025736
]
Laszlo Pinter commented on HIVE-22729:
--
[~pvary] [~dkuzmenko] [~klcopp] Can I get a +1 from you? Thank you
> Provide a failure reason for failed compactions
> ---
>
> Key: HIVE-22729
> URL: https://issues.apache.org/jira/browse/HIVE-22729
> Project: Hive
> Issue Type: Improvement
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-22729.01.patch, HIVE-22729.02.patch,
> HIVE-22729.03.patch, HIVE-22729.04.patch, HIVE-22729.05.patch,
> HIVE-22729.06.patch, HIVE-22729.07.patch, HIVE-22729.08.patch,
> HIVE-22729.09.patch
>
>
> We should provide a compaction failure reason as easily accessible as
> possible. Like in the result of the {{SHOW COMPACTIONS}} command.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shivam updated HIVE-22771:
--
Attachment: HIVE-22771.3.patch
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Assignee: Shivam
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-22771.2.patch, HIVE-22771.2.patch,
> HIVE-22771.3.patch, HIVE-22771.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate _idHash_ using below statement:
> *+In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+*
> *line 116: idHash = String.valueOf(Math.random());*
> The output of idHash can be similar to values like this : 7.145347157239135E-4
>
> And, in class _FileOutputCommitterContainer_ in package
> _org.apache.hive.hcatalog.mapreduce;_
> Uses below statement to compute final partition path:
> +*In org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java*+
> *line 366: String finalLocn = jobLocation.replaceAll(Path.SEPARATOR +
> SCRATCH_DIR_NAME + "{color:#ff}\\d\\.?
> d+"{color},"");*
> *line 367: partPath = new Path(finalLocn);*
>
> Regex used here is incorrect, since it will only remove integers after the
> *SCRATCH_DIR_NAME,* and hence will append 'E-4' (for the above example) in
> the final partition location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shivam updated HIVE-22771:
--
Status: In Progress (was: Patch Available)
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Assignee: Shivam
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-22771.2.patch, HIVE-22771.2.patch,
> HIVE-22771.3.patch, HIVE-22771.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate _idHash_ using below statement:
> *+In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+*
> *line 116: idHash = String.valueOf(Math.random());*
> The output of idHash can be similar to values like this : 7.145347157239135E-4
>
> And, in class _FileOutputCommitterContainer_ in package
> _org.apache.hive.hcatalog.mapreduce;_
> Uses below statement to compute final partition path:
> +*In org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java*+
> *line 366: String finalLocn = jobLocation.replaceAll(Path.SEPARATOR +
> SCRATCH_DIR_NAME + "{color:#ff}\\d\\.?
> d+"{color},"");*
> *line 367: partPath = new Path(finalLocn);*
>
> Regex used here is incorrect, since it will only remove integers after the
> *SCRATCH_DIR_NAME,* and hence will append 'E-4' (for the above example) in
> the final partition location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shivam updated HIVE-22771:
--
Status: Patch Available (was: In Progress)
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Assignee: Shivam
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-22771.2.patch, HIVE-22771.2.patch,
> HIVE-22771.3.patch, HIVE-22771.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate _idHash_ using below statement:
> *+In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+*
> *line 116: idHash = String.valueOf(Math.random());*
> The output of idHash can be similar to values like this : 7.145347157239135E-4
>
> And, in class _FileOutputCommitterContainer_ in package
> _org.apache.hive.hcatalog.mapreduce;_
> Uses below statement to compute final partition path:
> +*In org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java*+
> *line 366: String finalLocn = jobLocation.replaceAll(Path.SEPARATOR +
> SCRATCH_DIR_NAME + "{color:#ff}\\d\\.?
> d+"{color},"");*
> *line 367: partPath = new Path(finalLocn);*
>
> Regex used here is incorrect, since it will only remove integers after the
> *SCRATCH_DIR_NAME,* and hence will append 'E-4' (for the above example) in
> the final partition location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22515) Support cast to decimal64 in Vectorization
[ https://issues.apache.org/jira/browse/HIVE-22515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17025727#comment-17025727 ] Jason Dere commented on HIVE-22515: --- +1 > Support cast to decimal64 in Vectorization > -- > > Key: HIVE-22515 > URL: https://issues.apache.org/jira/browse/HIVE-22515 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22515.11.patch, HIVE-22515.5.patch, > HIVE-22515.8.patch, HIVE-22515.9.patch > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Support cast to decimal64 in Vectorization -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22736) Support replication across multiple encryption zones
[ https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22736: --- Attachment: (was: HIVE-22736-4.patch) > Support replication across multiple encryption zones > > > Key: HIVE-22736 > URL: https://issues.apache.org/jira/browse/HIVE-22736 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch > > Time Spent: 2h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22736) Support replication across multiple encryption zones
[ https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22736: --- Attachment: HIVE-22736.patch Status: Patch Available (was: In Progress) > Support replication across multiple encryption zones > > > Key: HIVE-22736 > URL: https://issues.apache.org/jira/browse/HIVE-22736 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22736-4.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch > > Time Spent: 2h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22736) Support replication across multiple encryption zones
[ https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22736: --- Status: In Progress (was: Patch Available) > Support replication across multiple encryption zones > > > Key: HIVE-22736 > URL: https://issues.apache.org/jira/browse/HIVE-22736 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22736-4.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch > > Time Spent: 2h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22736) Support replication across multiple encryption zones
[ https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22736: --- Attachment: (was: HIVE-22736-1.patch) > Support replication across multiple encryption zones > > > Key: HIVE-22736 > URL: https://issues.apache.org/jira/browse/HIVE-22736 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22736-4.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch > > Time Spent: 2h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22736) Support replication across multiple encryption zones
[ https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22736: --- Attachment: (was: HIVE-22736-3.patch) > Support replication across multiple encryption zones > > > Key: HIVE-22736 > URL: https://issues.apache.org/jira/browse/HIVE-22736 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22736-4.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch > > Time Spent: 2h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22736) Support replication across multiple encryption zones
[ https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22736: --- Attachment: (was: HIVE-22736-2.patch) > Support replication across multiple encryption zones > > > Key: HIVE-22736 > URL: https://issues.apache.org/jira/browse/HIVE-22736 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22736-4.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch > > Time Spent: 2h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17025686#comment-17025686
]
Hive QA commented on HIVE-22771:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
53s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} hcatalog/core in master has 36 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
27s{color} | {color:blue} hcatalog/hcatalog-pig-adapter in master has 2 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} hcatalog/core: The patch generated 0 new + 39
unchanged - 1 fixed = 39 total (was 40) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} The patch hcatalog-pig-adapter passed checkstyle
{color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 58s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20397/dev-support/hive-personality.sh
|
| git revision | master / 964f08a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20397/yetus/whitespace-eol.txt
|
| modules | C: hcatalog/core hcatalog/hcatalog-pig-adapter U: hcatalog |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20397/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Assignee: Shivam
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-22771.2.patch, HIVE-22771.2.patch, HIVE-22771.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate
[jira] [Commented] (HIVE-22786) Agg with distinct can be optimised in HASH mode
[
https://issues.apache.org/jira/browse/HIVE-22786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17025674#comment-17025674
]
Hive QA commented on HIVE-22786:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992082/HIVE-22786.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17955 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[explainuser_2]
(batchId=162)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20395/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20395/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20395/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12992082 - PreCommit-HIVE-Build
> Agg with distinct can be optimised in HASH mode
> ---
>
> Key: HIVE-22786
> URL: https://issues.apache.org/jira/browse/HIVE-22786
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-22786.1.patch, HIVE-22786.2.patch,
> HIVE-22786.3.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
