[jira] [Commented] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925089#comment-16925089
]
Peter Vary commented on HIVE-21508:
---
[~ananamj]: Add one patch every time (wait for the results), since the test
infra runs only on the last file. Also I would start with the master branch,
since other branches are often not clean.
Thanks,
Peter
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 3.2.0, 2.3.7
>
> Attachments: HIVE-21508.branch-2.3.patch, HIVE-21508.branch-2.patch,
> HIVE-21508.branch-3.1.patch, HIVE-21508.branch-3.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-20683) Add the Ability to push Dynamic Between and Bloom filters to Druid
[
https://issues.apache.org/jira/browse/HIVE-20683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925046#comment-16925046
]
Ashutosh Chauhan commented on HIVE-20683:
-
[~bslim] can you please review this?
> Add the Ability to push Dynamic Between and Bloom filters to Druid
> --
>
> Key: HIVE-20683
> URL: https://issues.apache.org/jira/browse/HIVE-20683
> Project: Hive
> Issue Type: New Feature
> Components: Druid integration
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20683.1.patch, HIVE-20683.2.patch,
> HIVE-20683.3.patch, HIVE-20683.4.patch, HIVE-20683.5.patch,
> HIVE-20683.6.patch, HIVE-20683.8.patch, HIVE-20683.patch
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> For optimizing joins, Hive generates BETWEEN filter with min-max and BLOOM
> filter for filtering one side of semi-join.
> Druid 0.13.0 will have support for Bloom filters (Added via
> https://github.com/apache/incubator-druid/pull/6222)
> Implementation details -
> # Hive generates and passes the filters as part of 'filterExpr' in TableScan.
> # DruidQueryBasedRecordReader gets this filter passed as part of the conf.
> # During execution phase, before sending the query to druid in
> DruidQueryBasedRecordReader we will deserialize this filter, translate it
> into a DruidDimFilter and add it to existing DruidQuery. Tez executor
> already ensures that when we start reading results from the record reader,
> all the dynamic values are initialized.
> # Explaining a druid query also prints the query sent to druid as
> {{druid.json.query}}. We also need to make sure to update the druid query
> with the filters. During explain we do not have the actual values for the
> dynamic values, so instead of values we will print the dynamic expression
> itself as part of druid query.
> Note:- This work needs druid to be updated to version 0.13.0
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22179) Break up DDLSemanticAnalyzer - extract Function related analyzers
[
https://issues.apache.org/jira/browse/HIVE-22179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925011#comment-16925011
]
Hive QA commented on HIVE-22179:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979767/HIVE-22179.01.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16748 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18500/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18500/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18500/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979767 - PreCommit-HIVE-Build
> Break up DDLSemanticAnalyzer - extract Function related analyzers
> -
>
> Key: HIVE-22179
> URL: https://issues.apache.org/jira/browse/HIVE-22179
> Project: Hive
> Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available, refactor-ddl
> Attachments: HIVE-22179.01.patch
>
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is
> to refactor it in order to have everything cut into more handleable classes
> under the package org.apache.hadoop.hive.ql.exec.ddl:
> * have a separate class for each analyzers
> * have a package for each operation, containing an analyzer, a description,
> and an operation, so the amount of classes under a package is more manageable
> Step #2: extract all the functin related analyzers from DDLSemanticAnalyzer
> and FunctionSemanticAnalyzer and MacroSemanticAnalyzer, and move them under
> the new package.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22179) Break up DDLSemanticAnalyzer - extract Function related analyzers
[
https://issues.apache.org/jira/browse/HIVE-22179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925003#comment-16925003
]
Hive QA commented on HIVE-22179:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
36s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} ql: The patch generated 0 new + 270 unchanged - 11
fixed = 270 total (was 281) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 11s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18500/dev-support/hive-personality.sh
|
| git revision | master / 5565311 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18500/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Break up DDLSemanticAnalyzer - extract Function related analyzers
> -
>
> Key: HIVE-22179
> URL: https://issues.apache.org/jira/browse/HIVE-22179
> Project: Hive
> Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available, refactor-ddl
> Attachments: HIVE-22179.01.patch
>
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is
> to refactor it in order to have everything cut into more handleable classes
> under the package org.apache.hadoop.hive.ql.exec.ddl:
> * have a separate class for each analyzers
> * have a package for each operation, containing an analyzer, a description,
> and an operation, so the amount of classes under a package is more manageable
> Step #2: extract all the functin related analyzers from DDLSemanticAnalyzer
> and FunctionSemanticAnalyzer and MacroSemanticAnalyzer, and move them under
> the new package.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22170) from_unixtime and unix_timestamp should use user session time zone
[
https://issues.apache.org/jira/browse/HIVE-22170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924992#comment-16924992
]
Hive QA commented on HIVE-22170:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979766/HIVE-22170.03.patch
{color:green}SUCCESS:{color} +1 due to 23 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 16748 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropParitionsCleanup
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropPartitionsCacheCrossSession
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testPartitionOps (batchId=233)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18499/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18499/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18499/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979766 - PreCommit-HIVE-Build
> from_unixtime and unix_timestamp should use user session time zone
> --
>
> Key: HIVE-22170
> URL: https://issues.apache.org/jira/browse/HIVE-22170
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0, 4.0.0, 3.2.0, 3.1.1, 3.1.2
>Reporter: Riju Trivedi
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-22170.01.patch, HIVE-22170.02.patch,
> HIVE-22170.03.patch
>
>
> According to documentation, that is the expected behavior (since session time
> zone was not present, system time zone was being used previously). This was
> incorrectly changed by HIVE-12192 / HIVE-20007. This JIRA should fix this
> issue.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22170) from_unixtime and unix_timestamp should use user session time zone
[
https://issues.apache.org/jira/browse/HIVE-22170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924990#comment-16924990
]
Hive QA commented on HIVE-22170:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
38s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
9s{color} | {color:red} ql: The patch generated 17 new + 4140 unchanged - 3
fixed = 4157 total (was 4143) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
50s{color} | {color:red} ql generated 2 new + 2243 unchanged - 3 fixed = 2245
total (was 2246) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 27s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Class
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorUDFUnixTimeStampDate
defines non-transient non-serializable instance field date In
VectorUDFUnixTimeStampDate.java:instance field date In
VectorUDFUnixTimeStampDate.java |
| | Class
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorUDFUnixTimeStampTimestamp
defines non-transient non-serializable instance field timestamp In
VectorUDFUnixTimeStampTimestamp.java:instance field timestamp In
VectorUDFUnixTimeStampTimestamp.java |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18499/dev-support/hive-personality.sh
|
| git revision | master / 5565311 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18499/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18499/yetus/whitespace-eol.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18499/yetus/new-findbugs-ql.html
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18499/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> from_unixtime and unix_timestamp should use user session time zone
> --
>
> Key: HIVE-22170
> URL: https://issues.apache.org/jira/browse/HIVE-22170
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0, 4.0.0, 3.2.0, 3.1.1, 3.1.2
>Reporter: Riju Trivedi
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-22170.01.patch, HIVE-22170.02.patch,
> HIVE-22170.03.patch
>
>
> According to documentation, that is the expected behavior (since session time
> zone
[jira] [Updated] (HIVE-22179) Break up DDLSemanticAnalyzer - extract Function related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22179: -- Description: DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is to refactor it in order to have everything cut into more handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: * have a separate class for each analyzers * have a package for each operation, containing an analyzer, a description, and an operation, so the amount of classes under a package is more manageable Step #2: extract all the functin related analyzers from DDLSemanticAnalyzer and FunctionSemanticAnalyzer and MacroSemanticAnalyzer, and move them under the new package. was: DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is to refactor it in order to have everything cut into more handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: * have a separate class for each analyzers * have a package for each operation, containing an analyzer, a description, and an operation, so the amount of classes under a package is more manageable Step #2: extract all the functin related analyzers from DDLSemanticAnalyzer and FunctionAnalyzer, and move them under the new package. > Break up DDLSemanticAnalyzer - extract Function related analyzers > - > > Key: HIVE-22179 > URL: https://issues.apache.org/jira/browse/HIVE-22179 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22179.01.patch > > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #2: extract all the functin related analyzers from DDLSemanticAnalyzer > and FunctionSemanticAnalyzer and MacroSemanticAnalyzer, and move them under > the new package. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22179) Break up DDLSemanticAnalyzer - extract Function related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22179: -- Attachment: HIVE-22179.01.patch > Break up DDLSemanticAnalyzer - extract Function related analyzers > - > > Key: HIVE-22179 > URL: https://issues.apache.org/jira/browse/HIVE-22179 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22179.01.patch > > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #2: extract all the functin related analyzers from DDLSemanticAnalyzer > and FunctionAnalyzer, and move them under the new package. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22179) Break up DDLSemanticAnalyzer - extract Function related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22179: -- Status: Patch Available (was: Open) > Break up DDLSemanticAnalyzer - extract Function related analyzers > - > > Key: HIVE-22179 > URL: https://issues.apache.org/jira/browse/HIVE-22179 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22179.01.patch > > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #2: extract all the functin related analyzers from DDLSemanticAnalyzer > and FunctionAnalyzer, and move them under the new package. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (HIVE-22179) Break up DDLSemanticAnalyzer - extract Function related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely reassigned HIVE-22179: - > Break up DDLSemanticAnalyzer - extract Function related analyzers > - > > Key: HIVE-22179 > URL: https://issues.apache.org/jira/browse/HIVE-22179 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #1: extract all the database related analyzers from DDLSemanticAnalyzer, > and move them under the new package. Also create the new internal framework. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22179) Break up DDLSemanticAnalyzer - extract Function related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22179: -- Description: DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is to refactor it in order to have everything cut into more handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: * have a separate class for each analyzers * have a package for each operation, containing an analyzer, a description, and an operation, so the amount of classes under a package is more manageable Step #2: extract all the functin related analyzers from DDLSemanticAnalyzer and FunctionAnalyzer, and move them under the new package. was: DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is to refactor it in order to have everything cut into more handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: * have a separate class for each analyzers * have a package for each operation, containing an analyzer, a description, and an operation, so the amount of classes under a package is more manageable Step #1: extract all the database related analyzers from DDLSemanticAnalyzer, and move them under the new package. Also create the new internal framework. > Break up DDLSemanticAnalyzer - extract Function related analyzers > - > > Key: HIVE-22179 > URL: https://issues.apache.org/jira/browse/HIVE-22179 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #2: extract all the functin related analyzers from DDLSemanticAnalyzer > and FunctionAnalyzer, and move them under the new package. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22143) Break up DDLSemanticAnalyzer - extract Database related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22143: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Break up DDLSemanticAnalyzer - extract Database related analyzers > - > > Key: HIVE-22143 > URL: https://issues.apache.org/jira/browse/HIVE-22143 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22143.01.patch, HIVE-22143.02.patch, > HIVE-22143.04.patch, HIVE-22143.05.patch > > Time Spent: 1h 50m > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #1: extract all the database related analyzers from DDLSemanticAnalyzer, > and move them under the new package. Also create the new internal framework. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22143) Break up DDLSemanticAnalyzer - extract Database related analyzers
[
https://issues.apache.org/jira/browse/HIVE-22143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924985#comment-16924985
]
Hive QA commented on HIVE-22143:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979765/HIVE-22143.05.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16748 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18498/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18498/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18498/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979765 - PreCommit-HIVE-Build
> Break up DDLSemanticAnalyzer - extract Database related analyzers
> -
>
> Key: HIVE-22143
> URL: https://issues.apache.org/jira/browse/HIVE-22143
> Project: Hive
> Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available, refactor-ddl
> Attachments: HIVE-22143.01.patch, HIVE-22143.02.patch,
> HIVE-22143.04.patch, HIVE-22143.05.patch
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is
> to refactor it in order to have everything cut into more handleable classes
> under the package org.apache.hadoop.hive.ql.exec.ddl:
> * have a separate class for each analyzers
> * have a package for each operation, containing an analyzer, a description,
> and an operation, so the amount of classes under a package is more manageable
> Step #1: extract all the database related analyzers from DDLSemanticAnalyzer,
> and move them under the new package. Also create the new internal framework.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22143) Break up DDLSemanticAnalyzer - extract Database related analyzers
[
https://issues.apache.org/jira/browse/HIVE-22143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924982#comment-16924982
]
Hive QA commented on HIVE-22143:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
23s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
41s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
32s{color} | {color:blue} hcatalog/core in master has 29 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} ql: The patch generated 0 new + 931 unchanged - 49
fixed = 931 total (was 980) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} The patch core passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 16s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18498/dev-support/hive-personality.sh
|
| git revision | master / 287e5d5 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql hcatalog/core U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18498/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Break up DDLSemanticAnalyzer - extract Database related analyzers
> -
>
> Key: HIVE-22143
> URL: https://issues.apache.org/jira/browse/HIVE-22143
> Project: Hive
> Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available, refactor-ddl
> Attachments: HIVE-22143.01.patch, HIVE-22143.02.patch,
> HIVE-22143.04.patch, HIVE-22143.05.patch
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is
> to refactor it in order to have everything cut into more handleable classes
> under the package org.apache.hadoop.hive.ql.exec.ddl:
> * have a separate class for each analyzers
> * have a package for each operation, containing an analyzer, a description,
> and an operation, so the amount of class
[jira] [Updated] (HIVE-22170) from_unixtime and unix_timestamp should use user session time zone
[ https://issues.apache.org/jira/browse/HIVE-22170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22170: --- Attachment: HIVE-22170.03.patch > from_unixtime and unix_timestamp should use user session time zone > -- > > Key: HIVE-22170 > URL: https://issues.apache.org/jira/browse/HIVE-22170 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0, 4.0.0, 3.2.0, 3.1.1, 3.1.2 >Reporter: Riju Trivedi >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-22170.01.patch, HIVE-22170.02.patch, > HIVE-22170.03.patch > > > According to documentation, that is the expected behavior (since session time > zone was not present, system time zone was being used previously). This was > incorrectly changed by HIVE-12192 / HIVE-20007. This JIRA should fix this > issue. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22143) Break up DDLSemanticAnalyzer - extract Database related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22143: -- Attachment: HIVE-22143.05.patch > Break up DDLSemanticAnalyzer - extract Database related analyzers > - > > Key: HIVE-22143 > URL: https://issues.apache.org/jira/browse/HIVE-22143 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22143.01.patch, HIVE-22143.02.patch, > HIVE-22143.04.patch, HIVE-22143.05.patch > > Time Spent: 1h 50m > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #1: extract all the database related analyzers from DDLSemanticAnalyzer, > and move them under the new package. Also create the new internal framework. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22143) Break up DDLSemanticAnalyzer - extract Database related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22143: -- Attachment: (was: HIVE-22143.05.patch) > Break up DDLSemanticAnalyzer - extract Database related analyzers > - > > Key: HIVE-22143 > URL: https://issues.apache.org/jira/browse/HIVE-22143 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22143.01.patch, HIVE-22143.02.patch, > HIVE-22143.04.patch, HIVE-22143.05.patch > > Time Spent: 1h 50m > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #1: extract all the database related analyzers from DDLSemanticAnalyzer, > and move them under the new package. Also create the new internal framework. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22143) Break up DDLSemanticAnalyzer - extract Database related analyzers
[
https://issues.apache.org/jira/browse/HIVE-22143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924976#comment-16924976
]
Hive QA commented on HIVE-22143:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979763/HIVE-22143.05.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16748 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.llap.cache.TestBuddyAllocator.testMTT[2] (batchId=361)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18497/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18497/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18497/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979763 - PreCommit-HIVE-Build
> Break up DDLSemanticAnalyzer - extract Database related analyzers
> -
>
> Key: HIVE-22143
> URL: https://issues.apache.org/jira/browse/HIVE-22143
> Project: Hive
> Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available, refactor-ddl
> Attachments: HIVE-22143.01.patch, HIVE-22143.02.patch,
> HIVE-22143.04.patch, HIVE-22143.05.patch
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is
> to refactor it in order to have everything cut into more handleable classes
> under the package org.apache.hadoop.hive.ql.exec.ddl:
> * have a separate class for each analyzers
> * have a package for each operation, containing an analyzer, a description,
> and an operation, so the amount of classes under a package is more manageable
> Step #1: extract all the database related analyzers from DDLSemanticAnalyzer,
> and move them under the new package. Also create the new internal framework.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22143) Break up DDLSemanticAnalyzer - extract Database related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924966#comment-16924966 ] Jesus Camacho Rodriguez commented on HIVE-22143: +1 > Break up DDLSemanticAnalyzer - extract Database related analyzers > - > > Key: HIVE-22143 > URL: https://issues.apache.org/jira/browse/HIVE-22143 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22143.01.patch, HIVE-22143.02.patch, > HIVE-22143.04.patch, HIVE-22143.05.patch > > Time Spent: 1h 50m > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #1: extract all the database related analyzers from DDLSemanticAnalyzer, > and move them under the new package. Also create the new internal framework. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22143) Break up DDLSemanticAnalyzer - extract Database related analyzers
[
https://issues.apache.org/jira/browse/HIVE-22143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924965#comment-16924965
]
Hive QA commented on HIVE-22143:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
26s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
32s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
32s{color} | {color:blue} hcatalog/core in master has 29 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} ql: The patch generated 0 new + 931 unchanged - 49
fixed = 931 total (was 980) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch core passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 38s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18497/dev-support/hive-personality.sh
|
| git revision | master / 287e5d5 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql hcatalog/core U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18497/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Break up DDLSemanticAnalyzer - extract Database related analyzers
> -
>
> Key: HIVE-22143
> URL: https://issues.apache.org/jira/browse/HIVE-22143
> Project: Hive
> Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available, refactor-ddl
> Attachments: HIVE-22143.01.patch, HIVE-22143.02.patch,
> HIVE-22143.04.patch, HIVE-22143.05.patch
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is
> to refactor it in order to have everything cut into more handleable classes
> under the package org.apache.hadoop.hive.ql.exec.ddl:
> * have a separate class for each analyzers
> * have a package for each operation, containing an analyzer, a description,
> and an operation, so the amount of class
[jira] [Updated] (HIVE-22143) Break up DDLSemanticAnalyzer - extract Database related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22143: -- Attachment: HIVE-22143.05.patch > Break up DDLSemanticAnalyzer - extract Database related analyzers > - > > Key: HIVE-22143 > URL: https://issues.apache.org/jira/browse/HIVE-22143 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22143.01.patch, HIVE-22143.02.patch, > HIVE-22143.04.patch, HIVE-22143.05.patch > > Time Spent: 1h 50m > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #1: extract all the database related analyzers from DDLSemanticAnalyzer, > and move them under the new package. Also create the new internal framework. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22143) Break up DDLSemanticAnalyzer - extract Database related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22143: -- Attachment: (was: HIVE-22143.05.patch) > Break up DDLSemanticAnalyzer - extract Database related analyzers > - > > Key: HIVE-22143 > URL: https://issues.apache.org/jira/browse/HIVE-22143 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22143.01.patch, HIVE-22143.02.patch, > HIVE-22143.04.patch, HIVE-22143.05.patch > > Time Spent: 1h 50m > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #1: extract all the database related analyzers from DDLSemanticAnalyzer, > and move them under the new package. Also create the new internal framework. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22143) Break up DDLSemanticAnalyzer - extract Database related analyzers
[
https://issues.apache.org/jira/browse/HIVE-22143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924888#comment-16924888
]
Hive QA commented on HIVE-22143:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979754/HIVE-22143.05.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18496/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18496/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18496/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2019-09-07 14:29:57.936
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-18496/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2019-09-07 14:29:57.939
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 287e5d5 HIVE-22099: Several date related UDFs can't handle
Julian dates properly since HIVE-20007 (Adam Szita, reviewed by Jesus Camacho
Rodriguez)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 287e5d5 HIVE-22099: Several date related UDFs can't handle
Julian dates properly since HIVE-20007 (Adam Szita, reviewed by Jesus Camacho
Rodriguez)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2019-09-07 14:29:58.551
+ rm -rf ../yetus_PreCommit-HIVE-Build-18496
+ mkdir ../yetus_PreCommit-HIVE-Build-18496
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-18496
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-18496/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Going to apply patch with: git apply -p0
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q
-Dmaven.repo.local=/data/hiveptest/working/maven
protoc-jar: executing: [/tmp/protoc1121108753710818960.exe, --version]
libprotoc 2.5.0
protoc-jar: executing: [/tmp/protoc1121108753710818960.exe,
-I/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore,
--java_out=/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/target/generated-sources,
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore/metastore.proto]
ANTLR Parser Generator Version 3.5.2
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process
(process-resource-bundles) on project hive-shims: Execution
process-resource-bundles of goal
org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process failed.
ConcurrentModificationException -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :hive-shims
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-18496
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979754 - PreCommit-HIVE-Build
> Break up DDLSema
[jira] [Commented] (HIVE-22149) Metastore: Unify codahale metrics.log json structure between hiveserver2 and metastore services
[
https://issues.apache.org/jira/browse/HIVE-22149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924887#comment-16924887
]
Hive QA commented on HIVE-22149:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979751/HIVE-22149.03.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16749 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18495/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18495/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18495/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979751 - PreCommit-HIVE-Build
> Metastore: Unify codahale metrics.log json structure between hiveserver2 and
> metastore services
> ---
>
> Key: HIVE-22149
> URL: https://issues.apache.org/jira/browse/HIVE-22149
> Project: Hive
> Issue Type: Bug
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-22149.01.patch, HIVE-22149.01.patch,
> HIVE-22149.01.patch, HIVE-22149.02.patch, HIVE-22149.03.patch,
> HIVE-22149.03.patch, HIVE-22149.03.patch, metrics_hiveserver2.log,
> metrics_metastore.log
>
>
> While fixing HIVE-22140 I found some really annoying differences between the
> codahale metric file structures between hiveserver2 and metastore, e.g.
> open_connections: can be found in "counters" for hs2, but in "gauges" for ms
> threads count: it's a proper "threads.count" for hs2, but a really ambiguous
> "count" for ms
> so I realized that "memory." and "threads." prefix is completely absent in ms
> metrics file, which is misleading
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22149) Metastore: Unify codahale metrics.log json structure between hiveserver2 and metastore services
[
https://issues.apache.org/jira/browse/HIVE-22149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924879#comment-16924879
]
Hive QA commented on HIVE-22149:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
24s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
5s{color} | {color:blue} standalone-metastore/metastore-server in master has
181 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
35s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
19s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 2 new + 439 unchanged - 1 fixed = 441 total (was 440) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 23s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18495/dev-support/hive-personality.sh
|
| git revision | master / 287e5d5 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18495/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| modules | C: standalone-metastore/metastore-server itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18495/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Metastore: Unify codahale metrics.log json structure between hiveserver2 and
> metastore services
> ---
>
> Key: HIVE-22149
> URL: https://issues.apache.org/jira/browse/HIVE-22149
> Project: Hive
> Issue Type: Bug
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-22149.01.patch, HIVE-22149.01.patch,
> HIVE-22149.01.patch, HIVE-22149.02.patch, HIVE-22149.03.patch,
> HIVE-22149.03.patch, HIVE-22149.03.patch, metrics_hiveserver2.log,
> metrics_metastore.log
>
>
> While fixing HIVE-22140 I found some really annoying differences between the
> codahale metric file structures between hiveserver2 and metastore, e.g.
> open_connections: can be found in "counters" for hs2, but in "gauges" for ms
> threads
[jira] [Commented] (HIVE-22178) Parquet FilterPredicate throws CastException after SchemaEvolution.
[
https://issues.apache.org/jira/browse/HIVE-22178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924870#comment-16924870
]
Hive QA commented on HIVE-22178:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979750/HIVE-22178.3.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16748 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18494/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18494/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18494/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979750 - PreCommit-HIVE-Build
> Parquet FilterPredicate throws CastException after SchemaEvolution.
> ---
>
> Key: HIVE-22178
> URL: https://issues.apache.org/jira/browse/HIVE-22178
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: Naresh P R
>Assignee: Naresh P R
>Priority: Major
> Attachments: HIVE-22178.1.patch, HIVE-22178.2.patch,
> HIVE-22178.3.patch
>
>
> Below are the repro steps.
> {code:java}
> create table parq_test(age int, name string) stored as parquet;
> insert into parq_test values(1, '');
> alter table parq_test change age age string;
> insert into parq_test values('b', '');
> select * from parq_test where age='b';{code}
> Exception thrown after changing column datatype is below
> {code:java}
> Caused by: java.lang.IllegalArgumentException: FilterPredicate column: age's
> declared type (org.apache.parquet.io.api.Binary) does not match the schema
> found in file metadata. Column age is of type: INT32
> Valid types for this column are: [class java.lang.Integer]
> at
> org.apache.parquet.filter2.predicate.ValidTypeMap.assertTypeValid(ValidTypeMap.java:126)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validateColumn(SchemaCompatibilityValidator.java:181)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validateColumnFilterPredicate(SchemaCompatibilityValidator.java:151)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.visit(SchemaCompatibilityValidator.java:85)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.visit(SchemaCompatibilityValidator.java:58)
> at
> org.apache.parquet.filter2.predicate.Operators$Eq.accept(Operators.java:181)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validate(SchemaCompatibilityValidator.java:63)
> at
> org.apache.parquet.filter2.compat.RowGroupFilter.visit(RowGroupFilter.java:92)
> at
> org.apache.parquet.filter2.compat.RowGroupFilter.visit(RowGroupFilter.java:43)
> at
> org.apache.parquet.filter2.compat.FilterCompat$FilterPredicateCompat.accept(FilterCompat.java:137)
> at
> org.apache.parquet.filter2.compat.RowGroupFilter.filterRowGroups(RowGroupFilter.java:64)
> at
> org.apache.hadoop.hive.ql.io.parquet.ParquetRecordReaderBase.getSplit(ParquetRecordReaderBase.java:111)
> at
> org.apache.hadoop.hive.ql.io.parquet.vector.VectorizedParquetRecordReader.(VectorizedParquetRecordReader.java:147)
> ... 31 more{code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22143) Break up DDLSemanticAnalyzer - extract Database related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22143: -- Attachment: (was: HIVE-22143.05.patch) > Break up DDLSemanticAnalyzer - extract Database related analyzers > - > > Key: HIVE-22143 > URL: https://issues.apache.org/jira/browse/HIVE-22143 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22143.01.patch, HIVE-22143.02.patch, > HIVE-22143.04.patch, HIVE-22143.05.patch > > Time Spent: 1h 50m > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #1: extract all the database related analyzers from DDLSemanticAnalyzer, > and move them under the new package. Also create the new internal framework. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22143) Break up DDLSemanticAnalyzer - extract Database related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22143: -- Attachment: HIVE-22143.05.patch > Break up DDLSemanticAnalyzer - extract Database related analyzers > - > > Key: HIVE-22143 > URL: https://issues.apache.org/jira/browse/HIVE-22143 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22143.01.patch, HIVE-22143.02.patch, > HIVE-22143.04.patch, HIVE-22143.05.patch > > Time Spent: 1h 50m > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #1: extract all the database related analyzers from DDLSemanticAnalyzer, > and move them under the new package. Also create the new internal framework. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22178) Parquet FilterPredicate throws CastException after SchemaEvolution.
[
https://issues.apache.org/jira/browse/HIVE-22178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924844#comment-16924844
]
Hive QA commented on HIVE-22178:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
44s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
37s{color} | {color:red} ql: The patch generated 20 new + 64 unchanged - 12
fixed = 84 total (was 76) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 23s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18494/dev-support/hive-personality.sh
|
| git revision | master / 287e5d5 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18494/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18494/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Parquet FilterPredicate throws CastException after SchemaEvolution.
> ---
>
> Key: HIVE-22178
> URL: https://issues.apache.org/jira/browse/HIVE-22178
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: Naresh P R
>Assignee: Naresh P R
>Priority: Major
> Attachments: HIVE-22178.1.patch, HIVE-22178.2.patch,
> HIVE-22178.3.patch
>
>
> Below are the repro steps.
> {code:java}
> create table parq_test(age int, name string) stored as parquet;
> insert into parq_test values(1, '');
> alter table parq_test change age age string;
> insert into parq_test values('b', '');
> select * from parq_test where age='b';{code}
> Exception thrown after changing column datatype is below
> {code:java}
> Caused by: java.lang.IllegalArgumentException: FilterPredicate column: age's
> declared type (org.apache.parquet.io.api.Binary) does not match the schema
> found in file metadata. Column age is of type: INT32
> Valid types for this column are: [class java.lang.Integer]
> at
> org.apache.parquet.filter2.predicate.ValidTypeMap.assertTypeValid(ValidTypeMap.java:126)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validateColumn(SchemaCompatibilityValidator.java:181)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validateColumnFilterPredicate(SchemaCompatibilityValidator.java:151)
> at
> org.apache.parque
[jira] [Commented] (HIVE-21510) Vectorization: add support for and/or for (constant,column) cases
[
https://issues.apache.org/jira/browse/HIVE-21510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924839#comment-16924839
]
Laszlo Bodor commented on HIVE-21510:
-
could you please review [~kgyrtkirk]?
> Vectorization: add support for and/or for (constant,column) cases
> -
>
> Key: HIVE-21510
> URL: https://issues.apache.org/jira/browse/HIVE-21510
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-21510.01.patch, HIVE-21510.02.patch,
> HIVE-21510.03.patch, HIVE-21510.04.patch
>
>
> After HIVE-21001 some selectExpressions will start using VectorUDFAdaptor for
> "null and x" expressions. Because right now there are 2-3 places from which
> rewrite will be done to the form of "null and/or x" form; it would be better
> to support it.
> {code}
> [...]
> selectExpressions: VectorUDFAdaptor((null and dt1 is null))
> [...]
> usesVectorUDFAdaptor: true
> [...]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22149) Metastore: Unify codahale metrics.log json structure between hiveserver2 and metastore services
[ https://issues.apache.org/jira/browse/HIVE-22149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-22149: Attachment: HIVE-22149.03.patch > Metastore: Unify codahale metrics.log json structure between hiveserver2 and > metastore services > --- > > Key: HIVE-22149 > URL: https://issues.apache.org/jira/browse/HIVE-22149 > Project: Hive > Issue Type: Bug >Reporter: Laszlo Bodor >Assignee: Laszlo Bodor >Priority: Major > Attachments: HIVE-22149.01.patch, HIVE-22149.01.patch, > HIVE-22149.01.patch, HIVE-22149.02.patch, HIVE-22149.03.patch, > HIVE-22149.03.patch, HIVE-22149.03.patch, metrics_hiveserver2.log, > metrics_metastore.log > > > While fixing HIVE-22140 I found some really annoying differences between the > codahale metric file structures between hiveserver2 and metastore, e.g. > open_connections: can be found in "counters" for hs2, but in "gauges" for ms > threads count: it's a proper "threads.count" for hs2, but a really ambiguous > "count" for ms > so I realized that "memory." and "threads." prefix is completely absent in ms > metrics file, which is misleading -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22178) Parquet FilterPredicate throws CastException after SchemaEvolution.
[
https://issues.apache.org/jira/browse/HIVE-22178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naresh P R updated HIVE-22178:
--
Attachment: HIVE-22178.3.patch
> Parquet FilterPredicate throws CastException after SchemaEvolution.
> ---
>
> Key: HIVE-22178
> URL: https://issues.apache.org/jira/browse/HIVE-22178
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: Naresh P R
>Assignee: Naresh P R
>Priority: Major
> Attachments: HIVE-22178.1.patch, HIVE-22178.2.patch,
> HIVE-22178.3.patch
>
>
> Below are the repro steps.
> {code:java}
> create table parq_test(age int, name string) stored as parquet;
> insert into parq_test values(1, '');
> alter table parq_test change age age string;
> insert into parq_test values('b', '');
> select * from parq_test where age='b';{code}
> Exception thrown after changing column datatype is below
> {code:java}
> Caused by: java.lang.IllegalArgumentException: FilterPredicate column: age's
> declared type (org.apache.parquet.io.api.Binary) does not match the schema
> found in file metadata. Column age is of type: INT32
> Valid types for this column are: [class java.lang.Integer]
> at
> org.apache.parquet.filter2.predicate.ValidTypeMap.assertTypeValid(ValidTypeMap.java:126)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validateColumn(SchemaCompatibilityValidator.java:181)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validateColumnFilterPredicate(SchemaCompatibilityValidator.java:151)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.visit(SchemaCompatibilityValidator.java:85)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.visit(SchemaCompatibilityValidator.java:58)
> at
> org.apache.parquet.filter2.predicate.Operators$Eq.accept(Operators.java:181)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validate(SchemaCompatibilityValidator.java:63)
> at
> org.apache.parquet.filter2.compat.RowGroupFilter.visit(RowGroupFilter.java:92)
> at
> org.apache.parquet.filter2.compat.RowGroupFilter.visit(RowGroupFilter.java:43)
> at
> org.apache.parquet.filter2.compat.FilterCompat$FilterPredicateCompat.accept(FilterCompat.java:137)
> at
> org.apache.parquet.filter2.compat.RowGroupFilter.filterRowGroups(RowGroupFilter.java:64)
> at
> org.apache.hadoop.hive.ql.io.parquet.ParquetRecordReaderBase.getSplit(ParquetRecordReaderBase.java:111)
> at
> org.apache.hadoop.hive.ql.io.parquet.vector.VectorizedParquetRecordReader.(VectorizedParquetRecordReader.java:147)
> ... 31 more{code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22143) Break up DDLSemanticAnalyzer - extract Database related analyzers
[
https://issues.apache.org/jira/browse/HIVE-22143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924819#comment-16924819
]
Hive QA commented on HIVE-22143:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979748/HIVE-22143.05.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18493/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18493/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18493/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2019-09-07 10:46:21.771
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-18493/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2019-09-07 10:46:21.775
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 287e5d5 HIVE-22099: Several date related UDFs can't handle
Julian dates properly since HIVE-20007 (Adam Szita, reviewed by Jesus Camacho
Rodriguez)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 287e5d5 HIVE-22099: Several date related UDFs can't handle
Julian dates properly since HIVE-20007 (Adam Szita, reviewed by Jesus Camacho
Rodriguez)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2019-09-07 10:46:22.598
+ rm -rf ../yetus_PreCommit-HIVE-Build-18493
+ mkdir ../yetus_PreCommit-HIVE-Build-18493
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-18493
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-18493/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Going to apply patch with: git apply -p0
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q
-Dmaven.repo.local=/data/hiveptest/working/maven
protoc-jar: executing: [/tmp/protoc2415085403864744638.exe, --version]
libprotoc 2.5.0
protoc-jar: executing: [/tmp/protoc2415085403864744638.exe,
-I/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore,
--java_out=/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/target/generated-sources,
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore/metastore.proto]
ANTLR Parser Generator Version 3.5.2
protoc-jar: executing: [/tmp/protoc6485902289057344749.exe, --version]
libprotoc 2.5.0
ANTLR Parser Generator Version 3.5.2
Output file
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-server/target/generated-sources/org/apache/hadoop/hive/metastore/parser/FilterParser.java
does not exist: must build
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/parser/Filter.g
org/apache/hadoop/hive/metastore/parser/Filter.g
log4j:WARN No appenders could be found for logger (DataNucleus.Persistence).
log4j:WARN Please initialize the log4j system properly.
DataNucleus Enhancer (version 4.1.17) for API "JDO"
DataNucleus Enhancer completed with success for 41 classes.
ANTLR Parser Generator Version 3.5.2
Output file
/data/hiveptest/working/apache-github-source-source/ql/target/generated-sources/antlr3/org/apache/hadoop/hive/ql/parse/HiveLexer.java
does not exist: must build
/data/hive
[jira] [Commented] (HIVE-22178) Parquet FilterPredicate throws CastException after SchemaEvolution.
[
https://issues.apache.org/jira/browse/HIVE-22178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924816#comment-16924816
]
Hive QA commented on HIVE-22178:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979743/HIVE-22178.2.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16744 tests
executed
*Failed tests:*
{noformat}
TestStatsReplicationScenariosACIDNoAutogather - did not produce a TEST-*.xml
file (likely timed out) (batchId=254)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18492/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18492/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18492/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979743 - PreCommit-HIVE-Build
> Parquet FilterPredicate throws CastException after SchemaEvolution.
> ---
>
> Key: HIVE-22178
> URL: https://issues.apache.org/jira/browse/HIVE-22178
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: Naresh P R
>Assignee: Naresh P R
>Priority: Major
> Attachments: HIVE-22178.1.patch, HIVE-22178.2.patch
>
>
> Below are the repro steps.
> {code:java}
> create table parq_test(age int, name string) stored as parquet;
> insert into parq_test values(1, '');
> alter table parq_test change age age string;
> insert into parq_test values('b', '');
> select * from parq_test where age='b';{code}
> Exception thrown after changing column datatype is below
> {code:java}
> Caused by: java.lang.IllegalArgumentException: FilterPredicate column: age's
> declared type (org.apache.parquet.io.api.Binary) does not match the schema
> found in file metadata. Column age is of type: INT32
> Valid types for this column are: [class java.lang.Integer]
> at
> org.apache.parquet.filter2.predicate.ValidTypeMap.assertTypeValid(ValidTypeMap.java:126)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validateColumn(SchemaCompatibilityValidator.java:181)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validateColumnFilterPredicate(SchemaCompatibilityValidator.java:151)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.visit(SchemaCompatibilityValidator.java:85)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.visit(SchemaCompatibilityValidator.java:58)
> at
> org.apache.parquet.filter2.predicate.Operators$Eq.accept(Operators.java:181)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validate(SchemaCompatibilityValidator.java:63)
> at
> org.apache.parquet.filter2.compat.RowGroupFilter.visit(RowGroupFilter.java:92)
> at
> org.apache.parquet.filter2.compat.RowGroupFilter.visit(RowGroupFilter.java:43)
> at
> org.apache.parquet.filter2.compat.FilterCompat$FilterPredicateCompat.accept(FilterCompat.java:137)
> at
> org.apache.parquet.filter2.compat.RowGroupFilter.filterRowGroups(RowGroupFilter.java:64)
> at
> org.apache.hadoop.hive.ql.io.parquet.ParquetRecordReaderBase.getSplit(ParquetRecordReaderBase.java:111)
> at
> org.apache.hadoop.hive.ql.io.parquet.vector.VectorizedParquetRecordReader.(VectorizedParquetRecordReader.java:147)
> ... 31 more{code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22143) Break up DDLSemanticAnalyzer - extract Database related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22143: -- Attachment: HIVE-22143.05.patch > Break up DDLSemanticAnalyzer - extract Database related analyzers > - > > Key: HIVE-22143 > URL: https://issues.apache.org/jira/browse/HIVE-22143 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22143.01.patch, HIVE-22143.02.patch, > HIVE-22143.04.patch, HIVE-22143.05.patch > > Time Spent: 1h 50m > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #1: extract all the database related analyzers from DDLSemanticAnalyzer, > and move them under the new package. Also create the new internal framework. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work logged] (HIVE-22143) Break up DDLSemanticAnalyzer - extract Database related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22143?focusedWorklogId=308333&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-308333 ] ASF GitHub Bot logged work on HIVE-22143: - Author: ASF GitHub Bot Created on: 07/Sep/19 09:53 Start Date: 07/Sep/19 09:53 Worklog Time Spent: 10m Work Description: miklosgergely commented on pull request #757: HIVE-22143 Break up DDLSemanticAnalyzer - extract Database related analyzers URL: https://github.com/apache/hive/pull/757#discussion_r321963312 ## File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/bootstrap/load/LoadDatabase.java ## @@ -21,9 +21,9 @@ import org.apache.hadoop.hive.metastore.api.Database; import org.apache.hadoop.hive.metastore.api.InvalidOperationException; import org.apache.hadoop.hive.ql.ddl.DDLWork; -import org.apache.hadoop.hive.ql.ddl.database.AlterDatabaseSetOwnerDesc; -import org.apache.hadoop.hive.ql.ddl.database.AlterDatabaseSetPropertiesDesc; -import org.apache.hadoop.hive.ql.ddl.database.CreateDatabaseDesc; +import org.apache.hadoop.hive.ql.ddl.database.create.CreateDatabaseDesc; +import org.apache.hadoop.hive.ql.ddl.database.setowner.AlterDatabaseSetOwnerDesc; Review comment: Created 3 sub packages under alter. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 308333) Time Spent: 1h 50m (was: 1h 40m) > Break up DDLSemanticAnalyzer - extract Database related analyzers > - > > Key: HIVE-22143 > URL: https://issues.apache.org/jira/browse/HIVE-22143 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22143.01.patch, HIVE-22143.02.patch, > HIVE-22143.04.patch > > Time Spent: 1h 50m > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #1: extract all the database related analyzers from DDLSemanticAnalyzer, > and move them under the new package. Also create the new internal framework. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work logged] (HIVE-22143) Break up DDLSemanticAnalyzer - extract Database related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22143?focusedWorklogId=308332&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-308332 ] ASF GitHub Bot logged work on HIVE-22143: - Author: ASF GitHub Bot Created on: 07/Sep/19 09:53 Start Date: 07/Sep/19 09:53 Worklog Time Spent: 10m Work Description: miklosgergely commented on pull request #757: HIVE-22143 Break up DDLSemanticAnalyzer - extract Database related analyzers URL: https://github.com/apache/hive/pull/757#discussion_r321963301 ## File path: hcatalog/core/src/main/java/org/apache/hive/hcatalog/cli/SemanticAnalysis/HCatSemanticAnalyzer.java ## @@ -21,10 +21,10 @@ import org.apache.hadoop.hive.metastore.api.Database; import org.apache.hadoop.hive.ql.ddl.DDLDesc; import org.apache.hadoop.hive.ql.ddl.DDLWork; -import org.apache.hadoop.hive.ql.ddl.database.DescDatabaseDesc; -import org.apache.hadoop.hive.ql.ddl.database.DropDatabaseDesc; -import org.apache.hadoop.hive.ql.ddl.database.ShowDatabasesDesc; -import org.apache.hadoop.hive.ql.ddl.database.SwitchDatabaseDesc; +import org.apache.hadoop.hive.ql.ddl.database.desc.DescDatabaseDesc; +import org.apache.hadoop.hive.ql.ddl.database.drop.DropDatabaseDesc; +import org.apache.hadoop.hive.ql.ddl.database.show.ShowDatabasesDesc; +import org.apache.hadoop.hive.ql.ddl.database.switchdb.SwitchDatabaseDesc; Review comment: Switched switch to use. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 308332) Time Spent: 1h 40m (was: 1.5h) > Break up DDLSemanticAnalyzer - extract Database related analyzers > - > > Key: HIVE-22143 > URL: https://issues.apache.org/jira/browse/HIVE-22143 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22143.01.patch, HIVE-22143.02.patch, > HIVE-22143.04.patch > > Time Spent: 1h 40m > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #1: extract all the database related analyzers from DDLSemanticAnalyzer, > and move them under the new package. Also create the new internal framework. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22178) Parquet FilterPredicate throws CastException after SchemaEvolution.
[
https://issues.apache.org/jira/browse/HIVE-22178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924800#comment-16924800
]
Hive QA commented on HIVE-22178:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
48s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
46s{color} | {color:red} ql: The patch generated 20 new + 64 unchanged - 12
fixed = 84 total (was 76) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 22s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18492/dev-support/hive-personality.sh
|
| git revision | master / 287e5d5 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18492/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18492/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Parquet FilterPredicate throws CastException after SchemaEvolution.
> ---
>
> Key: HIVE-22178
> URL: https://issues.apache.org/jira/browse/HIVE-22178
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: Naresh P R
>Assignee: Naresh P R
>Priority: Major
> Attachments: HIVE-22178.1.patch, HIVE-22178.2.patch
>
>
> Below are the repro steps.
> {code:java}
> create table parq_test(age int, name string) stored as parquet;
> insert into parq_test values(1, '');
> alter table parq_test change age age string;
> insert into parq_test values('b', '');
> select * from parq_test where age='b';{code}
> Exception thrown after changing column datatype is below
> {code:java}
> Caused by: java.lang.IllegalArgumentException: FilterPredicate column: age's
> declared type (org.apache.parquet.io.api.Binary) does not match the schema
> found in file metadata. Column age is of type: INT32
> Valid types for this column are: [class java.lang.Integer]
> at
> org.apache.parquet.filter2.predicate.ValidTypeMap.assertTypeValid(ValidTypeMap.java:126)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validateColumn(SchemaCompatibilityValidator.java:181)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validateColumnFilterPredicate(SchemaCompatibilityValidator.java:151)
> at
> org.apache.parquet.filter2.predicate.Sch
[jira] [Commented] (HIVE-22149) Metastore: Unify codahale metrics.log json structure between hiveserver2 and metastore services
[
https://issues.apache.org/jira/browse/HIVE-22149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924787#comment-16924787
]
Hive QA commented on HIVE-22149:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979741/HIVE-22149.03.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16749 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.llap.cache.TestBuddyAllocator.testMTT[2] (batchId=361)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18491/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18491/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18491/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979741 - PreCommit-HIVE-Build
> Metastore: Unify codahale metrics.log json structure between hiveserver2 and
> metastore services
> ---
>
> Key: HIVE-22149
> URL: https://issues.apache.org/jira/browse/HIVE-22149
> Project: Hive
> Issue Type: Bug
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-22149.01.patch, HIVE-22149.01.patch,
> HIVE-22149.01.patch, HIVE-22149.02.patch, HIVE-22149.03.patch,
> HIVE-22149.03.patch, metrics_hiveserver2.log, metrics_metastore.log
>
>
> While fixing HIVE-22140 I found some really annoying differences between the
> codahale metric file structures between hiveserver2 and metastore, e.g.
> open_connections: can be found in "counters" for hs2, but in "gauges" for ms
> threads count: it's a proper "threads.count" for hs2, but a really ambiguous
> "count" for ms
> so I realized that "memory." and "threads." prefix is completely absent in ms
> metrics file, which is misleading
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22178) Parquet FilterPredicate throws CastException after SchemaEvolution.
[
https://issues.apache.org/jira/browse/HIVE-22178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naresh P R updated HIVE-22178:
--
Attachment: HIVE-22178.2.patch
> Parquet FilterPredicate throws CastException after SchemaEvolution.
> ---
>
> Key: HIVE-22178
> URL: https://issues.apache.org/jira/browse/HIVE-22178
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: Naresh P R
>Assignee: Naresh P R
>Priority: Major
> Attachments: HIVE-22178.1.patch, HIVE-22178.2.patch
>
>
> Below are the repro steps.
> {code:java}
> create table parq_test(age int, name string) stored as parquet;
> insert into parq_test values(1, '');
> alter table parq_test change age age string;
> insert into parq_test values('b', '');
> select * from parq_test where age='b';{code}
> Exception thrown after changing column datatype is below
> {code:java}
> Caused by: java.lang.IllegalArgumentException: FilterPredicate column: age's
> declared type (org.apache.parquet.io.api.Binary) does not match the schema
> found in file metadata. Column age is of type: INT32
> Valid types for this column are: [class java.lang.Integer]
> at
> org.apache.parquet.filter2.predicate.ValidTypeMap.assertTypeValid(ValidTypeMap.java:126)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validateColumn(SchemaCompatibilityValidator.java:181)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validateColumnFilterPredicate(SchemaCompatibilityValidator.java:151)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.visit(SchemaCompatibilityValidator.java:85)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.visit(SchemaCompatibilityValidator.java:58)
> at
> org.apache.parquet.filter2.predicate.Operators$Eq.accept(Operators.java:181)
> at
> org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validate(SchemaCompatibilityValidator.java:63)
> at
> org.apache.parquet.filter2.compat.RowGroupFilter.visit(RowGroupFilter.java:92)
> at
> org.apache.parquet.filter2.compat.RowGroupFilter.visit(RowGroupFilter.java:43)
> at
> org.apache.parquet.filter2.compat.FilterCompat$FilterPredicateCompat.accept(FilterCompat.java:137)
> at
> org.apache.parquet.filter2.compat.RowGroupFilter.filterRowGroups(RowGroupFilter.java:64)
> at
> org.apache.hadoop.hive.ql.io.parquet.ParquetRecordReaderBase.getSplit(ParquetRecordReaderBase.java:111)
> at
> org.apache.hadoop.hive.ql.io.parquet.vector.VectorizedParquetRecordReader.(VectorizedParquetRecordReader.java:147)
> ... 31 more{code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22149) Metastore: Unify codahale metrics.log json structure between hiveserver2 and metastore services
[
https://issues.apache.org/jira/browse/HIVE-22149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924769#comment-16924769
]
Hive QA commented on HIVE-22149:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
7s{color} | {color:blue} standalone-metastore/metastore-server in master has
181 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
20s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 2 new + 439 unchanged - 1 fixed = 441 total (was 440) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 38s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18491/dev-support/hive-personality.sh
|
| git revision | master / 287e5d5 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18491/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| modules | C: standalone-metastore/metastore-server itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18491/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Metastore: Unify codahale metrics.log json structure between hiveserver2 and
> metastore services
> ---
>
> Key: HIVE-22149
> URL: https://issues.apache.org/jira/browse/HIVE-22149
> Project: Hive
> Issue Type: Bug
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-22149.01.patch, HIVE-22149.01.patch,
> HIVE-22149.01.patch, HIVE-22149.02.patch, HIVE-22149.03.patch,
> HIVE-22149.03.patch, metrics_hiveserver2.log, metrics_metastore.log
>
>
> While fixing HIVE-22140 I found some really annoying differences between the
> codahale metric file structures between hiveserver2 and metastore, e.g.
> open_connections: can be found in "counters" for hs2, but in "gauges" for ms
> threads count: it's a proper "t
[jira] [Updated] (HIVE-22149) Metastore: Unify codahale metrics.log json structure between hiveserver2 and metastore services
[ https://issues.apache.org/jira/browse/HIVE-22149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-22149: Attachment: HIVE-22149.03.patch > Metastore: Unify codahale metrics.log json structure between hiveserver2 and > metastore services > --- > > Key: HIVE-22149 > URL: https://issues.apache.org/jira/browse/HIVE-22149 > Project: Hive > Issue Type: Bug >Reporter: Laszlo Bodor >Assignee: Laszlo Bodor >Priority: Major > Attachments: HIVE-22149.01.patch, HIVE-22149.01.patch, > HIVE-22149.01.patch, HIVE-22149.02.patch, HIVE-22149.03.patch, > HIVE-22149.03.patch, metrics_hiveserver2.log, metrics_metastore.log > > > While fixing HIVE-22140 I found some really annoying differences between the > codahale metric file structures between hiveserver2 and metastore, e.g. > open_connections: can be found in "counters" for hs2, but in "gauges" for ms > threads count: it's a proper "threads.count" for hs2, but a really ambiguous > "count" for ms > so I realized that "memory." and "threads." prefix is completely absent in ms > metrics file, which is misleading -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22149) Metastore: Unify codahale metrics.log json structure between hiveserver2 and metastore services
[
https://issues.apache.org/jira/browse/HIVE-22149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924762#comment-16924762
]
Hive QA commented on HIVE-22149:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979739/HIVE-22149.03.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 16743 tests
executed
*Failed tests:*
{noformat}
TestStatsReplicationScenariosACIDNoAutogather - did not produce a TEST-*.xml
file (likely timed out) (batchId=254)
org.apache.hadoop.hive.llap.cache.TestBuddyAllocator.testMTT[2] (batchId=361)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18490/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18490/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18490/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979739 - PreCommit-HIVE-Build
> Metastore: Unify codahale metrics.log json structure between hiveserver2 and
> metastore services
> ---
>
> Key: HIVE-22149
> URL: https://issues.apache.org/jira/browse/HIVE-22149
> Project: Hive
> Issue Type: Bug
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-22149.01.patch, HIVE-22149.01.patch,
> HIVE-22149.01.patch, HIVE-22149.02.patch, HIVE-22149.03.patch,
> metrics_hiveserver2.log, metrics_metastore.log
>
>
> While fixing HIVE-22140 I found some really annoying differences between the
> codahale metric file structures between hiveserver2 and metastore, e.g.
> open_connections: can be found in "counters" for hs2, but in "gauges" for ms
> threads count: it's a proper "threads.count" for hs2, but a really ambiguous
> "count" for ms
> so I realized that "memory." and "threads." prefix is completely absent in ms
> metrics file, which is misleading
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22099) Several date related UDFs can't handle Julian dates properly since HIVE-20007
[
https://issues.apache.org/jira/browse/HIVE-22099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Szita updated HIVE-22099:
--
Fix Version/s: 4.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Several date related UDFs can't handle Julian dates properly since HIVE-20007
> -
>
> Key: HIVE-22099
> URL: https://issues.apache.org/jira/browse/HIVE-22099
> Project: Hive
> Issue Type: Bug
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22099.0.patch, HIVE-22099.1.patch,
> HIVE-22099.2.patch, HIVE-22099.3.patch, HIVE-22099.4.patch,
> HIVE-22099.5.patch, HIVE-22099.6.patch, HIVE-22099.7.patch,
> HIVE-22099.8.patch, HIVE-22099.9.patch
>
>
> Currently dates that belong to Julian calendar (before Oct 15, 1582) are
> handled improperly by date/timestamp UDFs.
> E.g. DateFormat UDF:
> Although the dates are in Julian calendar, the formatter insists to print
> these according to Gregorian calendar causing multiple days of difference in
> some cases:
>
> {code:java}
> beeline> select date_format('1001-01-05','dd---MM--');
> ++
> | _c0 |
> ++
> | 30---12--1000 |
> ++{code}
> I've observed similar problems in the following UDFs:
> * add_months
> * date_format
> * day
> * month
> * months_between
> * weekofyear
> * year
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22099) Several date related UDFs can't handle Julian dates properly since HIVE-20007
[
https://issues.apache.org/jira/browse/HIVE-22099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924755#comment-16924755
]
Adam Szita commented on HIVE-22099:
---
Committed to master. Thanks for the review Karen and Jesus!
> Several date related UDFs can't handle Julian dates properly since HIVE-20007
> -
>
> Key: HIVE-22099
> URL: https://issues.apache.org/jira/browse/HIVE-22099
> Project: Hive
> Issue Type: Bug
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Attachments: HIVE-22099.0.patch, HIVE-22099.1.patch,
> HIVE-22099.2.patch, HIVE-22099.3.patch, HIVE-22099.4.patch,
> HIVE-22099.5.patch, HIVE-22099.6.patch, HIVE-22099.7.patch,
> HIVE-22099.8.patch, HIVE-22099.9.patch
>
>
> Currently dates that belong to Julian calendar (before Oct 15, 1582) are
> handled improperly by date/timestamp UDFs.
> E.g. DateFormat UDF:
> Although the dates are in Julian calendar, the formatter insists to print
> these according to Gregorian calendar causing multiple days of difference in
> some cases:
>
> {code:java}
> beeline> select date_format('1001-01-05','dd---MM--');
> ++
> | _c0 |
> ++
> | 30---12--1000 |
> ++{code}
> I've observed similar problems in the following UDFs:
> * add_months
> * date_format
> * day
> * month
> * months_between
> * weekofyear
> * year
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
