[jira] [Work logged] (HIVE-27261) Backport HIVE-27258 to branch-3
[ https://issues.apache.org/jira/browse/HIVE-27261?focusedWorklogId=857533&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857533 ] ASF GitHub Bot logged work on HIVE-27261: - Author: ASF GitHub Bot Created on: 18/Apr/23 06:27 Start Date: 18/Apr/23 06:27 Worklog Time Spent: 10m Work Description: amanraj2520 commented on PR #4234: URL: https://github.com/apache/hive/pull/4234#issuecomment-1512510417 I retriggered and the failing test passed. Some other intermittent issue came up again. This is good to merge Issue Time Tracking --- Worklog Id: (was: 857533) Time Spent: 50m (was: 40m) > Backport HIVE-27258 to branch-3 > --- > > Key: HIVE-27261 > URL: https://issues.apache.org/jira/browse/HIVE-27261 > Project: Hive > Issue Type: Improvement >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > Backport HIVE-27258 to branch-3 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-25289) Fix external_jdbc_table3 and external_jdbc_table4
[ https://issues.apache.org/jira/browse/HIVE-25289?focusedWorklogId=857524&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857524 ] ASF GitHub Bot logged work on HIVE-25289: - Author: ASF GitHub Bot Created on: 18/Apr/23 06:05 Start Date: 18/Apr/23 06:05 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4242: URL: https://github.com/apache/hive/pull/4242#issuecomment-1512488130 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4242) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4242&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4242&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4242&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4242&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4242&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4242&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4242&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4242&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4242&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4242&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4242&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4242&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4242&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4242&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 857524) Time Spent: 0.5h (was: 20m) > Fix external_jdbc_table3 and external_jdbc_table4 > - > > Key: HIVE-25289 > URL: https://issues.apache.org/jira/browse/HIVE-25289 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > http://ci.hive.apache.org/job/hive-flaky-check/265/ > http://ci.hive.apache.org/job/hive-flaky-check/266/ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-23682) TestMetrics is flaky
[ https://issues.apache.org/jira/browse/HIVE-23682?focusedWorklogId=857521&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857521 ] ASF GitHub Bot logged work on HIVE-23682: - Author: ASF GitHub Bot Created on: 18/Apr/23 05:59 Start Date: 18/Apr/23 05:59 Worklog Time Spent: 10m Work Description: rkirtir commented on PR #4243: URL: https://github.com/apache/hive/pull/4243#issuecomment-1512481627 Flakey test jenkins job is successful http://ci.hive.apache.org/job/hive-flaky-check/656/console Issue Time Tracking --- Worklog Id: (was: 857521) Time Spent: 20m (was: 10m) > TestMetrics is flaky > > > Key: HIVE-23682 > URL: https://issues.apache.org/jira/browse/HIVE-23682 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > http://34.66.156.144:8080/job/hive-precommit/job/master/31/testReport/ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-23682) TestMetrics is flaky
[ https://issues.apache.org/jira/browse/HIVE-23682?focusedWorklogId=857520&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857520 ] ASF GitHub Bot logged work on HIVE-23682: - Author: ASF GitHub Bot Created on: 18/Apr/23 05:59 Start Date: 18/Apr/23 05:59 Worklog Time Spent: 10m Work Description: rkirtir opened a new pull request, #4243: URL: https://github.com/apache/hive/pull/4243 ### What changes were proposed in this pull request? HIVE-23682 ### Why are the changes needed? To enable test in testsuite ### Does this PR introduce _any_ user-facing change? NO ### How was this patch tested? testcse Issue Time Tracking --- Worklog Id: (was: 857520) Remaining Estimate: 0h Time Spent: 10m > TestMetrics is flaky > > > Key: HIVE-23682 > URL: https://issues.apache.org/jira/browse/HIVE-23682 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > http://34.66.156.144:8080/job/hive-precommit/job/master/31/testReport/ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-23682) TestMetrics is flaky
[ https://issues.apache.org/jira/browse/HIVE-23682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-23682: -- Labels: pull-request-available (was: ) > TestMetrics is flaky > > > Key: HIVE-23682 > URL: https://issues.apache.org/jira/browse/HIVE-23682 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > http://34.66.156.144:8080/job/hive-precommit/job/master/31/testReport/ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-25289) Fix external_jdbc_table3 and external_jdbc_table4
[ https://issues.apache.org/jira/browse/HIVE-25289?focusedWorklogId=857516&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857516 ] ASF GitHub Bot logged work on HIVE-25289: - Author: ASF GitHub Bot Created on: 18/Apr/23 05:02 Start Date: 18/Apr/23 05:02 Worklog Time Spent: 10m Work Description: rkirtir commented on PR #4242: URL: https://github.com/apache/hive/pull/4242#issuecomment-1512437299 Flakey test for external_jdbc_table3.q is successful http://ci.hive.apache.org/job/hive-flaky-check/654/ Flakey test for external_jdbc_table4.q is successful http://ci.hive.apache.org/job/hive-flaky-check/652/ Issue Time Tracking --- Worklog Id: (was: 857516) Time Spent: 20m (was: 10m) > Fix external_jdbc_table3 and external_jdbc_table4 > - > > Key: HIVE-25289 > URL: https://issues.apache.org/jira/browse/HIVE-25289 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > http://ci.hive.apache.org/job/hive-flaky-check/265/ > http://ci.hive.apache.org/job/hive-flaky-check/266/ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-25289) Fix external_jdbc_table3 and external_jdbc_table4
[ https://issues.apache.org/jira/browse/HIVE-25289?focusedWorklogId=857515&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857515 ] ASF GitHub Bot logged work on HIVE-25289: - Author: ASF GitHub Bot Created on: 18/Apr/23 05:01 Start Date: 18/Apr/23 05:01 Worklog Time Spent: 10m Work Description: rkirtir opened a new pull request, #4242: URL: https://github.com/apache/hive/pull/4242 ### What changes were proposed in this pull request? HIVE-25289 ### Why are the changes needed? To enable tests in test suite ### Does this PR introduce _any_ user-facing change? NO ### How was this patch tested? with test case Issue Time Tracking --- Worklog Id: (was: 857515) Remaining Estimate: 0h Time Spent: 10m > Fix external_jdbc_table3 and external_jdbc_table4 > - > > Key: HIVE-25289 > URL: https://issues.apache.org/jira/browse/HIVE-25289 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > http://ci.hive.apache.org/job/hive-flaky-check/265/ > http://ci.hive.apache.org/job/hive-flaky-check/266/ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-25289) Fix external_jdbc_table3 and external_jdbc_table4
[ https://issues.apache.org/jira/browse/HIVE-25289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-25289: -- Labels: pull-request-available (was: ) > Fix external_jdbc_table3 and external_jdbc_table4 > - > > Key: HIVE-25289 > URL: https://issues.apache.org/jira/browse/HIVE-25289 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > http://ci.hive.apache.org/job/hive-flaky-check/265/ > http://ci.hive.apache.org/job/hive-flaky-check/266/ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27268) Hive.getPartitionsByNames should not enforce SessionState to be available
[ https://issues.apache.org/jira/browse/HIVE-27268?focusedWorklogId=857514&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857514 ] ASF GitHub Bot logged work on HIVE-27268: - Author: ASF GitHub Bot Created on: 18/Apr/23 04:01 Start Date: 18/Apr/23 04:01 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4241: URL: https://github.com/apache/hive/pull/4241#issuecomment-1512400758 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4241) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4241&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4241&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4241&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4241&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4241&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4241&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4241&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4241&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4241&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4241&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4241&resolved=false&types=CODE_SMELL) [1 Code Smell](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4241&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4241&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4241&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 857514) Time Spent: 20m (was: 10m) > Hive.getPartitionsByNames should not enforce SessionState to be available > - > > Key: HIVE-27268 > URL: https://issues.apache.org/jira/browse/HIVE-27268 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.1.3 >Reporter: Henri Biestro >Priority: Minor > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > HIVE-24743, HIVE-24392 is enforcing to check for valid write Id list for > "Hive.getPartitionsByName". > This breaks basic API integration. For a user who needs to get basic > partition detail, he is forced to have SessionState. > Request in this ticket is to ensure that if SessionState.get() is null, it > should return empty validWriteIdList. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27261) Backport HIVE-27258 to branch-3
[ https://issues.apache.org/jira/browse/HIVE-27261?focusedWorklogId=857512&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857512 ] ASF GitHub Bot logged work on HIVE-27261: - Author: ASF GitHub Bot Created on: 18/Apr/23 03:10 Start Date: 18/Apr/23 03:10 Worklog Time Spent: 10m Work Description: amanraj2520 commented on PR #4234: URL: https://github.com/apache/hive/pull/4234#issuecomment-1512371871 @vihangk1 I think we can disable the test. Looks like it fails because of some thrift connection issue. If this passes in the local it should be fine according to me. Issue Time Tracking --- Worklog Id: (was: 857512) Time Spent: 40m (was: 0.5h) > Backport HIVE-27258 to branch-3 > --- > > Key: HIVE-27261 > URL: https://issues.apache.org/jira/browse/HIVE-27261 > Project: Hive > Issue Type: Improvement >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > Backport HIVE-27258 to branch-3 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27163) Column stats are not getting published after an insert query into an external table with custom location
[
https://issues.apache.org/jira/browse/HIVE-27163?focusedWorklogId=857510&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857510
]
ASF GitHub Bot logged work on HIVE-27163:
-
Author: ASF GitHub Bot
Created on: 18/Apr/23 02:51
Start Date: 18/Apr/23 02:51
Worklog Time Spent: 10m
Work Description: sonarcloud[bot] commented on PR #4228:
URL: https://github.com/apache/hive/pull/4228#issuecomment-1512359216
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4228)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4228&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4228&resolved=false&types=BUG)
[0
Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4228&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4228&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4228&resolved=false&types=VULNERABILITY)
[0
Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4228&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4228&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4228&resolved=false&types=SECURITY_HOTSPOT)
[0 Security
Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4228&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4228&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4228&resolved=false&types=CODE_SMELL)
[7 Code
Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4228&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4228&metric=coverage&view=list)
No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4228&metric=duplicated_lines_density&view=list)
No Duplication information
Issue Time Tracking
---
Worklog Id: (was: 857510)
Time Spent: 40m (was: 0.5h)
> Column stats are not getting published after an insert query into an external
> table with custom location
>
>
> Key: HIVE-27163
> URL: https://issues.apache.org/jira/browse/HIVE-27163
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Taraka Rama Rao Lethavadla
>Assignee: Zhihua Deng
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Test case details are below
> *test.q*
> {noformat}
> set hive.stats.column.autogather=true;
> set hive.stats.autogather=true;
> dfs ${system:test.dfs.mkdir} ${system:test.tmp.dir}/test;
> create external table test_custom(age int, name string) stored as orc
> location '/tmp/test';
> insert into test_custom select 1, 'test';
> desc formatted test_custom age;{noformat}
> *test.q.out*
>
>
> {noformat}
> A masked pattern was here
> PREHOOK: type: CREATETABLE
[jira] [Work logged] (HIVE-27268) Hive.getPartitionsByNames should not enforce SessionState to be available
[ https://issues.apache.org/jira/browse/HIVE-27268?focusedWorklogId=857506&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857506 ] ASF GitHub Bot logged work on HIVE-27268: - Author: ASF GitHub Bot Created on: 18/Apr/23 02:07 Start Date: 18/Apr/23 02:07 Worklog Time Spent: 10m Work Description: henrib opened a new pull request, #4241: URL: https://github.com/apache/hive/pull/4241 ### What changes were proposed in this pull request? When a tool runs without a session, create an empty id list. ### Why are the changes needed? To avoid a potential NPE and be consistent. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Very hard to produce through unit tests (need to fail session creation). Issue Time Tracking --- Worklog Id: (was: 857506) Remaining Estimate: 0h Time Spent: 10m > Hive.getPartitionsByNames should not enforce SessionState to be available > - > > Key: HIVE-27268 > URL: https://issues.apache.org/jira/browse/HIVE-27268 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.1.3 >Reporter: Henri Biestro >Priority: Minor > Fix For: 4.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > HIVE-24743, HIVE-24392 is enforcing to check for valid write Id list for > "Hive.getPartitionsByName". > This breaks basic API integration. For a user who needs to get basic > partition detail, he is forced to have SessionState. > Request in this ticket is to ensure that if SessionState.get() is null, it > should return empty validWriteIdList. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27268) Hive.getPartitionsByNames should not enforce SessionState to be available
[ https://issues.apache.org/jira/browse/HIVE-27268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-27268: -- Labels: pull-request-available (was: ) > Hive.getPartitionsByNames should not enforce SessionState to be available > - > > Key: HIVE-27268 > URL: https://issues.apache.org/jira/browse/HIVE-27268 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.1.3 >Reporter: Henri Biestro >Priority: Minor > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > HIVE-24743, HIVE-24392 is enforcing to check for valid write Id list for > "Hive.getPartitionsByName". > This breaks basic API integration. For a user who needs to get basic > partition detail, he is forced to have SessionState. > Request in this ticket is to ensure that if SessionState.get() is null, it > should return empty validWriteIdList. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26400) Provide docker images for Hive
[ https://issues.apache.org/jira/browse/HIVE-26400?focusedWorklogId=857503&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857503 ] ASF GitHub Bot logged work on HIVE-26400: - Author: ASF GitHub Bot Created on: 18/Apr/23 01:40 Start Date: 18/Apr/23 01:40 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4240: URL: https://github.com/apache/hive/pull/4240#issuecomment-1512315534 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4240) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4240&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4240&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4240&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4240&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4240&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4240&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4240&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4240&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4240&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4240&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4240&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4240&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4240&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4240&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 857503) Time Spent: 14h 50m (was: 14h 40m) > Provide docker images for Hive > -- > > Key: HIVE-26400 > URL: https://issues.apache.org/jira/browse/HIVE-26400 > Project: Hive > Issue Type: Sub-task > Components: Build Infrastructure >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Blocker > Labels: hive-4.0.0-must, pull-request-available > Time Spent: 14h 50m > Remaining Estimate: 0h > > Make Apache Hive be able to run inside docker container in pseudo-distributed > mode, with MySQL/Derby as its back database, provide the following: > * Quick-start/Debugging/Prepare a test env for Hive; > * Tools to build target image with specified version of Hive and its > dependencies; > * Images can be used as the basis for the Kubernetes operator. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27048) Backport HIVE-25313: Upgrade commons-codec to 1.15 in branch-3
[ https://issues.apache.org/jira/browse/HIVE-27048?focusedWorklogId=857499&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857499 ] ASF GitHub Bot logged work on HIVE-27048: - Author: ASF GitHub Bot Created on: 18/Apr/23 00:18 Start Date: 18/Apr/23 00:18 Worklog Time Spent: 10m Work Description: github-actions[bot] closed pull request #4031: HIVE-27048: Backport HIVE-25313: Upgrade commons-codec to 1.15 in branch-3 URL: https://github.com/apache/hive/pull/4031 Issue Time Tracking --- Worklog Id: (was: 857499) Time Spent: 0.5h (was: 20m) > Backport HIVE-25313: Upgrade commons-codec to 1.15 in branch-3 > -- > > Key: HIVE-27048 > URL: https://issues.apache.org/jira/browse/HIVE-27048 > Project: Hive > Issue Type: Improvement >Reporter: Raghav Aggarwal >Assignee: Raghav Aggarwal >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-27186) A persistent property store
[ https://issues.apache.org/jira/browse/HIVE-27186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sai Hemanth Gantasala reassigned HIVE-27186: Assignee: Henri Biestro > A persistent property store > > > Key: HIVE-27186 > URL: https://issues.apache.org/jira/browse/HIVE-27186 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 4.0.0-alpha-2 >Reporter: Henri Biestro >Assignee: Henri Biestro >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > WHAT > A persistent property store usable as a support facility for any metadata > augmentation feature. > WHY > When adding new meta-data oriented features, we usually need to persist > information linking the feature data and the HiveMetaStore objects it applies > to. Any information related to a database, a table or the cluster - like > statistics for example or any operational data state or data (think rolling > backup) - fall in this use-case. > Typically, accommodating such a feature requires modifying the Metastore > database schema by adding or altering a table. It also usually implies > modifying the thrift APIs to expose such meta-data to consumers. > The proposed feature wants to solve the persistence and query/transport for > these types of use-cases by exposing a 'key/(meta)value' store exposed as a > property system. > HOW > A property-value model is the simple and generic exposed API. > To provision for several usage scenarios, the model entry point is a > 'namespace' that qualifies the feature-component property manager. For > example, 'stats' could be the namespace for all properties related to the > 'statistics' feature. > The namespace identifies a manager that handles property-groups persisted as > property-maps. For instance, all statistics pertaining to a given table would > be collocated in the same property-group. As such, all properties (say number > of 'unique_values' per columns) for a given HMS table 'relation0' would all > be stored and persisted in the same property-map instance. > Property-maps may be decorated by an (optional) schema that may declare the > name and value-type of allowed properties (and their optional default value). > Each property is addressed by a name, a path uniquely identifying the > property in a given property map. > The manager also handles transforming property-map names to the property-map > keys used to persist them in the DB. > The API provides inserting/updating properties in bulk transactionally. It > also provides selection/projection to help reduce the volume of exchange > between client/server; selection can use (JEXL expression) predicates to > filter maps. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27032) Introduce liquibase for HMS schema evolution
[ https://issues.apache.org/jira/browse/HIVE-27032?focusedWorklogId=857490&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857490 ] ASF GitHub Bot logged work on HIVE-27032: - Author: ASF GitHub Bot Created on: 17/Apr/23 22:18 Start Date: 17/Apr/23 22:18 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4060: URL: https://github.com/apache/hive/pull/4060#issuecomment-1512160008 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4060) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4060&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4060&resolved=false&types=SECURITY_HOTSPOT) [4 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4060&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=CODE_SMELL) [206 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4060&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4060&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 857490) Time Spent: 2h 40m (was: 2.5h) > Introduce liquibase for HMS schema evolution > > > Key: HIVE-27032 > URL: https://issues.apache.org/jira/browse/HIVE-27032 > Project: Hive > Issue Type: Improvement >Reporter: László Végh >Assignee: László Végh >Priority: Major > Labels: pull-request-available > Time Spent: 2h 40m > Remaining Estimate: 0h > > Introduce liquibase, and replace current upgrade procedure with it. > The Schematool CLI API should remain untouched, while under the hood, > liquibase should be used for HMS schema evolution. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27261) Backport HIVE-27258 to branch-3
[ https://issues.apache.org/jira/browse/HIVE-27261?focusedWorklogId=857481&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857481 ] ASF GitHub Bot logged work on HIVE-27261: - Author: ASF GitHub Bot Created on: 17/Apr/23 21:24 Start Date: 17/Apr/23 21:24 Worklog Time Spent: 10m Work Description: vihangk1 commented on PR #4234: URL: https://github.com/apache/hive/pull/4234#issuecomment-1512104985 > [INFO] Running org.apache.hadoop.hive.cli.split5.TestSparkCliDriver [INFO] Tests run: 58, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 881.699 s - in org.apache.hadoop.hive.cli.split5.TestSparkCliDriver Issue Time Tracking --- Worklog Id: (was: 857481) Time Spent: 0.5h (was: 20m) > Backport HIVE-27258 to branch-3 > --- > > Key: HIVE-27261 > URL: https://issues.apache.org/jira/browse/HIVE-27261 > Project: Hive > Issue Type: Improvement >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Backport HIVE-27258 to branch-3 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27261) Backport HIVE-27258 to branch-3
[ https://issues.apache.org/jira/browse/HIVE-27261?focusedWorklogId=857480&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857480 ] ASF GitHub Bot logged work on HIVE-27261: - Author: ASF GitHub Bot Created on: 17/Apr/23 21:23 Start Date: 17/Apr/23 21:23 Worklog Time Spent: 10m Work Description: vihangk1 commented on PR #4234: URL: https://github.com/apache/hive/pull/4234#issuecomment-1512104790 The test which is failing doesn't fail for me on my local when I run the split. Also, looks totally unrelated. I think we should disable it for now and investigate it separately. @amanraj2520 What do you think? Issue Time Tracking --- Worklog Id: (was: 857480) Time Spent: 20m (was: 10m) > Backport HIVE-27258 to branch-3 > --- > > Key: HIVE-27261 > URL: https://issues.apache.org/jira/browse/HIVE-27261 > Project: Hive > Issue Type: Improvement >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Backport HIVE-27258 to branch-3 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-27268) Hive.getPartitionsByNames should not enforce SessionState to be available
Henri Biestro created HIVE-27268: Summary: Hive.getPartitionsByNames should not enforce SessionState to be available Key: HIVE-27268 URL: https://issues.apache.org/jira/browse/HIVE-27268 Project: Hive Issue Type: Bug Components: Hive Affects Versions: 3.1.3 Reporter: Henri Biestro Fix For: 4.0.0 HIVE-24743, HIVE-24392 is enforcing to check for valid write Id list for "Hive.getPartitionsByName". This breaks basic API integration. For a user who needs to get basic partition detail, he is forced to have SessionState. Request in this ticket is to ensure that if SessionState.get() is null, it should return empty validWriteIdList. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27032) Introduce liquibase for HMS schema evolution
[ https://issues.apache.org/jira/browse/HIVE-27032?focusedWorklogId=857436&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857436 ] ASF GitHub Bot logged work on HIVE-27032: - Author: ASF GitHub Bot Created on: 17/Apr/23 17:39 Start Date: 17/Apr/23 17:39 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4060: URL: https://github.com/apache/hive/pull/4060#issuecomment-1511805233 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4060) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4060&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4060&resolved=false&types=SECURITY_HOTSPOT) [4 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4060&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=CODE_SMELL) [206 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4060&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4060&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 857436) Time Spent: 2.5h (was: 2h 20m) > Introduce liquibase for HMS schema evolution > > > Key: HIVE-27032 > URL: https://issues.apache.org/jira/browse/HIVE-27032 > Project: Hive > Issue Type: Improvement >Reporter: László Végh >Assignee: László Végh >Priority: Major > Labels: pull-request-available > Time Spent: 2.5h > Remaining Estimate: 0h > > Introduce liquibase, and replace current upgrade procedure with it. > The Schematool CLI API should remain untouched, while under the hood, > liquibase should be used for HMS schema evolution. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27264) Literals in conjunction of two in expression are considered not equals if type precision is different
[
https://issues.apache.org/jira/browse/HIVE-27264?focusedWorklogId=857418&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857418
]
ASF GitHub Bot logged work on HIVE-27264:
-
Author: ASF GitHub Bot
Created on: 17/Apr/23 16:27
Start Date: 17/Apr/23 16:27
Worklog Time Spent: 10m
Work Description: sonarcloud[bot] commented on PR #4237:
URL: https://github.com/apache/hive/pull/4237#issuecomment-1511702198
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4237)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4237&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4237&resolved=false&types=BUG)
[0
Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4237&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4237&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4237&resolved=false&types=VULNERABILITY)
[0
Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4237&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4237&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4237&resolved=false&types=SECURITY_HOTSPOT)
[0 Security
Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4237&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4237&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4237&resolved=false&types=CODE_SMELL)
[0 Code
Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4237&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4237&metric=coverage&view=list)
No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4237&metric=duplicated_lines_density&view=list)
No Duplication information
Issue Time Tracking
---
Worklog Id: (was: 857418)
Time Spent: 1h 20m (was: 1h 10m)
> Literals in conjunction of two in expression are considered not equals if
> type precision is different
> -
>
> Key: HIVE-27264
> URL: https://issues.apache.org/jira/browse/HIVE-27264
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> {code}
> create table r_table (
> string_col varchar(30)
> );
> create table l_table (
> string_col varchar(14)
> );
> insert into r_table VALUES ('AAA111');
> insert into l_table VALUES ('AAA111');
> SELECT l_table.string_col from l_table, r_table
> WHERE r_table.string_col = l_table.string_col AND l_table.string_col IN
> ('AAA111', 'BBB222') AND r_table.string_col IN ('AAA111', 'BBB222');
> {code}
> Should give one row
> {code}
> AAA111
> {code}
> but it returns empty rs
[jira] [Work logged] (HIVE-26400) Provide docker images for Hive
[ https://issues.apache.org/jira/browse/HIVE-26400?focusedWorklogId=857414&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857414 ] ASF GitHub Bot logged work on HIVE-26400: - Author: ASF GitHub Bot Created on: 17/Apr/23 16:17 Start Date: 17/Apr/23 16:17 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4240: URL: https://github.com/apache/hive/pull/4240#issuecomment-1511689238 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4240) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4240&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4240&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4240&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4240&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4240&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4240&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4240&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4240&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4240&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4240&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4240&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4240&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4240&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4240&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 857414) Time Spent: 14h 40m (was: 14.5h) > Provide docker images for Hive > -- > > Key: HIVE-26400 > URL: https://issues.apache.org/jira/browse/HIVE-26400 > Project: Hive > Issue Type: Sub-task > Components: Build Infrastructure >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Blocker > Labels: hive-4.0.0-must, pull-request-available > Time Spent: 14h 40m > Remaining Estimate: 0h > > Make Apache Hive be able to run inside docker container in pseudo-distributed > mode, with MySQL/Derby as its back database, provide the following: > * Quick-start/Debugging/Prepare a test env for Hive; > * Tools to build target image with specified version of Hive and its > dependencies; > * Images can be used as the basis for the Kubernetes operator. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27264) Literals in conjunction of two in expression are considered not equals if type precision is different
[
https://issues.apache.org/jira/browse/HIVE-27264?focusedWorklogId=857413&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857413
]
ASF GitHub Bot logged work on HIVE-27264:
-
Author: ASF GitHub Bot
Created on: 17/Apr/23 16:16
Start Date: 17/Apr/23 16:16
Worklog Time Spent: 10m
Work Description: amansinha100 commented on code in PR #4237:
URL: https://github.com/apache/hive/pull/4237#discussion_r1168973804
##
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HivePointLookupOptimizerRule.java:
##
@@ -669,6 +670,22 @@ private static RexNode handleAND(RexBuilder rexBuilder,
RexCall call) {
return RexUtil.composeConjunction(rexBuilder, newOperands, false);
}
+private static void retainAll(Collection elementsToRetain,
Collection collection) {
+ collection.removeIf(rexNode -> elementsToRetain.stream().noneMatch(
+ rexNodeToRetain -> equalsWithSimilarType(rexNode,
rexNodeToRetain)));
+}
+
+private static boolean equalsWithSimilarType(RexNode rexNode1, RexNode
rexNode2) {
+ if (!(rexNode1 instanceof RexLiteral) || !(rexNode2 instanceof
RexLiteral)) {
+return rexNode1.equals(rexNode2);
+ }
+
+ RexLiteral rexLiteral1 = (RexLiteral) rexNode1;
+ RexLiteral rexLiteral2 = (RexLiteral) rexNode2;
+ return rexLiteral1.getValue().compareTo(rexLiteral2.getValue()) == 0 &&
+
rexLiteral1.getType().getSqlTypeName().equals(rexLiteral2.getType().getSqlTypeName());
Review Comment:
I wasn't worried about performance..just the fact that there was redundant
check. But yeah I understand the reason for having the check also within
equalsWithSimilarType. Ok to leave it as-is.
Issue Time Tracking
---
Worklog Id: (was: 857413)
Time Spent: 1h 10m (was: 1h)
> Literals in conjunction of two in expression are considered not equals if
> type precision is different
> -
>
> Key: HIVE-27264
> URL: https://issues.apache.org/jira/browse/HIVE-27264
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> {code}
> create table r_table (
> string_col varchar(30)
> );
> create table l_table (
> string_col varchar(14)
> );
> insert into r_table VALUES ('AAA111');
> insert into l_table VALUES ('AAA111');
> SELECT l_table.string_col from l_table, r_table
> WHERE r_table.string_col = l_table.string_col AND l_table.string_col IN
> ('AAA111', 'BBB222') AND r_table.string_col IN ('AAA111', 'BBB222');
> {code}
> Should give one row
> {code}
> AAA111
> {code}
> but it returns empty rs
> Workaround
> {code}
> set hive.optimize.point.lookup=false;
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26400) Provide docker images for Hive
[ https://issues.apache.org/jira/browse/HIVE-26400?focusedWorklogId=857386&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857386 ] ASF GitHub Bot logged work on HIVE-26400: - Author: ASF GitHub Bot Created on: 17/Apr/23 14:00 Start Date: 17/Apr/23 14:00 Worklog Time Spent: 10m Work Description: TuroczyX commented on PR #3448: URL: https://github.com/apache/hive/pull/3448#issuecomment-1511410250 Excellent! Thank you all of your effort @dengzhhu653! Issue Time Tracking --- Worklog Id: (was: 857386) Time Spent: 14.5h (was: 14h 20m) > Provide docker images for Hive > -- > > Key: HIVE-26400 > URL: https://issues.apache.org/jira/browse/HIVE-26400 > Project: Hive > Issue Type: Sub-task > Components: Build Infrastructure >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Blocker > Labels: hive-4.0.0-must, pull-request-available > Time Spent: 14.5h > Remaining Estimate: 0h > > Make Apache Hive be able to run inside docker container in pseudo-distributed > mode, with MySQL/Derby as its back database, provide the following: > * Quick-start/Debugging/Prepare a test env for Hive; > * Tools to build target image with specified version of Hive and its > dependencies; > * Images can be used as the basis for the Kubernetes operator. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27020) Implement a separate handler to handle aborted transaction cleanup
[ https://issues.apache.org/jira/browse/HIVE-27020?focusedWorklogId=857385&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857385 ] ASF GitHub Bot logged work on HIVE-27020: - Author: ASF GitHub Bot Created on: 17/Apr/23 13:57 Start Date: 17/Apr/23 13:57 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4091: URL: https://github.com/apache/hive/pull/4091#issuecomment-1511405791 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4091) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4091&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4091&resolved=false&types=BUG) [1 Bug](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4091&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4091&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4091&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4091&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4091&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4091&resolved=false&types=SECURITY_HOTSPOT) [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4091&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4091&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4091&resolved=false&types=CODE_SMELL) [10 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4091&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4091&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4091&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 857385) Time Spent: 14.5h (was: 14h 20m) > Implement a separate handler to handle aborted transaction cleanup > -- > > Key: HIVE-27020 > URL: https://issues.apache.org/jira/browse/HIVE-27020 > Project: Hive > Issue Type: Sub-task >Reporter: Sourabh Badhya >Assignee: Sourabh Badhya >Priority: Major > Labels: pull-request-available > Time Spent: 14.5h > Remaining Estimate: 0h > > As described in the parent task, once the cleaner is separated into different > entities, implement a separate handler which can create requests for aborted > transactions cleanup. This would move the aborted transaction cleanup > exclusively to the cleaner. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27032) Introduce liquibase for HMS schema evolution
[ https://issues.apache.org/jira/browse/HIVE-27032?focusedWorklogId=857379&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857379 ] ASF GitHub Bot logged work on HIVE-27032: - Author: ASF GitHub Bot Created on: 17/Apr/23 13:35 Start Date: 17/Apr/23 13:35 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4060: URL: https://github.com/apache/hive/pull/4060#issuecomment-1511361599 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4060) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4060&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4060&resolved=false&types=SECURITY_HOTSPOT) [4 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4060&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=CODE_SMELL) [202 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4060&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4060&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 857379) Time Spent: 2h 20m (was: 2h 10m) > Introduce liquibase for HMS schema evolution > > > Key: HIVE-27032 > URL: https://issues.apache.org/jira/browse/HIVE-27032 > Project: Hive > Issue Type: Improvement >Reporter: László Végh >Assignee: László Végh >Priority: Major > Labels: pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > Introduce liquibase, and replace current upgrade procedure with it. > The Schematool CLI API should remain untouched, while under the hood, > liquibase should be used for HMS schema evolution. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26400) Provide docker images for Hive
[ https://issues.apache.org/jira/browse/HIVE-26400?focusedWorklogId=857377&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857377 ] ASF GitHub Bot logged work on HIVE-26400: - Author: ASF GitHub Bot Created on: 17/Apr/23 13:32 Start Date: 17/Apr/23 13:32 Worklog Time Spent: 10m Work Description: dengzhhu653 commented on PR #3448: URL: https://github.com/apache/hive/pull/3448#issuecomment-1511357356 > I think it would be a good idea to add ports for development / debugging like this ones: -p9866:9866 -p1:1 -p10001:10001 -p9000:9000 -p8000:8000 -p3306:3306 -p50070:50070 -p50030:50030 For debug purpose, you can launch the container with: ```shell docker run -d -p 9083:9083 -p 8009:8009 --env SERVICE_OPTS="-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8009" --env SERVICE_NAME=metastore --name metastore-standalone apache/hive:4.0.0-SNAPSHOT ``` SERVICE_OPTS will finally propagate to the JVM args of the service. I opened a new PR for testing it out, https://github.com/apache/hive/pull/4240 cc @TuroczyX @deniskuzZ. Issue Time Tracking --- Worklog Id: (was: 857377) Time Spent: 14h 20m (was: 14h 10m) > Provide docker images for Hive > -- > > Key: HIVE-26400 > URL: https://issues.apache.org/jira/browse/HIVE-26400 > Project: Hive > Issue Type: Sub-task > Components: Build Infrastructure >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Blocker > Labels: hive-4.0.0-must, pull-request-available > Time Spent: 14h 20m > Remaining Estimate: 0h > > Make Apache Hive be able to run inside docker container in pseudo-distributed > mode, with MySQL/Derby as its back database, provide the following: > * Quick-start/Debugging/Prepare a test env for Hive; > * Tools to build target image with specified version of Hive and its > dependencies; > * Images can be used as the basis for the Kubernetes operator. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26400) Provide docker images for Hive
[ https://issues.apache.org/jira/browse/HIVE-26400?focusedWorklogId=857376&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857376 ] ASF GitHub Bot logged work on HIVE-26400: - Author: ASF GitHub Bot Created on: 17/Apr/23 13:26 Start Date: 17/Apr/23 13:26 Worklog Time Spent: 10m Work Description: dengzhhu653 opened a new pull request, #4240: URL: https://github.com/apache/hive/pull/4240 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? Issue Time Tracking --- Worklog Id: (was: 857376) Time Spent: 14h 10m (was: 14h) > Provide docker images for Hive > -- > > Key: HIVE-26400 > URL: https://issues.apache.org/jira/browse/HIVE-26400 > Project: Hive > Issue Type: Sub-task > Components: Build Infrastructure >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Blocker > Labels: hive-4.0.0-must, pull-request-available > Time Spent: 14h 10m > Remaining Estimate: 0h > > Make Apache Hive be able to run inside docker container in pseudo-distributed > mode, with MySQL/Derby as its back database, provide the following: > * Quick-start/Debugging/Prepare a test env for Hive; > * Tools to build target image with specified version of Hive and its > dependencies; > * Images can be used as the basis for the Kubernetes operator. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27199) Read TIMESTAMP WITH LOCAL TIME ZONE columns from text files using custom formats
[
https://issues.apache.org/jira/browse/HIVE-27199?focusedWorklogId=857367&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857367
]
ASF GitHub Bot logged work on HIVE-27199:
-
Author: ASF GitHub Bot
Created on: 17/Apr/23 12:58
Start Date: 17/Apr/23 12:58
Worklog Time Spent: 10m
Work Description: TuroczyX commented on code in PR #4170:
URL: https://github.com/apache/hive/pull/4170#discussion_r1168652988
##
common/src/java/org/apache/hive/common/util/TimestampParser.java:
##
@@ -199,6 +205,19 @@ public Timestamp parseTimestamp(final String text) {
}
+ public TimestampTZ parseTimestamp(String text, ZoneId defaultTimeZone) {
+Objects.requireNonNull(text);
+for (DateTimeFormatter f : dtFormatters) {
+ try {
+return TimestampTZUtil.parse(text, defaultTimeZone, f);
+ } catch (DateTimeException e) {
Review Comment:
Oh I See. Thanks for the clarification.
Issue Time Tracking
---
Worklog Id: (was: 857367)
Time Spent: 1.5h (was: 1h 20m)
> Read TIMESTAMP WITH LOCAL TIME ZONE columns from text files using custom
> formats
>
>
> Key: HIVE-27199
> URL: https://issues.apache.org/jira/browse/HIVE-27199
> Project: Hive
> Issue Type: Improvement
> Components: Serializers/Deserializers
>Affects Versions: 4.0.0-alpha-2
>Reporter: Stamatis Zampetakis
>Assignee: Stamatis Zampetakis
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Timestamp values come in many flavors and formats and there is no single
> representation that can satisfy everyone especially when such values are
> stored in plain text/csv files.
> HIVE-9298, added a special SERDE property, {{{}timestamp.formats{}}}, that
> allows to provide custom timestamp patterns to parse correctly TIMESTAMP
> values coming from files.
> However, when the column type is TIMESTAMP WITH LOCAL TIME ZONE (LTZ) it is
> not possible to use a custom pattern thus when the built-in Hive parser does
> not match the expected format a NULL value is returned.
> Consider a text file, F1, with the following values:
> {noformat}
> 2016-05-03 12:26:34
> 2016-05-03T12:26:34
> {noformat}
> and a table with a column declared as LTZ.
> {code:sql}
> CREATE TABLE ts_table (ts TIMESTAMP WITH LOCAL TIME ZONE);
> LOAD DATA LOCAL INPATH './F1' INTO TABLE ts_table;
> SELECT * FROM ts_table;
> 2016-05-03 12:26:34.0 US/Pacific
> NULL
> {code}
> In order to give more flexibility to the users relying on the TIMESTAMP WITH
> LOCAL TIME ZONE datatype and also align the behavior with the TIMESTAMP type
> this JIRA aims to reuse the {{timestamp.formats}} property for both TIMESTAMP
> types.
> The work here focuses exclusively on simple text files but the same could be
> done for other SERDE such as JSON etc.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27199) Read TIMESTAMP WITH LOCAL TIME ZONE columns from text files using custom formats
[
https://issues.apache.org/jira/browse/HIVE-27199?focusedWorklogId=857366&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857366
]
ASF GitHub Bot logged work on HIVE-27199:
-
Author: ASF GitHub Bot
Created on: 17/Apr/23 12:54
Start Date: 17/Apr/23 12:54
Worklog Time Spent: 10m
Work Description: zabetak commented on code in PR #4170:
URL: https://github.com/apache/hive/pull/4170#discussion_r1168643738
##
common/src/java/org/apache/hive/common/util/TimestampParser.java:
##
@@ -199,6 +205,19 @@ public Timestamp parseTimestamp(final String text) {
}
+ public TimestampTZ parseTimestamp(String text, ZoneId defaultTimeZone) {
+Objects.requireNonNull(text);
+for (DateTimeFormatter f : dtFormatters) {
+ try {
+return TimestampTZUtil.parse(text, defaultTimeZone, f);
+ } catch (DateTimeException e) {
Review Comment:
The `catch` is used to capture the `DateTimeException` and restrain/ignore
it. Without the `catch` the exception will propagate which is not what we want
here; the intention is to try all available parsers till we find one that can
parse the value or till we run out of options.
Issue Time Tracking
---
Worklog Id: (was: 857366)
Time Spent: 1h 20m (was: 1h 10m)
> Read TIMESTAMP WITH LOCAL TIME ZONE columns from text files using custom
> formats
>
>
> Key: HIVE-27199
> URL: https://issues.apache.org/jira/browse/HIVE-27199
> Project: Hive
> Issue Type: Improvement
> Components: Serializers/Deserializers
>Affects Versions: 4.0.0-alpha-2
>Reporter: Stamatis Zampetakis
>Assignee: Stamatis Zampetakis
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Timestamp values come in many flavors and formats and there is no single
> representation that can satisfy everyone especially when such values are
> stored in plain text/csv files.
> HIVE-9298, added a special SERDE property, {{{}timestamp.formats{}}}, that
> allows to provide custom timestamp patterns to parse correctly TIMESTAMP
> values coming from files.
> However, when the column type is TIMESTAMP WITH LOCAL TIME ZONE (LTZ) it is
> not possible to use a custom pattern thus when the built-in Hive parser does
> not match the expected format a NULL value is returned.
> Consider a text file, F1, with the following values:
> {noformat}
> 2016-05-03 12:26:34
> 2016-05-03T12:26:34
> {noformat}
> and a table with a column declared as LTZ.
> {code:sql}
> CREATE TABLE ts_table (ts TIMESTAMP WITH LOCAL TIME ZONE);
> LOAD DATA LOCAL INPATH './F1' INTO TABLE ts_table;
> SELECT * FROM ts_table;
> 2016-05-03 12:26:34.0 US/Pacific
> NULL
> {code}
> In order to give more flexibility to the users relying on the TIMESTAMP WITH
> LOCAL TIME ZONE datatype and also align the behavior with the TIMESTAMP type
> this JIRA aims to reuse the {{timestamp.formats}} property for both TIMESTAMP
> types.
> The work here focuses exclusively on simple text files but the same could be
> done for other SERDE such as JSON etc.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27082) AggregateStatsCache.findBestMatch() in Metastore should test the inclusion of default partition name
[ https://issues.apache.org/jira/browse/HIVE-27082?focusedWorklogId=857361&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857361 ] ASF GitHub Bot logged work on HIVE-27082: - Author: ASF GitHub Bot Created on: 17/Apr/23 12:34 Start Date: 17/Apr/23 12:34 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4239: URL: https://github.com/apache/hive/pull/4239#issuecomment-1511256265 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4239) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4239&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4239&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4239&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4239&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4239&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4239&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4239&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4239&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4239&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4239&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4239&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4239&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4239&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4239&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 857361) Time Spent: 20m (was: 10m) > AggregateStatsCache.findBestMatch() in Metastore should test the inclusion of > default partition name > > > Key: HIVE-27082 > URL: https://issues.apache.org/jira/browse/HIVE-27082 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Affects Versions: 3.1.3, 4.0.0-alpha-2 >Reporter: Sungwoo Park >Assignee: Seonggon Namgung >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > This JIRA deals with non-determinisitic behavior of Hive in generating DAGs. > The non-determinstic behavior of Hive in generating DAGs is due to the logic > in AggregateStatsCache.findBestMatch() called from AggregateStatsCache.get(), > as well as the disproportionate distribution of Nulls in > HIVE_DEFAULT_PARTITION. > Here is what is happening in the case of the TPC-DS dataset. Let us use > web_sales table and ws_w
[jira] [Updated] (HIVE-27267) Incorrect results when doing bucket map join on decimal bucketed column with subquery
[
https://issues.apache.org/jira/browse/HIVE-27267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sourabh Badhya updated HIVE-27267:
--
Description:
The following queries when run on a Hive cluster produce no results -
Repro queries -
{code:java}
set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
set hive.support.concurrency=true;
set hive.convert.join.bucket.mapjoin.tez=true;
drop table if exists test_external_source;
create external table test_external_source (date_col date, string_col string,
decimal_col decimal(38,0)) stored as orc tblproperties
('external.table.purge'='true');
insert into table test_external_source values ('2022-08-30', 'pipeline',
'5005905545593'), ('2022-08-16', 'pipeline',
'5005905545593'), ('2022-09-01', 'pipeline',
'5006008686831'), ('2022-08-30', 'pipeline',
'5005992620837'), ('2022-09-01', 'pipeline',
'5005992620837'), ('2022-09-01', 'pipeline',
'5005992621067'), ('2022-08-30', 'pipeline',
'5005992621067');
drop table if exists test_external_target;
create external table test_external_target (date_col date, string_col string,
decimal_col decimal(38,0)) stored as orc tblproperties
('external.table.purge'='true');
insert into table test_external_target values ('2017-05-17', 'pipeline',
'5000441610525'), ('2018-12-20', 'pipeline',
'5001048981030'), ('2020-06-30', 'pipeline',
'5002332575516'), ('2021-08-16', 'pipeline',
'5003897973989'), ('2017-06-06', 'pipeline',
'5000449148729'), ('2017-09-08', 'pipeline',
'5000525378314'), ('2022-08-30', 'pipeline',
'5005905545593'), ('2022-08-16', 'pipeline',
'5005905545593'), ('2018-05-03', 'pipeline',
'5000750826355'), ('2020-01-10', 'pipeline',
'5001816579677'), ('2021-11-01', 'pipeline',
'5004269423714'), ('2017-11-07', 'pipeline',
'5000585901787'), ('2019-10-15', 'pipeline',
'5001598843430'), ('2020-04-01', 'pipeline',
'5002035795461'), ('2020-02-24', 'pipeline',
'5001932600185'), ('2020-04-27', 'pipeline',
'5002108160849'), ('2016-07-05', 'pipeline',
'554405114'), ('2020-06-02', 'pipeline',
'5002234387967'), ('2020-08-21', 'pipeline',
'5002529168758'), ('2021-02-17', 'pipeline',
'5003158511687');
drop table if exists target_table;
drop table if exists source_table;
create table target_table(date_col date, string_col string, decimal_col
decimal(38,0)) clustered by (decimal_col) into 7 buckets stored as orc
tblproperties ('bucketing_version'='2', 'transactional'='true',
'transactional_properties'='default');
create table source_table(date_col date, string_col string, decimal_col
decimal(38,0)) clustered by (decimal_col) into 7 buckets stored as orc
tblproperties ('bucketing_version'='2', 'transactional'='true',
'transactional_properties'='default');
insert into table target_table select * from test_external_target;
insert into table source_table select * from test_external_source; {code}
Query which is under investigation -
{code:java}
select * from target_table inner join (select distinct date_col, 'pipeline'
string_col, decimal_col from source_table where coalesce(decimal_col,'') =
'5005905545593') s on s.date_col = target_table.date_col AND
s.string_col = target_table.string_col AND s.decimal_col =
target_table.decimal_col; {code}
Expected result of the query - 2 records
{code:java}
++--++-+---++
| target_table.date_col | target_table.string_col |
target_table.decimal_col | s.date_col | s.string_col |
s.decimal_col |
++--++-+---++
| 2022-08-16 | pipeline |
5005905545593 | 2022-08-16 | pipeline |
5005905545593 |
| 2022-08-30 | pipeline |
5005905545593 | 2022-08-30 | pipeline |
5005905545593 |
++--++-+---++
{code}
Actual result of the query - No records
{code:java}
++--+---+-+---++
| target_table.date_col | target_table.string_col | target_table.decimal_col
| s.date_col | s.string_co
[jira] [Created] (HIVE-27267) Incorrect results when doing bucket map join on decimal bucketed column with subquery
Sourabh Badhya created HIVE-27267:
-
Summary: Incorrect results when doing bucket map join on decimal
bucketed column with subquery
Key: HIVE-27267
URL: https://issues.apache.org/jira/browse/HIVE-27267
Project: Hive
Issue Type: Bug
Reporter: Sourabh Badhya
The following queries when run on a Hive cluster produce no results -
Repro queries -
{code:java}
set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
set hive.support.concurrency=true;
set hive.convert.join.bucket.mapjoin.tez=true;
drop table if exists test_external_source;
create external table test_external_source (date_col date, string_col string,
decimal_col decimal(38,0)) stored as orc tblproperties
('external.table.purge'='true');
insert into table test_external_source values ('2022-08-30', 'pipeline',
'5005905545593'), ('2022-08-16', 'pipeline',
'5005905545593'), ('2022-09-01', 'pipeline',
'5006008686831'), ('2022-08-30', 'pipeline',
'5005992620837'), ('2022-09-01', 'pipeline',
'5005992620837'), ('2022-09-01', 'pipeline',
'5005992621067'), ('2022-08-30', 'pipeline',
'5005992621067');
drop table if exists test_external_target;
create external table test_external_target (date_col date, string_col string,
decimal_col decimal(38,0)) stored as orc tblproperties
('external.table.purge'='true');
insert into table test_external_target values ('2017-05-17', 'pipeline',
'5000441610525'), ('2018-12-20', 'pipeline',
'5001048981030'), ('2020-06-30', 'pipeline',
'5002332575516'), ('2021-08-16', 'pipeline',
'5003897973989'), ('2017-06-06', 'pipeline',
'5000449148729'), ('2017-09-08', 'pipeline',
'5000525378314'), ('2022-08-30', 'pipeline',
'5005905545593'), ('2022-08-16', 'pipeline',
'5005905545593'), ('2018-05-03', 'pipeline',
'5000750826355'), ('2020-01-10', 'pipeline',
'5001816579677'), ('2021-11-01', 'pipeline',
'5004269423714'), ('2017-11-07', 'pipeline',
'5000585901787'), ('2019-10-15', 'pipeline',
'5001598843430'), ('2020-04-01', 'pipeline',
'5002035795461'), ('2020-02-24', 'pipeline',
'5001932600185'), ('2020-04-27', 'pipeline',
'5002108160849'), ('2016-07-05', 'pipeline',
'554405114'), ('2020-06-02', 'pipeline',
'5002234387967'), ('2020-08-21', 'pipeline',
'5002529168758'), ('2021-02-17', 'pipeline',
'5003158511687');
drop table if exists target_table;
drop table if exists source_table;
create table target_table(date_col date, string_col string, decimal_col
decimal(38,0)) clustered by (decimal_col) into 7 buckets stored as orc
tblproperties ('bucketing_version'='2', 'transactional'='true',
'transactional_properties'='default');
create table source_table(date_col date, string_col string, decimal_col
decimal(38,0)) clustered by (decimal_col) into 7 buckets stored as orc
tblproperties ('bucketing_version'='2', 'transactional'='true',
'transactional_properties'='default');
insert into table target_table select * from test_external_target;
insert into table source_table select * from test_external_source; {code}
Query which is under investigation -
{code:java}
select * from target_table inner join (select distinct date_col, 'pipeline'
string_col, decimal_col from source_table where coalesce(decimal_col,'') =
'5005905545593') s on s.date_col = target_table.date_col AND
s.string_col = target_table.string_col AND s.decimal_col =
target_table.decimal_col; {code}
Expected result of the query - 2 records
{code:java}
++--++-+---++
| target_table.date_col | target_table.string_col |
target_table.decimal_col | s.date_col | s.string_col |
s.decimal_col |
++--++-+---++
| 2022-08-16 | pipeline |
5005905545593 | 2022-08-16 | pipeline |
5005905545593 |
| 2022-08-30 | pipeline |
5005905545593 | 2022-08-30 | pipeline |
5005905545593 |
++--++-+---++
{code}
Actual result of the query - No records
{code:java}
++--+--
[jira] [Updated] (HIVE-27267) Incorrect results when doing bucket map join on decimal bucketed column with subquery
[
https://issues.apache.org/jira/browse/HIVE-27267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sourabh Badhya updated HIVE-27267:
--
Description:
The following queries when run on a Hive cluster produce no results -
Repro queries -
{code:java}
set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
set hive.support.concurrency=true;
set hive.convert.join.bucket.mapjoin.tez=true;
drop table if exists test_external_source;
create external table test_external_source (date_col date, string_col string,
decimal_col decimal(38,0)) stored as orc tblproperties
('external.table.purge'='true');
insert into table test_external_source values ('2022-08-30', 'pipeline',
'5005905545593'), ('2022-08-16', 'pipeline',
'5005905545593'), ('2022-09-01', 'pipeline',
'5006008686831'), ('2022-08-30', 'pipeline',
'5005992620837'), ('2022-09-01', 'pipeline',
'5005992620837'), ('2022-09-01', 'pipeline',
'5005992621067'), ('2022-08-30', 'pipeline',
'5005992621067');
drop table if exists test_external_target;
create external table test_external_target (date_col date, string_col string,
decimal_col decimal(38,0)) stored as orc tblproperties
('external.table.purge'='true');
insert into table test_external_target values ('2017-05-17', 'pipeline',
'5000441610525'), ('2018-12-20', 'pipeline',
'5001048981030'), ('2020-06-30', 'pipeline',
'5002332575516'), ('2021-08-16', 'pipeline',
'5003897973989'), ('2017-06-06', 'pipeline',
'5000449148729'), ('2017-09-08', 'pipeline',
'5000525378314'), ('2022-08-30', 'pipeline',
'5005905545593'), ('2022-08-16', 'pipeline',
'5005905545593'), ('2018-05-03', 'pipeline',
'5000750826355'), ('2020-01-10', 'pipeline',
'5001816579677'), ('2021-11-01', 'pipeline',
'5004269423714'), ('2017-11-07', 'pipeline',
'5000585901787'), ('2019-10-15', 'pipeline',
'5001598843430'), ('2020-04-01', 'pipeline',
'5002035795461'), ('2020-02-24', 'pipeline',
'5001932600185'), ('2020-04-27', 'pipeline',
'5002108160849'), ('2016-07-05', 'pipeline',
'554405114'), ('2020-06-02', 'pipeline',
'5002234387967'), ('2020-08-21', 'pipeline',
'5002529168758'), ('2021-02-17', 'pipeline',
'5003158511687');
drop table if exists target_table;
drop table if exists source_table;
create table target_table(date_col date, string_col string, decimal_col
decimal(38,0)) clustered by (decimal_col) into 7 buckets stored as orc
tblproperties ('bucketing_version'='2', 'transactional'='true',
'transactional_properties'='default');
create table source_table(date_col date, string_col string, decimal_col
decimal(38,0)) clustered by (decimal_col) into 7 buckets stored as orc
tblproperties ('bucketing_version'='2', 'transactional'='true',
'transactional_properties'='default');
insert into table target_table select * from test_external_target;
insert into table source_table select * from test_external_source; {code}
Query which is under investigation -
{code:java}
select * from target_table inner join (select distinct date_col, 'pipeline'
string_col, decimal_col from source_table where coalesce(decimal_col,'') =
'5005905545593') s on s.date_col = target_table.date_col AND
s.string_col = target_table.string_col AND s.decimal_col =
target_table.decimal_col; {code}
Expected result of the query - 2 records
{code:java}
++--++-+---++
| target_table.date_col | target_table.string_col |
target_table.decimal_col | s.date_col | s.string_col |
s.decimal_col |
++--++-+---++
| 2022-08-16 | pipeline |
5005905545593 | 2022-08-16 | pipeline |
5005905545593 |
| 2022-08-30 | pipeline |
5005905545593 | 2022-08-30 | pipeline |
5005905545593 |
++--++-+---++
{code}
Actual result of the query - No records
{code:java}
++--+---+-+---++
| target_table.date_col | target_table.string_col | target_table.decimal_col
| s.date_col | s.string_co
[jira] [Work logged] (HIVE-27199) Read TIMESTAMP WITH LOCAL TIME ZONE columns from text files using custom formats
[
https://issues.apache.org/jira/browse/HIVE-27199?focusedWorklogId=857357&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857357
]
ASF GitHub Bot logged work on HIVE-27199:
-
Author: ASF GitHub Bot
Created on: 17/Apr/23 12:02
Start Date: 17/Apr/23 12:02
Worklog Time Spent: 10m
Work Description: TuroczyX commented on code in PR #4170:
URL: https://github.com/apache/hive/pull/4170#discussion_r1168581467
##
common/src/java/org/apache/hive/common/util/TimestampParser.java:
##
@@ -199,6 +205,19 @@ public Timestamp parseTimestamp(final String text) {
}
+ public TimestampTZ parseTimestamp(String text, ZoneId defaultTimeZone) {
+Objects.requireNonNull(text);
+for (DateTimeFormatter f : dtFormatters) {
+ try {
+return TimestampTZUtil.parse(text, defaultTimeZone, f);
+ } catch (DateTimeException e) {
Review Comment:
```
try {
return TimestampTZUtil.parse(text, defaultTimeZone, f);
} catch (DateTimeException e) {
}
```
If the catch does not anything it is a bit cleaner if it is just try
finally. (Again it is just my style :) It is not a big deal. ) Just when I see
a catch it would mean I will handle somehow an exception. If I do not do I
would rather use finally.
Issue Time Tracking
---
Worklog Id: (was: 857357)
Time Spent: 1h 10m (was: 1h)
> Read TIMESTAMP WITH LOCAL TIME ZONE columns from text files using custom
> formats
>
>
> Key: HIVE-27199
> URL: https://issues.apache.org/jira/browse/HIVE-27199
> Project: Hive
> Issue Type: Improvement
> Components: Serializers/Deserializers
>Affects Versions: 4.0.0-alpha-2
>Reporter: Stamatis Zampetakis
>Assignee: Stamatis Zampetakis
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Timestamp values come in many flavors and formats and there is no single
> representation that can satisfy everyone especially when such values are
> stored in plain text/csv files.
> HIVE-9298, added a special SERDE property, {{{}timestamp.formats{}}}, that
> allows to provide custom timestamp patterns to parse correctly TIMESTAMP
> values coming from files.
> However, when the column type is TIMESTAMP WITH LOCAL TIME ZONE (LTZ) it is
> not possible to use a custom pattern thus when the built-in Hive parser does
> not match the expected format a NULL value is returned.
> Consider a text file, F1, with the following values:
> {noformat}
> 2016-05-03 12:26:34
> 2016-05-03T12:26:34
> {noformat}
> and a table with a column declared as LTZ.
> {code:sql}
> CREATE TABLE ts_table (ts TIMESTAMP WITH LOCAL TIME ZONE);
> LOAD DATA LOCAL INPATH './F1' INTO TABLE ts_table;
> SELECT * FROM ts_table;
> 2016-05-03 12:26:34.0 US/Pacific
> NULL
> {code}
> In order to give more flexibility to the users relying on the TIMESTAMP WITH
> LOCAL TIME ZONE datatype and also align the behavior with the TIMESTAMP type
> this JIRA aims to reuse the {{timestamp.formats}} property for both TIMESTAMP
> types.
> The work here focuses exclusively on simple text files but the same could be
> done for other SERDE such as JSON etc.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27199) Read TIMESTAMP WITH LOCAL TIME ZONE columns from text files using custom formats
[
https://issues.apache.org/jira/browse/HIVE-27199?focusedWorklogId=857352&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857352
]
ASF GitHub Bot logged work on HIVE-27199:
-
Author: ASF GitHub Bot
Created on: 17/Apr/23 11:43
Start Date: 17/Apr/23 11:43
Worklog Time Spent: 10m
Work Description: zabetak commented on code in PR #4170:
URL: https://github.com/apache/hive/pull/4170#discussion_r1168561538
##
common/src/java/org/apache/hive/common/util/TimestampParser.java:
##
@@ -199,6 +205,19 @@ public Timestamp parseTimestamp(final String text) {
}
+ public TimestampTZ parseTimestamp(String text, ZoneId defaultTimeZone) {
+Objects.requireNonNull(text);
+for (DateTimeFormatter f : dtFormatters) {
+ try {
+return TimestampTZUtil.parse(text, defaultTimeZone, f);
+ } catch (DateTimeException e) {
Review Comment:
Thanks for the comments! Can you elaborate a bit on what you had in mind
around try finally?
Issue Time Tracking
---
Worklog Id: (was: 857352)
Time Spent: 1h (was: 50m)
> Read TIMESTAMP WITH LOCAL TIME ZONE columns from text files using custom
> formats
>
>
> Key: HIVE-27199
> URL: https://issues.apache.org/jira/browse/HIVE-27199
> Project: Hive
> Issue Type: Improvement
> Components: Serializers/Deserializers
>Affects Versions: 4.0.0-alpha-2
>Reporter: Stamatis Zampetakis
>Assignee: Stamatis Zampetakis
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Timestamp values come in many flavors and formats and there is no single
> representation that can satisfy everyone especially when such values are
> stored in plain text/csv files.
> HIVE-9298, added a special SERDE property, {{{}timestamp.formats{}}}, that
> allows to provide custom timestamp patterns to parse correctly TIMESTAMP
> values coming from files.
> However, when the column type is TIMESTAMP WITH LOCAL TIME ZONE (LTZ) it is
> not possible to use a custom pattern thus when the built-in Hive parser does
> not match the expected format a NULL value is returned.
> Consider a text file, F1, with the following values:
> {noformat}
> 2016-05-03 12:26:34
> 2016-05-03T12:26:34
> {noformat}
> and a table with a column declared as LTZ.
> {code:sql}
> CREATE TABLE ts_table (ts TIMESTAMP WITH LOCAL TIME ZONE);
> LOAD DATA LOCAL INPATH './F1' INTO TABLE ts_table;
> SELECT * FROM ts_table;
> 2016-05-03 12:26:34.0 US/Pacific
> NULL
> {code}
> In order to give more flexibility to the users relying on the TIMESTAMP WITH
> LOCAL TIME ZONE datatype and also align the behavior with the TIMESTAMP type
> this JIRA aims to reuse the {{timestamp.formats}} property for both TIMESTAMP
> types.
> The work here focuses exclusively on simple text files but the same could be
> done for other SERDE such as JSON etc.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26400) Provide docker images for Hive
[ https://issues.apache.org/jira/browse/HIVE-26400?focusedWorklogId=857348&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857348 ] ASF GitHub Bot logged work on HIVE-26400: - Author: ASF GitHub Bot Created on: 17/Apr/23 11:35 Start Date: 17/Apr/23 11:35 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #3448: URL: https://github.com/apache/hive/pull/3448#issuecomment-1511175822 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3448) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3448&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3448&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3448&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3448&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3448&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3448&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3448&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3448&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3448&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3448&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3448&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3448&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3448&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3448&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 857348) Time Spent: 14h (was: 13h 50m) > Provide docker images for Hive > -- > > Key: HIVE-26400 > URL: https://issues.apache.org/jira/browse/HIVE-26400 > Project: Hive > Issue Type: Sub-task > Components: Build Infrastructure >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Blocker > Labels: hive-4.0.0-must, pull-request-available > Time Spent: 14h > Remaining Estimate: 0h > > Make Apache Hive be able to run inside docker container in pseudo-distributed > mode, with MySQL/Derby as its back database, provide the following: > * Quick-start/Debugging/Prepare a test env for Hive; > * Tools to build target image with specified version of Hive and its > dependencies; > * Images can be used as the basis for the Kubernetes operator. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27264) Literals in conjunction of two in expression are considered not equals if type precision is different
[
https://issues.apache.org/jira/browse/HIVE-27264?focusedWorklogId=857343&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857343
]
ASF GitHub Bot logged work on HIVE-27264:
-
Author: ASF GitHub Bot
Created on: 17/Apr/23 11:09
Start Date: 17/Apr/23 11:09
Worklog Time Spent: 10m
Work Description: sonarcloud[bot] commented on PR #4237:
URL: https://github.com/apache/hive/pull/4237#issuecomment-1511143807
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4237)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4237&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4237&resolved=false&types=BUG)
[0
Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4237&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4237&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4237&resolved=false&types=VULNERABILITY)
[0
Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4237&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4237&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4237&resolved=false&types=SECURITY_HOTSPOT)
[0 Security
Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4237&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4237&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4237&resolved=false&types=CODE_SMELL)
[0 Code
Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4237&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4237&metric=coverage&view=list)
No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4237&metric=duplicated_lines_density&view=list)
No Duplication information
Issue Time Tracking
---
Worklog Id: (was: 857343)
Time Spent: 1h (was: 50m)
> Literals in conjunction of two in expression are considered not equals if
> type precision is different
> -
>
> Key: HIVE-27264
> URL: https://issues.apache.org/jira/browse/HIVE-27264
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> {code}
> create table r_table (
> string_col varchar(30)
> );
> create table l_table (
> string_col varchar(14)
> );
> insert into r_table VALUES ('AAA111');
> insert into l_table VALUES ('AAA111');
> SELECT l_table.string_col from l_table, r_table
> WHERE r_table.string_col = l_table.string_col AND l_table.string_col IN
> ('AAA111', 'BBB222') AND r_table.string_col IN ('AAA111', 'BBB222');
> {code}
> Should give one row
> {code}
> AAA111
> {code}
> but it returns empty rs
> Workaroun
[jira] [Assigned] (HIVE-24815) Remove "IDXS" Table from Metastore Schema
[ https://issues.apache.org/jira/browse/HIVE-24815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Riju Trivedi reassigned HIVE-24815: --- Assignee: Riju Trivedi (was: Hunter Logan) > Remove "IDXS" Table from Metastore Schema > - > > Key: HIVE-24815 > URL: https://issues.apache.org/jira/browse/HIVE-24815 > Project: Hive > Issue Type: Improvement > Components: Metastore, Standalone Metastore >Affects Versions: 3.1.0, 3.0.0, 3.1.1, 3.1.2, 3.2.0, 4.0.0 >Reporter: Hunter Logan >Assignee: Riju Trivedi >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > In Hive 3 the rarely used "INDEXES" was removed from the DDL > https://issues.apache.org/jira/browse/HIVE-18448 > > There are a few issues here: > # The Standalone-Metastore schema for Hive 3+ all include the "IDXS" table, > which has no function. > ** > [https://github.com/apache/hive/tree/master/standalone-metastore/metastore-server/src/main/sql/mysql] > # The upgrade schemas from 2.x -> 3.x do not do any cleanup of the IDXS table > ** If a user used the "INDEXES" feature in 2.x and then upgrades their > metastore to 3.x+ they cannot drop any table that has an index on it due to > "IDXS_FK1" constraint since the TBLS entry is referenced in the IDXS table > ** Since INDEX is no longer in the DDL they cannot run any command from Hive > to drop the index. > ** Users can manually connect to the metastore and either drop the IDXS > table or the foreign key constraint > > Since indexes provide no benefits in Hive 3+ it should be fine to drop them > completely in the schema upgrade scripts. At the very least the 2.x -> 3.x+ > scripts should drop the fk constraint. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27199) Read TIMESTAMP WITH LOCAL TIME ZONE columns from text files using custom formats
[
https://issues.apache.org/jira/browse/HIVE-27199?focusedWorklogId=857339&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857339
]
ASF GitHub Bot logged work on HIVE-27199:
-
Author: ASF GitHub Bot
Created on: 17/Apr/23 10:40
Start Date: 17/Apr/23 10:40
Worklog Time Spent: 10m
Work Description: TuroczyX commented on code in PR #4170:
URL: https://github.com/apache/hive/pull/4170#discussion_r1168500611
##
common/src/java/org/apache/hive/common/util/TimestampParser.java:
##
@@ -199,6 +205,19 @@ public Timestamp parseTimestamp(final String text) {
}
+ public TimestampTZ parseTimestamp(String text, ZoneId defaultTimeZone) {
+Objects.requireNonNull(text);
+for (DateTimeFormatter f : dtFormatters) {
+ try {
+return TimestampTZUtil.parse(text, defaultTimeZone, f);
+ } catch (DateTimeException e) {
Review Comment:
Also, from pattern perspective a TryParse would be more elegant in this
case. Of course it is just preferences, but I like this pattern. Way more
descriptive from code reading perspective.
https://learn.microsoft.com/en-us/dotnet/api/system.int32.tryparse?view=net-8.0#system-int32-tryparse(system-string-system-int32@)
I know ref and out keyword are not exists in Java but with return type it is
possible to handle. (Just FYI, no need to change)
Issue Time Tracking
---
Worklog Id: (was: 857339)
Time Spent: 50m (was: 40m)
> Read TIMESTAMP WITH LOCAL TIME ZONE columns from text files using custom
> formats
>
>
> Key: HIVE-27199
> URL: https://issues.apache.org/jira/browse/HIVE-27199
> Project: Hive
> Issue Type: Improvement
> Components: Serializers/Deserializers
>Affects Versions: 4.0.0-alpha-2
>Reporter: Stamatis Zampetakis
>Assignee: Stamatis Zampetakis
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Timestamp values come in many flavors and formats and there is no single
> representation that can satisfy everyone especially when such values are
> stored in plain text/csv files.
> HIVE-9298, added a special SERDE property, {{{}timestamp.formats{}}}, that
> allows to provide custom timestamp patterns to parse correctly TIMESTAMP
> values coming from files.
> However, when the column type is TIMESTAMP WITH LOCAL TIME ZONE (LTZ) it is
> not possible to use a custom pattern thus when the built-in Hive parser does
> not match the expected format a NULL value is returned.
> Consider a text file, F1, with the following values:
> {noformat}
> 2016-05-03 12:26:34
> 2016-05-03T12:26:34
> {noformat}
> and a table with a column declared as LTZ.
> {code:sql}
> CREATE TABLE ts_table (ts TIMESTAMP WITH LOCAL TIME ZONE);
> LOAD DATA LOCAL INPATH './F1' INTO TABLE ts_table;
> SELECT * FROM ts_table;
> 2016-05-03 12:26:34.0 US/Pacific
> NULL
> {code}
> In order to give more flexibility to the users relying on the TIMESTAMP WITH
> LOCAL TIME ZONE datatype and also align the behavior with the TIMESTAMP type
> this JIRA aims to reuse the {{timestamp.formats}} property for both TIMESTAMP
> types.
> The work here focuses exclusively on simple text files but the same could be
> done for other SERDE such as JSON etc.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27199) Read TIMESTAMP WITH LOCAL TIME ZONE columns from text files using custom formats
[
https://issues.apache.org/jira/browse/HIVE-27199?focusedWorklogId=857336&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857336
]
ASF GitHub Bot logged work on HIVE-27199:
-
Author: ASF GitHub Bot
Created on: 17/Apr/23 10:34
Start Date: 17/Apr/23 10:34
Worklog Time Spent: 10m
Work Description: TuroczyX commented on code in PR #4170:
URL: https://github.com/apache/hive/pull/4170#discussion_r1168494775
##
common/src/java/org/apache/hive/common/util/TimestampParser.java:
##
@@ -199,6 +205,19 @@ public Timestamp parseTimestamp(final String text) {
}
+ public TimestampTZ parseTimestamp(String text, ZoneId defaultTimeZone) {
+Objects.requireNonNull(text);
+for (DateTimeFormatter f : dtFormatters) {
+ try {
+return TimestampTZUtil.parse(text, defaultTimeZone, f);
+ } catch (DateTimeException e) {
Review Comment:
Try finally won't better in this case?
Issue Time Tracking
---
Worklog Id: (was: 857336)
Time Spent: 40m (was: 0.5h)
> Read TIMESTAMP WITH LOCAL TIME ZONE columns from text files using custom
> formats
>
>
> Key: HIVE-27199
> URL: https://issues.apache.org/jira/browse/HIVE-27199
> Project: Hive
> Issue Type: Improvement
> Components: Serializers/Deserializers
>Affects Versions: 4.0.0-alpha-2
>Reporter: Stamatis Zampetakis
>Assignee: Stamatis Zampetakis
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Timestamp values come in many flavors and formats and there is no single
> representation that can satisfy everyone especially when such values are
> stored in plain text/csv files.
> HIVE-9298, added a special SERDE property, {{{}timestamp.formats{}}}, that
> allows to provide custom timestamp patterns to parse correctly TIMESTAMP
> values coming from files.
> However, when the column type is TIMESTAMP WITH LOCAL TIME ZONE (LTZ) it is
> not possible to use a custom pattern thus when the built-in Hive parser does
> not match the expected format a NULL value is returned.
> Consider a text file, F1, with the following values:
> {noformat}
> 2016-05-03 12:26:34
> 2016-05-03T12:26:34
> {noformat}
> and a table with a column declared as LTZ.
> {code:sql}
> CREATE TABLE ts_table (ts TIMESTAMP WITH LOCAL TIME ZONE);
> LOAD DATA LOCAL INPATH './F1' INTO TABLE ts_table;
> SELECT * FROM ts_table;
> 2016-05-03 12:26:34.0 US/Pacific
> NULL
> {code}
> In order to give more flexibility to the users relying on the TIMESTAMP WITH
> LOCAL TIME ZONE datatype and also align the behavior with the TIMESTAMP type
> this JIRA aims to reuse the {{timestamp.formats}} property for both TIMESTAMP
> types.
> The work here focuses exclusively on simple text files but the same could be
> done for other SERDE such as JSON etc.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27110) Upgrade com.esotericsoftware:Kyro version to 5.2.0
[
https://issues.apache.org/jira/browse/HIVE-27110?focusedWorklogId=857335&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857335
]
ASF GitHub Bot logged work on HIVE-27110:
-
Author: ASF GitHub Bot
Created on: 17/Apr/23 10:33
Start Date: 17/Apr/23 10:33
Worklog Time Spent: 10m
Work Description: TuroczyX commented on PR #4087:
URL: https://github.com/apache/hive/pull/4087#issuecomment-1511097708
@Indhumathi27 and it is done :) Thanks @abstractdog
Issue Time Tracking
---
Worklog Id: (was: 857335)
Time Spent: 1h 10m (was: 1h)
> Upgrade com.esotericsoftware:Kyro version to 5.2.0
> --
>
> Key: HIVE-27110
> URL: https://issues.apache.org/jira/browse/HIVE-27110
> Project: Hive
> Issue Type: Task
>Reporter: Indhumathi Muthumurugesh
>Assignee: Indhumathi Muthumurugesh
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> The kryo package (5.0.3) is vulnerable to Uncaught
> Exceptionconditions([https://cwe.mitre.org/data/definitions/248.html)|https://cwe.mitre.org/data/definitions/248.html).]
> The `{*}readReferenceOrNull(){*}` method in the kryo class fails to process
> exceptions that may occur due to undefined ArrayList references in the
> `{*}RefernceResolver`{*} interface while processing input data during
> serialization or deserialization, which may lead to application crash or
> uninformative exceptions.
>
> The above CWE-248 can be avoided by upgrading the kryo version to 5.2.0,
> which has addressed the issue.
> For Reference:
> [https://github.com/EsotericSoftware/kryo/blob/kryo-parent-5.2.0/src/com/esotericsoftware/kryo/Kryo.java]
> Issue: https://github.com/EsotericSoftware/kryo/issues/829
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27110) Upgrade com.esotericsoftware:Kyro version to 5.2.0
[
https://issues.apache.org/jira/browse/HIVE-27110?focusedWorklogId=857329&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857329
]
ASF GitHub Bot logged work on HIVE-27110:
-
Author: ASF GitHub Bot
Created on: 17/Apr/23 09:56
Start Date: 17/Apr/23 09:56
Worklog Time Spent: 10m
Work Description: abstractdog merged PR #4087:
URL: https://github.com/apache/hive/pull/4087
Issue Time Tracking
---
Worklog Id: (was: 857329)
Time Spent: 1h (was: 50m)
> Upgrade com.esotericsoftware:Kyro version to 5.2.0
> --
>
> Key: HIVE-27110
> URL: https://issues.apache.org/jira/browse/HIVE-27110
> Project: Hive
> Issue Type: Task
>Reporter: Indhumathi Muthumurugesh
>Assignee: Indhumathi Muthumurugesh
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> The kryo package (5.0.3) is vulnerable to Uncaught
> Exceptionconditions([https://cwe.mitre.org/data/definitions/248.html)|https://cwe.mitre.org/data/definitions/248.html).]
> The `{*}readReferenceOrNull(){*}` method in the kryo class fails to process
> exceptions that may occur due to undefined ArrayList references in the
> `{*}RefernceResolver`{*} interface while processing input data during
> serialization or deserialization, which may lead to application crash or
> uninformative exceptions.
>
> The above CWE-248 can be avoided by upgrading the kryo version to 5.2.0,
> which has addressed the issue.
> For Reference:
> [https://github.com/EsotericSoftware/kryo/blob/kryo-parent-5.2.0/src/com/esotericsoftware/kryo/Kryo.java]
> Issue: https://github.com/EsotericSoftware/kryo/issues/829
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-27110) Upgrade com.esotericsoftware:Kyro version to 5.2.0
[
https://issues.apache.org/jira/browse/HIVE-27110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-27110:
Fix Version/s: 4.0.0
> Upgrade com.esotericsoftware:Kyro version to 5.2.0
> --
>
> Key: HIVE-27110
> URL: https://issues.apache.org/jira/browse/HIVE-27110
> Project: Hive
> Issue Type: Task
>Reporter: Indhumathi Muthumurugesh
>Assignee: Indhumathi Muthumurugesh
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> The kryo package (5.0.3) is vulnerable to Uncaught
> Exceptionconditions([https://cwe.mitre.org/data/definitions/248.html)|https://cwe.mitre.org/data/definitions/248.html).]
> The `{*}readReferenceOrNull(){*}` method in the kryo class fails to process
> exceptions that may occur due to undefined ArrayList references in the
> `{*}RefernceResolver`{*} interface while processing input data during
> serialization or deserialization, which may lead to application crash or
> uninformative exceptions.
>
> The above CWE-248 can be avoided by upgrading the kryo version to 5.2.0,
> which has addressed the issue.
> For Reference:
> [https://github.com/EsotericSoftware/kryo/blob/kryo-parent-5.2.0/src/com/esotericsoftware/kryo/Kryo.java]
> Issue: https://github.com/EsotericSoftware/kryo/issues/829
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HIVE-27110) Upgrade com.esotericsoftware:Kyro version to 5.2.0
[
https://issues.apache.org/jira/browse/HIVE-27110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor resolved HIVE-27110.
-
Resolution: Fixed
> Upgrade com.esotericsoftware:Kyro version to 5.2.0
> --
>
> Key: HIVE-27110
> URL: https://issues.apache.org/jira/browse/HIVE-27110
> Project: Hive
> Issue Type: Task
>Reporter: Indhumathi Muthumurugesh
>Assignee: Indhumathi Muthumurugesh
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> The kryo package (5.0.3) is vulnerable to Uncaught
> Exceptionconditions([https://cwe.mitre.org/data/definitions/248.html)|https://cwe.mitre.org/data/definitions/248.html).]
> The `{*}readReferenceOrNull(){*}` method in the kryo class fails to process
> exceptions that may occur due to undefined ArrayList references in the
> `{*}RefernceResolver`{*} interface while processing input data during
> serialization or deserialization, which may lead to application crash or
> uninformative exceptions.
>
> The above CWE-248 can be avoided by upgrading the kryo version to 5.2.0,
> which has addressed the issue.
> For Reference:
> [https://github.com/EsotericSoftware/kryo/blob/kryo-parent-5.2.0/src/com/esotericsoftware/kryo/Kryo.java]
> Issue: https://github.com/EsotericSoftware/kryo/issues/829
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HIVE-27110) Upgrade com.esotericsoftware:Kyro version to 5.2.0
[
https://issues.apache.org/jira/browse/HIVE-27110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17712997#comment-17712997
]
László Bodor commented on HIVE-27110:
-
merged to master, thanks [~Indhumathi27] for the patch!
> Upgrade com.esotericsoftware:Kyro version to 5.2.0
> --
>
> Key: HIVE-27110
> URL: https://issues.apache.org/jira/browse/HIVE-27110
> Project: Hive
> Issue Type: Task
>Reporter: Indhumathi Muthumurugesh
>Assignee: Indhumathi Muthumurugesh
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> The kryo package (5.0.3) is vulnerable to Uncaught
> Exceptionconditions([https://cwe.mitre.org/data/definitions/248.html)|https://cwe.mitre.org/data/definitions/248.html).]
> The `{*}readReferenceOrNull(){*}` method in the kryo class fails to process
> exceptions that may occur due to undefined ArrayList references in the
> `{*}RefernceResolver`{*} interface while processing input data during
> serialization or deserialization, which may lead to application crash or
> uninformative exceptions.
>
> The above CWE-248 can be avoided by upgrading the kryo version to 5.2.0,
> which has addressed the issue.
> For Reference:
> [https://github.com/EsotericSoftware/kryo/blob/kryo-parent-5.2.0/src/com/esotericsoftware/kryo/Kryo.java]
> Issue: https://github.com/EsotericSoftware/kryo/issues/829
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27110) Upgrade com.esotericsoftware:Kyro version to 5.2.0
[
https://issues.apache.org/jira/browse/HIVE-27110?focusedWorklogId=857328&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857328
]
ASF GitHub Bot logged work on HIVE-27110:
-
Author: ASF GitHub Bot
Created on: 17/Apr/23 09:54
Start Date: 17/Apr/23 09:54
Worklog Time Spent: 10m
Work Description: abstractdog commented on PR #4087:
URL: https://github.com/apache/hive/pull/4087#issuecomment-1511041509
LGTM, pending precommit tests
Issue Time Tracking
---
Worklog Id: (was: 857328)
Time Spent: 50m (was: 40m)
> Upgrade com.esotericsoftware:Kyro version to 5.2.0
> --
>
> Key: HIVE-27110
> URL: https://issues.apache.org/jira/browse/HIVE-27110
> Project: Hive
> Issue Type: Task
>Reporter: Indhumathi Muthumurugesh
>Assignee: Indhumathi Muthumurugesh
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> The kryo package (5.0.3) is vulnerable to Uncaught
> Exceptionconditions([https://cwe.mitre.org/data/definitions/248.html)|https://cwe.mitre.org/data/definitions/248.html).]
> The `{*}readReferenceOrNull(){*}` method in the kryo class fails to process
> exceptions that may occur due to undefined ArrayList references in the
> `{*}RefernceResolver`{*} interface while processing input data during
> serialization or deserialization, which may lead to application crash or
> uninformative exceptions.
>
> The above CWE-248 can be avoided by upgrading the kryo version to 5.2.0,
> which has addressed the issue.
> For Reference:
> [https://github.com/EsotericSoftware/kryo/blob/kryo-parent-5.2.0/src/com/esotericsoftware/kryo/Kryo.java]
> Issue: https://github.com/EsotericSoftware/kryo/issues/829
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26400) Provide docker images for Hive
[ https://issues.apache.org/jira/browse/HIVE-26400?focusedWorklogId=857323&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857323 ] ASF GitHub Bot logged work on HIVE-26400: - Author: ASF GitHub Bot Created on: 17/Apr/23 09:37 Start Date: 17/Apr/23 09:37 Worklog Time Spent: 10m Work Description: zratkai commented on PR #3448: URL: https://github.com/apache/hive/pull/3448#issuecomment-1511018569 I think it would be a good idea to add ports for development / debugging like this ones: -p9866:9866 -p1:1 -p10001:10001 -p9000:9000 -p8000:8000 -p3306:3306 -p50070:50070 -p50030:50030 Issue Time Tracking --- Worklog Id: (was: 857323) Time Spent: 13h 50m (was: 13h 40m) > Provide docker images for Hive > -- > > Key: HIVE-26400 > URL: https://issues.apache.org/jira/browse/HIVE-26400 > Project: Hive > Issue Type: Sub-task > Components: Build Infrastructure >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Blocker > Labels: hive-4.0.0-must, pull-request-available > Time Spent: 13h 50m > Remaining Estimate: 0h > > Make Apache Hive be able to run inside docker container in pseudo-distributed > mode, with MySQL/Derby as its back database, provide the following: > * Quick-start/Debugging/Prepare a test env for Hive; > * Tools to build target image with specified version of Hive and its > dependencies; > * Images can be used as the basis for the Kubernetes operator. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27020) Implement a separate handler to handle aborted transaction cleanup
[ https://issues.apache.org/jira/browse/HIVE-27020?focusedWorklogId=857322&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857322 ] ASF GitHub Bot logged work on HIVE-27020: - Author: ASF GitHub Bot Created on: 17/Apr/23 09:32 Start Date: 17/Apr/23 09:32 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4091: URL: https://github.com/apache/hive/pull/4091#issuecomment-1511012807 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4091) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4091&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4091&resolved=false&types=BUG) [1 Bug](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4091&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4091&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4091&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4091&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4091&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4091&resolved=false&types=SECURITY_HOTSPOT) [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4091&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4091&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4091&resolved=false&types=CODE_SMELL) [10 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4091&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4091&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4091&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 857322) Time Spent: 14h 20m (was: 14h 10m) > Implement a separate handler to handle aborted transaction cleanup > -- > > Key: HIVE-27020 > URL: https://issues.apache.org/jira/browse/HIVE-27020 > Project: Hive > Issue Type: Sub-task >Reporter: Sourabh Badhya >Assignee: Sourabh Badhya >Priority: Major > Labels: pull-request-available > Time Spent: 14h 20m > Remaining Estimate: 0h > > As described in the parent task, once the cleaner is separated into different > entities, implement a separate handler which can create requests for aborted > transactions cleanup. This would move the aborted transaction cleanup > exclusively to the cleaner. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27082) AggregateStatsCache.findBestMatch() in Metastore should test the inclusion of default partition name
[ https://issues.apache.org/jira/browse/HIVE-27082?focusedWorklogId=857321&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857321 ] ASF GitHub Bot logged work on HIVE-27082: - Author: ASF GitHub Bot Created on: 17/Apr/23 09:22 Start Date: 17/Apr/23 09:22 Worklog Time Spent: 10m Work Description: ngsg opened a new pull request, #4239: URL: https://github.com/apache/hive/pull/4239 ### What changes were proposed in this pull request? AggregateStatsCache.findBestMatch() in Metastore should test the inclusion of default partition name \_\_HIVE_DEFAULT_PARTITION\_\_. ### Why are the changes needed? To avoid the non-deterministic behavior of Hive in generating DAGs. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Manually tested with TPC-DS 10TB Issue Time Tracking --- Worklog Id: (was: 857321) Remaining Estimate: 0h Time Spent: 10m > AggregateStatsCache.findBestMatch() in Metastore should test the inclusion of > default partition name > > > Key: HIVE-27082 > URL: https://issues.apache.org/jira/browse/HIVE-27082 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Affects Versions: 3.1.3, 4.0.0-alpha-2 >Reporter: Sungwoo Park >Assignee: Seonggon Namgung >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > This JIRA deals with non-determinisitic behavior of Hive in generating DAGs. > The non-determinstic behavior of Hive in generating DAGs is due to the logic > in AggregateStatsCache.findBestMatch() called from AggregateStatsCache.get(), > as well as the disproportionate distribution of Nulls in > HIVE_DEFAULT_PARTITION. > Here is what is happening in the case of the TPC-DS dataset. Let us use > web_sales table and ws_web_site_sk column in the 10TB TPC-DS dataset as a > running example. > In the course of running TPC-DS queries, Hive asks MetaStore about the column > statistics of 1823 partNames in the web_sales/ws_web_site_sk combination, > either without HIVE_DEFAULT_PARTITION or with HIVE_DEFAULT_PARTITION. > --- Without HIVE_DEFAULT_PARTITION, it reports a total of 901180 nulls. > --- With HIVE_DEFAULT_PARTITION, however, it report a total of 1800087 nulls, > almost twice as many. > The first call to MetaStore returns the correct result, but all subsequent > requests are likely to return the same result from the cache, irrespective of > the inclusion of HIVE_DEFAULT_PARTITION. This is because > AggregateStatsCache.findBestMatch() treats HIVE_DEFAULT_PARTITION in the same > way as other partNames, and the difference in the size of partNames[] is just > 1. The outcome depends on the duration of intervening queries, so everything > is now non-deterministic. > If a wrong value of numNulls is returned, Hive generates a different DAG > which make takes much longer than the correct one. The problem is > particularly pronounced here because of the huge number of nulls in > HIVE_DEFAULT_PARTITION. It is ironic to see that the query optimizer is so > efficient that a single wrong guess of numNulls creates a very inefficient > DAG. > Note that this behavior cannot be avoided by setting > hive.metastore.aggregate.stats.cache.max.variance to zero because the > difference in the number of partNames[] between the argument and the entry in > the cache is just 1. > So, AggregateStatsCache.findBestMatch() should treat HIVE_DEFAULT_PARTITION > in a special way, by not returning the result in the cache if there is a > difference in the inclusion of partName HIVE_DEFAULT_PARTITION (or should > provide the use with an option to activate this feature). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27110) Upgrade com.esotericsoftware:Kyro version to 5.2.0
[
https://issues.apache.org/jira/browse/HIVE-27110?focusedWorklogId=857315&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857315
]
ASF GitHub Bot logged work on HIVE-27110:
-
Author: ASF GitHub Bot
Created on: 17/Apr/23 09:12
Start Date: 17/Apr/23 09:12
Worklog Time Spent: 10m
Work Description: Indhumathi27 commented on PR #4087:
URL: https://github.com/apache/hive/pull/4087#issuecomment-1510984338
> Do you need any help with this?
@TuroczyX please help to review the PR
Issue Time Tracking
---
Worklog Id: (was: 857315)
Time Spent: 40m (was: 0.5h)
> Upgrade com.esotericsoftware:Kyro version to 5.2.0
> --
>
> Key: HIVE-27110
> URL: https://issues.apache.org/jira/browse/HIVE-27110
> Project: Hive
> Issue Type: Task
>Reporter: Indhumathi Muthumurugesh
>Assignee: Indhumathi Muthumurugesh
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> The kryo package (5.0.3) is vulnerable to Uncaught
> Exceptionconditions([https://cwe.mitre.org/data/definitions/248.html)|https://cwe.mitre.org/data/definitions/248.html).]
> The `{*}readReferenceOrNull(){*}` method in the kryo class fails to process
> exceptions that may occur due to undefined ArrayList references in the
> `{*}RefernceResolver`{*} interface while processing input data during
> serialization or deserialization, which may lead to application crash or
> uninformative exceptions.
>
> The above CWE-248 can be avoided by upgrading the kryo version to 5.2.0,
> which has addressed the issue.
> For Reference:
> [https://github.com/EsotericSoftware/kryo/blob/kryo-parent-5.2.0/src/com/esotericsoftware/kryo/Kryo.java]
> Issue: https://github.com/EsotericSoftware/kryo/issues/829
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (HIVE-27082) AggregateStatsCache.findBestMatch() in Metastore should test the inclusion of default partition name
[ https://issues.apache.org/jira/browse/HIVE-27082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seonggon Namgung reassigned HIVE-27082: --- Assignee: Seonggon Namgung (was: Sungwoo Park) > AggregateStatsCache.findBestMatch() in Metastore should test the inclusion of > default partition name > > > Key: HIVE-27082 > URL: https://issues.apache.org/jira/browse/HIVE-27082 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Affects Versions: 3.1.3, 4.0.0-alpha-2 >Reporter: Sungwoo Park >Assignee: Seonggon Namgung >Priority: Major > Labels: pull-request-available > > This JIRA deals with non-determinisitic behavior of Hive in generating DAGs. > The non-determinstic behavior of Hive in generating DAGs is due to the logic > in AggregateStatsCache.findBestMatch() called from AggregateStatsCache.get(), > as well as the disproportionate distribution of Nulls in > HIVE_DEFAULT_PARTITION. > Here is what is happening in the case of the TPC-DS dataset. Let us use > web_sales table and ws_web_site_sk column in the 10TB TPC-DS dataset as a > running example. > In the course of running TPC-DS queries, Hive asks MetaStore about the column > statistics of 1823 partNames in the web_sales/ws_web_site_sk combination, > either without HIVE_DEFAULT_PARTITION or with HIVE_DEFAULT_PARTITION. > --- Without HIVE_DEFAULT_PARTITION, it reports a total of 901180 nulls. > --- With HIVE_DEFAULT_PARTITION, however, it report a total of 1800087 nulls, > almost twice as many. > The first call to MetaStore returns the correct result, but all subsequent > requests are likely to return the same result from the cache, irrespective of > the inclusion of HIVE_DEFAULT_PARTITION. This is because > AggregateStatsCache.findBestMatch() treats HIVE_DEFAULT_PARTITION in the same > way as other partNames, and the difference in the size of partNames[] is just > 1. The outcome depends on the duration of intervening queries, so everything > is now non-deterministic. > If a wrong value of numNulls is returned, Hive generates a different DAG > which make takes much longer than the correct one. The problem is > particularly pronounced here because of the huge number of nulls in > HIVE_DEFAULT_PARTITION. It is ironic to see that the query optimizer is so > efficient that a single wrong guess of numNulls creates a very inefficient > DAG. > Note that this behavior cannot be avoided by setting > hive.metastore.aggregate.stats.cache.max.variance to zero because the > difference in the number of partNames[] between the argument and the entry in > the cache is just 1. > So, AggregateStatsCache.findBestMatch() should treat HIVE_DEFAULT_PARTITION > in a special way, by not returning the result in the cache if there is a > difference in the inclusion of partName HIVE_DEFAULT_PARTITION (or should > provide the use with an option to activate this feature). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27264) Literals in conjunction of two in expression are considered not equals if type precision is different
[
https://issues.apache.org/jira/browse/HIVE-27264?focusedWorklogId=857311&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857311
]
ASF GitHub Bot logged work on HIVE-27264:
-
Author: ASF GitHub Bot
Created on: 17/Apr/23 08:49
Start Date: 17/Apr/23 08:49
Worklog Time Spent: 10m
Work Description: kasakrisz commented on code in PR #4237:
URL: https://github.com/apache/hive/pull/4237#discussion_r1168377479
##
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HivePointLookupOptimizerRule.java:
##
@@ -669,6 +670,22 @@ private static RexNode handleAND(RexBuilder rexBuilder,
RexCall call) {
return RexUtil.composeConjunction(rexBuilder, newOperands, false);
}
+private static void retainAll(Collection elementsToRetain,
Collection collection) {
+ collection.removeIf(rexNode -> elementsToRetain.stream().noneMatch(
+ rexNodeToRetain -> equalsWithSimilarType(rexNode,
rexNodeToRetain)));
+}
+
+private static boolean equalsWithSimilarType(RexNode rexNode1, RexNode
rexNode2) {
+ if (!(rexNode1 instanceof RexLiteral) || !(rexNode2 instanceof
RexLiteral)) {
+return rexNode1.equals(rexNode2);
+ }
+
+ RexLiteral rexLiteral1 = (RexLiteral) rexNode1;
+ RexLiteral rexLiteral2 = (RexLiteral) rexNode2;
+ return rexLiteral1.getValue().compareTo(rexLiteral2.getValue()) == 0 &&
+
rexLiteral1.getType().getSqlTypeName().equals(rexLiteral2.getType().getSqlTypeName());
Review Comment:
`SqlTypeName` comparison is used in two methods:
* `shareSameType`: used for checking whether all constants in both
collections are has the same sql type.
* `retainAll`: removes all elements from the 2nd collection which is not
present in the first one. The original `collection.retainAll` method uses the
`Object.equals` method for comparing elements but in this case it is too strict
and this customization of the method uses the `equalsWithSimilarType`.
I can combine the two but it would result a 20-30 lines code with nested
loops harder to understand.
If your concern is performance we are comparing Enum constants here it
should be fast.
https://github.com/apache/calcite/blob/68b02dfd4af15bc94a91a0cd2a30655d04439555/core/src/main/java/org/apache/calcite/sql/type/SqlTypeName.java#L52
Issue Time Tracking
---
Worklog Id: (was: 857311)
Time Spent: 50m (was: 40m)
> Literals in conjunction of two in expression are considered not equals if
> type precision is different
> -
>
> Key: HIVE-27264
> URL: https://issues.apache.org/jira/browse/HIVE-27264
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> {code}
> create table r_table (
> string_col varchar(30)
> );
> create table l_table (
> string_col varchar(14)
> );
> insert into r_table VALUES ('AAA111');
> insert into l_table VALUES ('AAA111');
> SELECT l_table.string_col from l_table, r_table
> WHERE r_table.string_col = l_table.string_col AND l_table.string_col IN
> ('AAA111', 'BBB222') AND r_table.string_col IN ('AAA111', 'BBB222');
> {code}
> Should give one row
> {code}
> AAA111
> {code}
> but it returns empty rs
> Workaround
> {code}
> set hive.optimize.point.lookup=false;
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27199) Read TIMESTAMP WITH LOCAL TIME ZONE columns from text files using custom formats
[
https://issues.apache.org/jira/browse/HIVE-27199?focusedWorklogId=857304&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857304
]
ASF GitHub Bot logged work on HIVE-27199:
-
Author: ASF GitHub Bot
Created on: 17/Apr/23 08:17
Start Date: 17/Apr/23 08:17
Worklog Time Spent: 10m
Work Description: sonarcloud[bot] commented on PR #4170:
URL: https://github.com/apache/hive/pull/4170#issuecomment-1510907431
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4170)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4170&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4170&resolved=false&types=BUG)
[0
Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4170&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4170&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4170&resolved=false&types=VULNERABILITY)
[0
Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4170&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4170&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4170&resolved=false&types=SECURITY_HOTSPOT)
[0 Security
Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4170&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4170&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4170&resolved=false&types=CODE_SMELL)
[0 Code
Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4170&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4170&metric=coverage&view=list)
No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4170&metric=duplicated_lines_density&view=list)
No Duplication information
Issue Time Tracking
---
Worklog Id: (was: 857304)
Time Spent: 0.5h (was: 20m)
> Read TIMESTAMP WITH LOCAL TIME ZONE columns from text files using custom
> formats
>
>
> Key: HIVE-27199
> URL: https://issues.apache.org/jira/browse/HIVE-27199
> Project: Hive
> Issue Type: Improvement
> Components: Serializers/Deserializers
>Affects Versions: 4.0.0-alpha-2
>Reporter: Stamatis Zampetakis
>Assignee: Stamatis Zampetakis
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Timestamp values come in many flavors and formats and there is no single
> representation that can satisfy everyone especially when such values are
> stored in plain text/csv files.
> HIVE-9298, added a special SERDE property, {{{}timestamp.formats{}}}, that
> allows to provide custom timestamp patterns to parse correctly TIMESTAMP
> values coming from files.
> However, when the column type is TIMESTAMP WITH LOCAL TIME ZONE (LTZ) it is
> not poss
[jira] [Work logged] (HIVE-27264) Literals in conjunction of two in expression are considered not equals if type precision is different
[
https://issues.apache.org/jira/browse/HIVE-27264?focusedWorklogId=857301&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857301
]
ASF GitHub Bot logged work on HIVE-27264:
-
Author: ASF GitHub Bot
Created on: 17/Apr/23 08:08
Start Date: 17/Apr/23 08:08
Worklog Time Spent: 10m
Work Description: kasakrisz commented on code in PR #4237:
URL: https://github.com/apache/hive/pull/4237#discussion_r1168331370
##
ql/src/test/queries/clientpositive/pointlookup6.q:
##
@@ -0,0 +1,17 @@
+create table r_table (
Review Comment:
Changing `hive.optimize.point.lookup` to `false` changes the plan in the
test output so I wouldn't add this explicitly. It would also turn off the whole
point lookup optimization functionality and in general we don't want to turn
off any functionality because it would lead to dead functionalities and make
very hard to re-enable it after a while.
I added
```
set hive.optimize.point.lookup.min=2
```
It makes sense in this test case since we have only two constants in the
`IN` expressions.
Issue Time Tracking
---
Worklog Id: (was: 857301)
Time Spent: 40m (was: 0.5h)
> Literals in conjunction of two in expression are considered not equals if
> type precision is different
> -
>
> Key: HIVE-27264
> URL: https://issues.apache.org/jira/browse/HIVE-27264
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> {code}
> create table r_table (
> string_col varchar(30)
> );
> create table l_table (
> string_col varchar(14)
> );
> insert into r_table VALUES ('AAA111');
> insert into l_table VALUES ('AAA111');
> SELECT l_table.string_col from l_table, r_table
> WHERE r_table.string_col = l_table.string_col AND l_table.string_col IN
> ('AAA111', 'BBB222') AND r_table.string_col IN ('AAA111', 'BBB222');
> {code}
> Should give one row
> {code}
> AAA111
> {code}
> but it returns empty rs
> Workaround
> {code}
> set hive.optimize.point.lookup=false;
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
