[jira] [Work logged] (HIVE-27216) Upgrade postgresql to 42.5.1 from 9.x
[ https://issues.apache.org/jira/browse/HIVE-27216?focusedWorklogId=854689&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854689 ] ASF GitHub Bot logged work on HIVE-27216: - Author: ASF GitHub Bot Created on: 04/Apr/23 06:50 Start Date: 04/Apr/23 06:50 Worklog Time Spent: 10m Work Description: amanraj2520 commented on PR #4195: URL: https://github.com/apache/hive/pull/4195#issuecomment-1495439209 @vihangk1 Can you also review this Issue Time Tracking --- Worklog Id: (was: 854689) Time Spent: 0.5h (was: 20m) > Upgrade postgresql to 42.5.1 from 9.x > - > > Key: HIVE-27216 > URL: https://issues.apache.org/jira/browse/HIVE-27216 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > This ticket involves partial cherry pick of #HIVE-23965 and complete cherry > picks of HIVE-26253 and HIVE-26914 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27216) Upgrade postgresql to 42.5.1 from 9.x
[ https://issues.apache.org/jira/browse/HIVE-27216?focusedWorklogId=854688&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854688 ] ASF GitHub Bot logged work on HIVE-27216: - Author: ASF GitHub Bot Created on: 04/Apr/23 06:50 Start Date: 04/Apr/23 06:50 Worklog Time Spent: 10m Work Description: amanraj2520 commented on PR #4195: URL: https://github.com/apache/hive/pull/4195#issuecomment-1495438709 @zabetak Can you please review this. I have used a ticket raised by you in OSS master #HIVE-23965 but I did not cherry pick the entire ticket since there were a lot of merge conflicts because of the tests. The remaining tickets I have cherry-picked completely. Issue Time Tracking --- Worklog Id: (was: 854688) Time Spent: 20m (was: 10m) > Upgrade postgresql to 42.5.1 from 9.x > - > > Key: HIVE-27216 > URL: https://issues.apache.org/jira/browse/HIVE-27216 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > This ticket involves partial cherry pick of #HIVE-23965 and complete cherry > picks of HIVE-26253 and HIVE-26914 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27216) Upgrade postgresql to 42.5.1 from 9.x
[ https://issues.apache.org/jira/browse/HIVE-27216?focusedWorklogId=854687&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854687 ] ASF GitHub Bot logged work on HIVE-27216: - Author: ASF GitHub Bot Created on: 04/Apr/23 06:48 Start Date: 04/Apr/23 06:48 Worklog Time Spent: 10m Work Description: amanraj2520 opened a new pull request, #4195: URL: https://github.com/apache/hive/pull/4195 ### What changes were proposed in this pull request? Upgrade postgresql to 42.5.1 from 9.x in branch-3 ### Why are the changes needed? The changes are needed to fix critical CVE's like mentioned here https://mvnrepository.com/artifact/org.postgresql/postgresql ### Does this PR introduce _any_ user-facing change? No this just upgrades the postgres version ### How was this patch tested? Jenkins build Issue Time Tracking --- Worklog Id: (was: 854687) Remaining Estimate: 0h Time Spent: 10m > Upgrade postgresql to 42.5.1 from 9.x > - > > Key: HIVE-27216 > URL: https://issues.apache.org/jira/browse/HIVE-27216 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > This ticket involves partial cherry pick of #HIVE-23965 and complete cherry > picks of HIVE-26253 and HIVE-26914 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27216) Upgrade postgresql to 42.5.1 from 9.x
[ https://issues.apache.org/jira/browse/HIVE-27216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-27216: -- Labels: pull-request-available (was: ) > Upgrade postgresql to 42.5.1 from 9.x > - > > Key: HIVE-27216 > URL: https://issues.apache.org/jira/browse/HIVE-27216 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > This ticket involves partial cherry pick of #HIVE-23965 and complete cherry > picks of HIVE-26253 and HIVE-26914 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-27216) Upgrade postgresql to 42.5.1 from 9.x
[ https://issues.apache.org/jira/browse/HIVE-27216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aman Raj reassigned HIVE-27216: --- > Upgrade postgresql to 42.5.1 from 9.x > - > > Key: HIVE-27216 > URL: https://issues.apache.org/jira/browse/HIVE-27216 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > > This ticket involves partial cherry pick of #HIVE-23965 and complete cherry > picks of HIVE-26253 and HIVE-26914 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-26750) Component Upgrades for 3.2.0 release
[ https://issues.apache.org/jira/browse/HIVE-26750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aman Raj updated HIVE-26750: Parent: (was: HIVE-26748) Issue Type: Task (was: Sub-task) > Component Upgrades for 3.2.0 release > > > Key: HIVE-26750 > URL: https://issues.apache.org/jira/browse/HIVE-26750 > Project: Hive > Issue Type: Task >Affects Versions: 3.1.3 >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Blocker > Labels: hive-3.2.0-must > > Creating subtask for upgrading major component versions -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work started] (HIVE-26750) Component Upgrades for 3.2.0 release
[ https://issues.apache.org/jira/browse/HIVE-26750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-26750 started by Aman Raj. --- > Component Upgrades for 3.2.0 release > > > Key: HIVE-26750 > URL: https://issues.apache.org/jira/browse/HIVE-26750 > Project: Hive > Issue Type: Task >Affects Versions: 3.1.3 >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Blocker > Labels: hive-3.2.0-must > > Creating subtask for upgrading major component versions -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27205) Update jackson-databind for CVE fix for CVE-2022-42003
[ https://issues.apache.org/jira/browse/HIVE-27205?focusedWorklogId=854670&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854670 ] ASF GitHub Bot logged work on HIVE-27205: - Author: ASF GitHub Bot Created on: 04/Apr/23 06:00 Start Date: 04/Apr/23 06:00 Worklog Time Spent: 10m Work Description: ayushtkn commented on code in PR #4184: URL: https://github.com/apache/hive/pull/4184#discussion_r1156766247 ## pom.xml: ## @@ -148,7 +148,7 @@ 4.5.13 4.4.13 2.5.1 -2.12.7 +2.12.7.1 Review Comment: You are confused with jackson-databind version and jackson bom version, 2.12.7.1 is the version for jackson-databind https://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-databind/2.12.7.1 Issue Time Tracking --- Worklog Id: (was: 854670) Time Spent: 40m (was: 0.5h) > Update jackson-databind for CVE fix for CVE-2022-42003 > -- > > Key: HIVE-27205 > URL: https://issues.apache.org/jira/browse/HIVE-27205 > Project: Hive > Issue Type: Task >Reporter: Diksha >Assignee: Diksha >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > Update jackson-databind for CVE fix for CVE-2022-42003 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27205) Update jackson-databind for CVE fix for CVE-2022-42003
[ https://issues.apache.org/jira/browse/HIVE-27205?focusedWorklogId=854668&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854668 ] ASF GitHub Bot logged work on HIVE-27205: - Author: ASF GitHub Bot Created on: 04/Apr/23 05:45 Start Date: 04/Apr/23 05:45 Worklog Time Spent: 10m Work Description: abstractdog commented on PR #4184: URL: https://github.com/apache/hive/pull/4184#issuecomment-1495380276 here are the jackson-bom versions for instance: https://mvnrepository.com/artifact/com.fasterxml.jackson/jackson-bom I cannot see 2.12.7.1, is it a valid version? Issue Time Tracking --- Worklog Id: (was: 854668) Time Spent: 0.5h (was: 20m) > Update jackson-databind for CVE fix for CVE-2022-42003 > -- > > Key: HIVE-27205 > URL: https://issues.apache.org/jira/browse/HIVE-27205 > Project: Hive > Issue Type: Task >Reporter: Diksha >Assignee: Diksha >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Update jackson-databind for CVE fix for CVE-2022-42003 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27201) Inconsistency between session Hive and thread-local Hive may cause HS2 deadlock
[ https://issues.apache.org/jira/browse/HIVE-27201?focusedWorklogId=854654&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854654 ] ASF GitHub Bot logged work on HIVE-27201: - Author: ASF GitHub Bot Created on: 04/Apr/23 02:55 Start Date: 04/Apr/23 02:55 Worklog Time Spent: 10m Work Description: dengzhhu653 commented on PR #4180: URL: https://github.com/apache/hive/pull/4180#issuecomment-1495271139 Hello @scarlin-cloudera, @sankarh cloud you please take a look? Issue Time Tracking --- Worklog Id: (was: 854654) Time Spent: 2h 50m (was: 2h 40m) > Inconsistency between session Hive and thread-local Hive may cause HS2 > deadlock > --- > > Key: HIVE-27201 > URL: https://issues.apache.org/jira/browse/HIVE-27201 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Major > Labels: pull-request-available > Time Spent: 2h 50m > Remaining Estimate: 0h > > The HiveServer2’s server handler can switch to process the operation from > other session, in such case, the Hive cached in ThreadLocal is not the same > as the Hive in SessionState, and can be referenced by another session. > If the two handlers swap their sessions to process the DatabaseMetaData > request, and the HiveMetastoreClientFactory obtains the Hive via Hive.get(), > then there is a chance that the deadlock can happen. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26400) Provide docker images for Hive
[ https://issues.apache.org/jira/browse/HIVE-26400?focusedWorklogId=854649&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854649 ] ASF GitHub Bot logged work on HIVE-26400: - Author: ASF GitHub Bot Created on: 04/Apr/23 01:59 Start Date: 04/Apr/23 01:59 Worklog Time Spent: 10m Work Description: dengzhhu653 commented on PR #3448: URL: https://github.com/apache/hive/pull/3448#issuecomment-1495231356 > > > Seems like the build is broken. @deniskuzZ Could you please re-start? > > > > > > A fork of this gets a green run: #4133. I think the broken build is may due to some time consuming tests running in the same split. > > Do you need any help? Thank you @TuroczyX. The build would get a green run if I open another jira, I think the fix doesn't change any codes, nor build, so it's safe to go into master if the change itself looks fine, we can ignore the build failure in such case. Issue Time Tracking --- Worklog Id: (was: 854649) Time Spent: 12.5h (was: 12h 20m) > Provide docker images for Hive > -- > > Key: HIVE-26400 > URL: https://issues.apache.org/jira/browse/HIVE-26400 > Project: Hive > Issue Type: Sub-task > Components: Build Infrastructure >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Blocker > Labels: hive-4.0.0-must, pull-request-available > Time Spent: 12.5h > Remaining Estimate: 0h > > Make Apache Hive be able to run inside docker container in pseudo-distributed > mode, with MySQL/Derby as its back database, provide the following: > * Quick-start/Debugging/Prepare a test env for Hive; > * Tools to build target image with specified version of Hive and its > dependencies; > * Images can be used as the basis for the Kubernetes operator. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-27194) Support expression in limit and offset clauses
[ https://issues.apache.org/jira/browse/HIVE-27194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17708185#comment-17708185 ] vamshi kolanu commented on HIVE-27194: -- Thanks for the review [~zabetak]. Currently, different databases behave differently. As part of this PR, we want to start supporting constant expressions in limit and offset clauses for Hive. Limit and offset clauses support: ||Database||Integers ||Constant expressions|| Complex Expressions|| |Current Hive |Yes|No|No| |Impala|Yes|Yes|No| |Postgres|Yes|Yes|Yes| |Snowflake|Yes|No |No| |Mysql|Yes|No |No| > Support expression in limit and offset clauses > -- > > Key: HIVE-27194 > URL: https://issues.apache.org/jira/browse/HIVE-27194 > Project: Hive > Issue Type: Task > Components: Hive >Reporter: vamshi kolanu >Assignee: vamshi kolanu >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > As part of this task, support expressions in both limit and offset clauses. > Currently, these clauses are only supporting integers. > For example: The following expressions will be supported after this change. > 1. select key from (select * from src limit (1+2*3)) q1; > 2. select key from (select * from src limit (1+2*3) offset (3*4*5)) q1; -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26985) Create a trackable hive configuration object
[ https://issues.apache.org/jira/browse/HIVE-26985?focusedWorklogId=854641&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854641 ] ASF GitHub Bot logged work on HIVE-26985: - Author: ASF GitHub Bot Created on: 04/Apr/23 00:19 Start Date: 04/Apr/23 00:19 Worklog Time Spent: 10m Work Description: github-actions[bot] commented on PR #4002: URL: https://github.com/apache/hive/pull/4002#issuecomment-1495163372 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the d...@hive.apache.org list if the patch is in need of reviews. Issue Time Tracking --- Worklog Id: (was: 854641) Time Spent: 50m (was: 40m) > Create a trackable hive configuration object > > > Key: HIVE-26985 > URL: https://issues.apache.org/jira/browse/HIVE-26985 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Attachments: hive.log > > Time Spent: 50m > Remaining Estimate: 0h > > During configuration-related investigations, I want to be able to easily find > out when and how a certain configuration is changed. I'm looking for an > improvement that simply logs if "hive.a.b.c" is changed from "hello" to > "asdf" or even null and on which thread/codepath. > Not sure if there is already a trackable configuration object in hadoop that > we can reuse, or we need to implement it in hive. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27186) A persistent property store
[ https://issues.apache.org/jira/browse/HIVE-27186?focusedWorklogId=854640&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854640 ] ASF GitHub Bot logged work on HIVE-27186: - Author: ASF GitHub Bot Created on: 03/Apr/23 23:50 Start Date: 03/Apr/23 23:50 Worklog Time Spent: 10m Work Description: nrg4878 commented on PR #4194: URL: https://github.com/apache/hive/pull/4194#issuecomment-1495136665 @vihangk1 @pvary @zabetak @kasakrisz @dengzhhu653 Could I please request a code review for this feature in HMS? There are unit tests as well. Any help much appreciated. Thank you in advance Issue Time Tracking --- Worklog Id: (was: 854640) Time Spent: 20m (was: 10m) > A persistent property store > > > Key: HIVE-27186 > URL: https://issues.apache.org/jira/browse/HIVE-27186 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 4.0.0-alpha-2 >Reporter: Henri Biestro >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > WHAT > A persistent property store usable as a support facility for any metadata > augmentation feature. > WHY > When adding new meta-data oriented features, we usually need to persist > information linking the feature data and the HiveMetaStore objects it applies > to. Any information related to a database, a table or the cluster - like > statistics for example or any operational data state or data (think rolling > backup) - fall in this use-case. > Typically, accommodating such a feature requires modifying the Metastore > database schema by adding or altering a table. It also usually implies > modifying the thrift APIs to expose such meta-data to consumers. > The proposed feature wants to solve the persistence and query/transport for > these types of use-cases by exposing a 'key/(meta)value' store exposed as a > property system. > HOW > A property-value model is the simple and generic exposed API. > To provision for several usage scenarios, the model entry point is a > 'namespace' that qualifies the feature-component property manager. For > example, 'stats' could be the namespace for all properties related to the > 'statistics' feature. > The namespace identifies a manager that handles property-groups persisted as > property-maps. For instance, all statistics pertaining to a given table would > be collocated in the same property-group. As such, all properties (say number > of 'unique_values' per columns) for a given HMS table 'relation0' would all > be stored and persisted in the same property-map instance. > Property-maps may be decorated by an (optional) schema that may declare the > name and value-type of allowed properties (and their optional default value). > Each property is addressed by a name, a path uniquely identifying the > property in a given property map. > The manager also handles transforming property-map names to the property-map > keys used to persist them in the DB. > The API provides inserting/updating properties in bulk transactionally. It > also provides selection/projection to help reduce the volume of exchange > between client/server; selection can use (JEXL expression) predicates to > filter maps. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27180) Remove JsonSerde from hcatalog, Upgrade should update changed FQN for JsonSerDe in HMS DB
[
https://issues.apache.org/jira/browse/HIVE-27180?focusedWorklogId=854639&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854639
]
ASF GitHub Bot logged work on HIVE-27180:
-
Author: ASF GitHub Bot
Created on: 03/Apr/23 23:36
Start Date: 03/Apr/23 23:36
Worklog Time Spent: 10m
Work Description: sonarcloud[bot] commented on PR #4159:
URL: https://github.com/apache/hive/pull/4159#issuecomment-1495125701
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4159)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4159&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4159&resolved=false&types=BUG)
[1
Bug](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4159&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4159&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4159&resolved=false&types=VULNERABILITY)
[0
Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4159&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4159&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4159&resolved=false&types=SECURITY_HOTSPOT)
[0 Security
Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4159&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4159&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4159&resolved=false&types=CODE_SMELL)
[1 Code
Smell](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4159&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4159&metric=coverage&view=list)
No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4159&metric=duplicated_lines_density&view=list)
No Duplication information
Issue Time Tracking
---
Worklog Id: (was: 854639)
Time Spent: 2h (was: 1h 50m)
> Remove JsonSerde from hcatalog, Upgrade should update changed FQN for
> JsonSerDe in HMS DB
> --
>
> Key: HIVE-27180
> URL: https://issues.apache.org/jira/browse/HIVE-27180
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Riju Trivedi
>Assignee: Riju Trivedi
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h
> Remaining Estimate: 0h
>
> As Hcatalog JsonSerDe uses the "serde2" version as a back end, Remove
> o{*}rg.apache.hive.hcatalog.data.JsonSerDe{*} from hive-hcatalog. Fix tests
> to use the new Serde class org.apache.hadoop.hive.serde2.JsonSerDe. Hive
> Upgrade schema script can update the SERDES table to alter the class name to
> the new class name, the old tables would work automatically.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26400) Provide docker images for Hive
[ https://issues.apache.org/jira/browse/HIVE-26400?focusedWorklogId=854638&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854638 ] ASF GitHub Bot logged work on HIVE-26400: - Author: ASF GitHub Bot Created on: 03/Apr/23 23:27 Start Date: 03/Apr/23 23:27 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #3448: URL: https://github.com/apache/hive/pull/3448#issuecomment-1495118727 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3448) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3448&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3448&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3448&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3448&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3448&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3448&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3448&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3448&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3448&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3448&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3448&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3448&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3448&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3448&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 854638) Time Spent: 12h 20m (was: 12h 10m) > Provide docker images for Hive > -- > > Key: HIVE-26400 > URL: https://issues.apache.org/jira/browse/HIVE-26400 > Project: Hive > Issue Type: Sub-task > Components: Build Infrastructure >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Blocker > Labels: hive-4.0.0-must, pull-request-available > Time Spent: 12h 20m > Remaining Estimate: 0h > > Make Apache Hive be able to run inside docker container in pseudo-distributed > mode, with MySQL/Derby as its back database, provide the following: > * Quick-start/Debugging/Prepare a test env for Hive; > * Tools to build target image with specified version of Hive and its > dependencies; > * Images can be used as the basis for the Kubernetes operator. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26997) Iceberg: Vectorization gets disabled at runtime in merge-into statements
[
https://issues.apache.org/jira/browse/HIVE-26997?focusedWorklogId=854634&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854634
]
ASF GitHub Bot logged work on HIVE-26997:
-
Author: ASF GitHub Bot
Created on: 03/Apr/23 22:26
Start Date: 03/Apr/23 22:26
Worklog Time Spent: 10m
Work Description: sonarcloud[bot] commented on PR #4162:
URL: https://github.com/apache/hive/pull/4162#issuecomment-1495066438
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4162)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4162&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4162&resolved=false&types=BUG)
[0
Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4162&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4162&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4162&resolved=false&types=VULNERABILITY)
[0
Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4162&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4162&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4162&resolved=false&types=SECURITY_HOTSPOT)
[0 Security
Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4162&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4162&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4162&resolved=false&types=CODE_SMELL)
[0 Code
Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4162&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4162&metric=coverage&view=list)
No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4162&metric=duplicated_lines_density&view=list)
No Duplication information
Issue Time Tracking
---
Worklog Id: (was: 854634)
Time Spent: 3h (was: 2h 50m)
> Iceberg: Vectorization gets disabled at runtime in merge-into statements
>
>
> Key: HIVE-26997
> URL: https://issues.apache.org/jira/browse/HIVE-26997
> Project: Hive
> Issue Type: Improvement
> Components: Iceberg integration
>Reporter: Rajesh Balamohan
>Assignee: Zsolt Miskolczi
>Priority: Major
> Labels: pull-request-available
> Attachments: explain_merge_into.txt
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
> *Query:*
> Think of "ssv" table as a table containing trickle feed data in the following
> query. "store_sales_delete_1" is the destination table.
>
> {noformat}
> MERGE INTO tpcds_1000_iceberg_mor_v4.store_sales_delete_1 t USING
> tpcds_1000_update.ssv s ON (t.ss_item_sk = s.ss_item_sk
>
> AND t.ss_customer_sk=s.ss_customer_sk
>
[jira] [Work logged] (HIVE-26537) Deprecate older APIs in the HMS
[ https://issues.apache.org/jira/browse/HIVE-26537?focusedWorklogId=854633&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854633 ] ASF GitHub Bot logged work on HIVE-26537: - Author: ASF GitHub Bot Created on: 03/Apr/23 21:53 Start Date: 03/Apr/23 21:53 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #3599: URL: https://github.com/apache/hive/pull/3599#issuecomment-1495035163 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3599) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3599&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3599&resolved=false&types=BUG) [1 Bug](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3599&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3599&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3599&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3599&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3599&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3599&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3599&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3599&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3599&resolved=false&types=CODE_SMELL) [91 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3599&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3599&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3599&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 854633) Time Spent: 6.5h (was: 6h 20m) > Deprecate older APIs in the HMS > --- > > Key: HIVE-26537 > URL: https://issues.apache.org/jira/browse/HIVE-26537 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0-alpha-1, 4.0.0-alpha-2 >Reporter: Sai Hemanth Gantasala >Assignee: Sai Hemanth Gantasala >Priority: Critical > Labels: hive-4.0.0-must, pull-request-available > Time Spent: 6.5h > Remaining Estimate: 0h > > This Jira is to track the clean-up(deprecate older APIs and point the HMS > client to the newer APIs) work in the hive metastore server. > More details will be added here soon. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27135) AcidUtils#getHdfsDirSnapshots() throws FNFE when a directory is removed in HDFS
[
https://issues.apache.org/jira/browse/HIVE-27135?focusedWorklogId=854625&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854625
]
ASF GitHub Bot logged work on HIVE-27135:
-
Author: ASF GitHub Bot
Created on: 03/Apr/23 20:31
Start Date: 03/Apr/23 20:31
Worklog Time Spent: 10m
Work Description: sonarcloud[bot] commented on PR #4114:
URL: https://github.com/apache/hive/pull/4114#issuecomment-1494943052
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4114)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4114&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4114&resolved=false&types=BUG)
[0
Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4114&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4114&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4114&resolved=false&types=VULNERABILITY)
[0
Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4114&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4114&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4114&resolved=false&types=SECURITY_HOTSPOT)
[0 Security
Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4114&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4114&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4114&resolved=false&types=CODE_SMELL)
[0 Code
Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4114&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4114&metric=coverage&view=list)
No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4114&metric=duplicated_lines_density&view=list)
No Duplication information
Issue Time Tracking
---
Worklog Id: (was: 854625)
Time Spent: 7h 20m (was: 7h 10m)
> AcidUtils#getHdfsDirSnapshots() throws FNFE when a directory is removed in
> HDFS
> ---
>
> Key: HIVE-27135
> URL: https://issues.apache.org/jira/browse/HIVE-27135
> Project: Hive
> Issue Type: Bug
>Reporter: Dayakar M
>Assignee: Dayakar M
>Priority: Major
> Labels: pull-request-available
> Time Spent: 7h 20m
> Remaining Estimate: 0h
>
> AcidUtils#getHdfsDirSnapshots() throws FileNotFoundException when a directory
> is removed in HDFS while fetching HDFS Snapshots.
> Below testcode can be used to reproduce this issue.
> {code:java}
> @Test
> public void
> testShouldNotThrowFNFEWhenHiveStagingDirectoryIsRemovedWhileFetchingHDFSSnapshots()
> throws Exception {
> MockFileSystem fs = new MockFileSystem(new HiveConf(),
> new MockFile("mock:/tbl/part1/.hive-staging_dir/-ext-10002", 500, new
> byte[0]),
> new MockFile("mock:/tbl/part2/.hive-staging_dir", 500, new byte[0])
[jira] [Work logged] (HIVE-27177) Add alter table...Convert to Iceberg command
[
https://issues.apache.org/jira/browse/HIVE-27177?focusedWorklogId=854616&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854616
]
ASF GitHub Bot logged work on HIVE-27177:
-
Author: ASF GitHub Bot
Created on: 03/Apr/23 19:37
Start Date: 03/Apr/23 19:37
Worklog Time Spent: 10m
Work Description: sonarcloud[bot] commented on PR #4155:
URL: https://github.com/apache/hive/pull/4155#issuecomment-1494870215
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4155)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4155&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4155&resolved=false&types=BUG)
[0
Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4155&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4155&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4155&resolved=false&types=VULNERABILITY)
[0
Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4155&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4155&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4155&resolved=false&types=SECURITY_HOTSPOT)
[0 Security
Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4155&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4155&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4155&resolved=false&types=CODE_SMELL)
[3 Code
Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4155&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4155&metric=coverage&view=list)
No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4155&metric=duplicated_lines_density&view=list)
No Duplication information
Issue Time Tracking
---
Worklog Id: (was: 854616)
Time Spent: 1h 20m (was: 1h 10m)
> Add alter table...Convert to Iceberg command
>
>
> Key: HIVE-27177
> URL: https://issues.apache.org/jira/browse/HIVE-27177
> Project: Hive
> Issue Type: Improvement
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Add an alter table convert to Iceberg [TBLPROPERTIES('','')] to
> convert exiting external tables to iceberg tables
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26997) Iceberg: Vectorization gets disabled at runtime in merge-into statements
[
https://issues.apache.org/jira/browse/HIVE-26997?focusedWorklogId=854614&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854614
]
ASF GitHub Bot logged work on HIVE-26997:
-
Author: ASF GitHub Bot
Created on: 03/Apr/23 19:23
Start Date: 03/Apr/23 19:23
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #4162:
URL: https://github.com/apache/hive/pull/4162#discussion_r1156368895
##
ql/src/java/org/apache/hadoop/hive/ql/parse/RewriteSemanticAnalyzer.java:
##
@@ -670,19 +670,25 @@ public List getSortKeys() {
}
protected static class NonNativeAcidColumnAppender extends ColumnAppender {
+private String deletePrefix;
Review Comment:
yep
Issue Time Tracking
---
Worklog Id: (was: 854614)
Time Spent: 2h 50m (was: 2h 40m)
> Iceberg: Vectorization gets disabled at runtime in merge-into statements
>
>
> Key: HIVE-26997
> URL: https://issues.apache.org/jira/browse/HIVE-26997
> Project: Hive
> Issue Type: Improvement
> Components: Iceberg integration
>Reporter: Rajesh Balamohan
>Assignee: Zsolt Miskolczi
>Priority: Major
> Labels: pull-request-available
> Attachments: explain_merge_into.txt
>
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> *Query:*
> Think of "ssv" table as a table containing trickle feed data in the following
> query. "store_sales_delete_1" is the destination table.
>
> {noformat}
> MERGE INTO tpcds_1000_iceberg_mor_v4.store_sales_delete_1 t USING
> tpcds_1000_update.ssv s ON (t.ss_item_sk = s.ss_item_sk
>
> AND t.ss_customer_sk=s.ss_customer_sk
>
> AND t.ss_sold_date_sk = "2451181"
>
> AND ((Floor((s.ss_item_sk) / 1000) * 1000) BETWEEN 1000 AND
> 2000)
>
> AND s.ss_ext_discount_amt < 0.0) WHEN matched
> AND t.ss_ext_discount_amt IS NULL THEN
> UPDATE
> SET ss_ext_discount_amt = 0.0 WHEN NOT matched THEN
> INSERT (ss_sold_time_sk,
> ss_item_sk,
> ss_customer_sk,
> ss_cdemo_sk,
> ss_hdemo_sk,
> ss_addr_sk,
> ss_store_sk,
> ss_promo_sk,
> ss_ticket_number,
> ss_quantity,
> ss_wholesale_cost,
> ss_list_price,
> ss_sales_price,
> ss_ext_discount_amt,
> ss_ext_sales_price,
> ss_ext_wholesale_cost,
> ss_ext_list_price,
> ss_ext_tax,
> ss_coupon_amt,
> ss_net_paid,
> ss_net_paid_inc_tax,
> ss_net_profit,
> ss_sold_date_sk)
> VALUES (s.ss_sold_time_sk,
> s.ss_item_sk,
> s.ss_customer_sk,
> s.ss_cdemo_sk,
> s.ss_hdemo_sk,
> s.ss_addr_sk,
> s.ss_store_sk,
> s.ss_promo_sk,
> s.ss_ticket_number,
> s.ss_quantity,
> s.ss_wholesale_cost,
> s.ss_list_price,
> s.ss_sales_price,
> s.ss_ext_discount_amt,
> s.ss_ext_sales_price,
> s.ss_ext_wholesale_cost,
> s.ss_ext_list_price,
> s.ss_ext_tax,
> s.ss_coupon_amt,
> s.ss_net_paid,
> s.ss_net_paid_inc_tax,
> s.ss_net_profit,
> "2451181")
> {noformat}
>
>
> *Issue:*
> # Map phase is not getting vectorized due to "PARTITION_{_}SPEC{_}_ID" column
> {noformat}
> Map notVectorizedReason: Select expression for SELECT operator: Virtual
> column PARTITION__SPEC__ID is not supported {noformat}
>
> 2. "Reducer 2" stage isn't vectorized.
> {noformat}
> Reduce notVectorizedReason: exception: java.lang.RuntimeException: Full Outer
> Small Table Key Mapping duplicate column 0 in ordered column map {0=(value
> column: 30, type info: int), 1=(value column: 31, type info: int)} when
> adding value column 53, type into int stack trace:
> org.apache.hadoop.hive.ql.exec.vector.VectorColumnOrderedMap.add(VectorColumnOrderedMap.java:102),
>
> org.apache.hadoop.hive.ql.exec.vector.VectorColumnSourceMapping.add(VectorColumnSourceMapping.java:41),
>
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.canSpecializeMapJoin(Vectorizer.java:3865),
>
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.validateAndVectorizeOperator(Vectorizer.java:5246),
>
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.doProcessChild(Vectorizer.java:988),
>
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.doProcessChildre
[jira] [Work logged] (HIVE-27032) Introduce liquibase for HMS schema evolution
[ https://issues.apache.org/jira/browse/HIVE-27032?focusedWorklogId=854607&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854607 ] ASF GitHub Bot logged work on HIVE-27032: - Author: ASF GitHub Bot Created on: 03/Apr/23 19:02 Start Date: 03/Apr/23 19:02 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4060: URL: https://github.com/apache/hive/pull/4060#issuecomment-1494826493 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4060) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=BUG) [12 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4060&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4060&resolved=false&types=SECURITY_HOTSPOT) [4 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4060&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=CODE_SMELL) [207 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4060&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4060&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 854607) Time Spent: 1h 20m (was: 1h 10m) > Introduce liquibase for HMS schema evolution > > > Key: HIVE-27032 > URL: https://issues.apache.org/jira/browse/HIVE-27032 > Project: Hive > Issue Type: Improvement >Reporter: László Végh >Assignee: László Végh >Priority: Major > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > > Introduce liquibase, and replace current upgrade procedure with it. > The Schematool CLI API should remain untouched, while under the hood, > liquibase should be used for HMS schema evolution. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26400) Provide docker images for Hive
[ https://issues.apache.org/jira/browse/HIVE-26400?focusedWorklogId=854603&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854603 ] ASF GitHub Bot logged work on HIVE-26400: - Author: ASF GitHub Bot Created on: 03/Apr/23 18:26 Start Date: 03/Apr/23 18:26 Worklog Time Spent: 10m Work Description: TuroczyX commented on PR #3448: URL: https://github.com/apache/hive/pull/3448#issuecomment-1494781322 > > Seems like the build is broken. @deniskuzZ Could you please re-start? > > A fork of this gets a green run: #4133. I think the broken build is may due to some time consuming tests running in the same split. Do you need any help? Issue Time Tracking --- Worklog Id: (was: 854603) Time Spent: 12h 10m (was: 12h) > Provide docker images for Hive > -- > > Key: HIVE-26400 > URL: https://issues.apache.org/jira/browse/HIVE-26400 > Project: Hive > Issue Type: Sub-task > Components: Build Infrastructure >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Blocker > Labels: hive-4.0.0-must, pull-request-available > Time Spent: 12h 10m > Remaining Estimate: 0h > > Make Apache Hive be able to run inside docker container in pseudo-distributed > mode, with MySQL/Derby as its back database, provide the following: > * Quick-start/Debugging/Prepare a test env for Hive; > * Tools to build target image with specified version of Hive and its > dependencies; > * Images can be used as the basis for the Kubernetes operator. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27164) Create Temp Txn Table As Select is failing at tablePath validation
[
https://issues.apache.org/jira/browse/HIVE-27164?focusedWorklogId=854599&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854599
]
ASF GitHub Bot logged work on HIVE-27164:
-
Author: ASF GitHub Bot
Created on: 03/Apr/23 17:41
Start Date: 03/Apr/23 17:41
Worklog Time Spent: 10m
Work Description: sonarcloud[bot] commented on PR #4176:
URL: https://github.com/apache/hive/pull/4176#issuecomment-1494725600
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4176)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4176&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4176&resolved=false&types=BUG)
[0
Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4176&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4176&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4176&resolved=false&types=VULNERABILITY)
[0
Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4176&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4176&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4176&resolved=false&types=SECURITY_HOTSPOT)
[0 Security
Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4176&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4176&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4176&resolved=false&types=CODE_SMELL)
[4 Code
Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4176&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4176&metric=coverage&view=list)
No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4176&metric=duplicated_lines_density&view=list)
No Duplication information
Issue Time Tracking
---
Worklog Id: (was: 854599)
Time Spent: 2h 20m (was: 2h 10m)
> Create Temp Txn Table As Select is failing at tablePath validation
> --
>
> Key: HIVE-27164
> URL: https://issues.apache.org/jira/browse/HIVE-27164
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, Metastore
>Reporter: Naresh P R
>Assignee: Venugopal Reddy K
>Priority: Major
> Labels: pull-request-available
> Attachments: mm_cttas.q
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> After HIVE-25303, every CTAS goes for
> HiveMetaStore$HMSHandler#translate_table_dryrun() call to fetch table
> location for CTAS queries which fails with following exception for temp
> tables if MetastoreDefaultTransformer is set.
> {code:java}
> 2023-03-17 16:41:23,390 INFO
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer:
> [pool-6-thread-196]: Starting translation for CreateTable for processor
> HMSClient-@localhost with [EXTWRITE, EXTREAD, HIVEBUCKET2, HIVEFULLACIDREAD,
[jira] [Work logged] (HIVE-27200) Backport HIVE-24928 to branch-3
[ https://issues.apache.org/jira/browse/HIVE-27200?focusedWorklogId=854595&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854595 ] ASF GitHub Bot logged work on HIVE-27200: - Author: ASF GitHub Bot Created on: 03/Apr/23 17:28 Start Date: 03/Apr/23 17:28 Worklog Time Spent: 10m Work Description: yigress commented on PR #4175: URL: https://github.com/apache/hive/pull/4175#issuecomment-1494709007 @lcspinter original author, can you help review this for branch-3? Issue Time Tracking --- Worklog Id: (was: 854595) Time Spent: 20m (was: 10m) > Backport HIVE-24928 to branch-3 > --- > > Key: HIVE-27200 > URL: https://issues.apache.org/jira/browse/HIVE-27200 > Project: Hive > Issue Type: Improvement > Components: StorageHandler >Reporter: Yi Zhang >Assignee: Yi Zhang >Priority: Critical > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > This is to backport HIVE-24928 so that for HiveStorageHandler table 'ANALYZE > TABLE ... COMPUTE STATISTICS' can use storagehandler to provide basic stats > with BasicStatsNoJobTask -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26997) Iceberg: Vectorization gets disabled at runtime in merge-into statements
[
https://issues.apache.org/jira/browse/HIVE-26997?focusedWorklogId=854594&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854594
]
ASF GitHub Bot logged work on HIVE-26997:
-
Author: ASF GitHub Bot
Created on: 03/Apr/23 17:23
Start Date: 03/Apr/23 17:23
Worklog Time Spent: 10m
Work Description: sonarcloud[bot] commented on PR #4162:
URL: https://github.com/apache/hive/pull/4162#issuecomment-1494704131
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4162)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4162&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4162&resolved=false&types=BUG)
[0
Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4162&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4162&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4162&resolved=false&types=VULNERABILITY)
[0
Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4162&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4162&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4162&resolved=false&types=SECURITY_HOTSPOT)
[0 Security
Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4162&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4162&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4162&resolved=false&types=CODE_SMELL)
[0 Code
Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4162&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4162&metric=coverage&view=list)
No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4162&metric=duplicated_lines_density&view=list)
No Duplication information
Issue Time Tracking
---
Worklog Id: (was: 854594)
Time Spent: 2h 40m (was: 2.5h)
> Iceberg: Vectorization gets disabled at runtime in merge-into statements
>
>
> Key: HIVE-26997
> URL: https://issues.apache.org/jira/browse/HIVE-26997
> Project: Hive
> Issue Type: Improvement
> Components: Iceberg integration
>Reporter: Rajesh Balamohan
>Assignee: Zsolt Miskolczi
>Priority: Major
> Labels: pull-request-available
> Attachments: explain_merge_into.txt
>
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> *Query:*
> Think of "ssv" table as a table containing trickle feed data in the following
> query. "store_sales_delete_1" is the destination table.
>
> {noformat}
> MERGE INTO tpcds_1000_iceberg_mor_v4.store_sales_delete_1 t USING
> tpcds_1000_update.ssv s ON (t.ss_item_sk = s.ss_item_sk
>
> AND t.ss_customer_sk=s.ss_customer_sk
>
[jira] [Updated] (HIVE-27186) A persistent property store
[ https://issues.apache.org/jira/browse/HIVE-27186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-27186: -- Labels: pull-request-available (was: ) > A persistent property store > > > Key: HIVE-27186 > URL: https://issues.apache.org/jira/browse/HIVE-27186 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 4.0.0-alpha-2 >Reporter: Henri Biestro >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > WHAT > A persistent property store usable as a support facility for any metadata > augmentation feature. > WHY > When adding new meta-data oriented features, we usually need to persist > information linking the feature data and the HiveMetaStore objects it applies > to. Any information related to a database, a table or the cluster - like > statistics for example or any operational data state or data (think rolling > backup) - fall in this use-case. > Typically, accommodating such a feature requires modifying the Metastore > database schema by adding or altering a table. It also usually implies > modifying the thrift APIs to expose such meta-data to consumers. > The proposed feature wants to solve the persistence and query/transport for > these types of use-cases by exposing a 'key/(meta)value' store exposed as a > property system. > HOW > A property-value model is the simple and generic exposed API. > To provision for several usage scenarios, the model entry point is a > 'namespace' that qualifies the feature-component property manager. For > example, 'stats' could be the namespace for all properties related to the > 'statistics' feature. > The namespace identifies a manager that handles property-groups persisted as > property-maps. For instance, all statistics pertaining to a given table would > be collocated in the same property-group. As such, all properties (say number > of 'unique_values' per columns) for a given HMS table 'relation0' would all > be stored and persisted in the same property-map instance. > Property-maps may be decorated by an (optional) schema that may declare the > name and value-type of allowed properties (and their optional default value). > Each property is addressed by a name, a path uniquely identifying the > property in a given property map. > The manager also handles transforming property-map names to the property-map > keys used to persist them in the DB. > The API provides inserting/updating properties in bulk transactionally. It > also provides selection/projection to help reduce the volume of exchange > between client/server; selection can use (JEXL expression) predicates to > filter maps. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27186) A persistent property store
[ https://issues.apache.org/jira/browse/HIVE-27186?focusedWorklogId=854591&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854591 ] ASF GitHub Bot logged work on HIVE-27186: - Author: ASF GitHub Bot Created on: 03/Apr/23 17:11 Start Date: 03/Apr/23 17:11 Worklog Time Spent: 10m Work Description: henrib opened a new pull request, #4194: URL: https://github.com/apache/hive/pull/4194 [https://issues.apache.org/jira/browse/HIVE-27186](https://issues.apache.org/jira/browse/HIVE-27186) A persistent property store usable as a support facility for any metadata augmentation feature. ### What changes were proposed in this pull request? A property-value model is the simple and generic exposed API. To provision for several usage scenarios, the model entry point is a 'namespace' that qualifies the feature-component property manager. For example, 'stats' could be the namespace for all properties related to the 'statistics' feature. The namespace identifies a manager that handles property-groups persisted as property-maps. For instance, all statistics pertaining to a given table would be collocated in the same property-group. As such, all properties (say number of 'unique_values' per columns) for a given HMS table 'relation0' would all be stored and persisted in the same property-map instance. Property-maps may be decorated by an (optional) schema that may declare the name and value-type of allowed properties (and their optional default value). Each property is addressed by a name, a path uniquely identifying the property in a given property map. The manager also handles transforming property-map names to the property-map keys used to persist them in the DB. The API provides inserting/updating properties in bulk transactionally. It also provides selection/projection to help reduce the volume of exchange between client/server; selection can use (JEXL expression) predicates to filter maps. ### Why are the changes needed? When adding new meta-data oriented features, we usually need to persist information linking the feature data and the HiveMetaStore objects it applies to. Any information related to a database, a table or the cluster - like statistics for example or any operational data state or data (think rolling backup) - fall in this use-case. Typically, accommodating such a feature requires modifying the Metastore database schema by adding or altering a table. It also usually implies modifying the thrift APIs to expose such meta-data to consumers. The proposed feature wants to solve the persistence and query/transport for these types of use-cases by exposing a 'key/(meta)value' store exposed as a property system. ### Does this PR introduce _any_ user-facing change? It introduces new API calls. ### How was this patch tested? Junit + coverage Issue Time Tracking --- Worklog Id: (was: 854591) Remaining Estimate: 0h Time Spent: 10m > A persistent property store > > > Key: HIVE-27186 > URL: https://issues.apache.org/jira/browse/HIVE-27186 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 4.0.0-alpha-2 >Reporter: Henri Biestro >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > WHAT > A persistent property store usable as a support facility for any metadata > augmentation feature. > WHY > When adding new meta-data oriented features, we usually need to persist > information linking the feature data and the HiveMetaStore objects it applies > to. Any information related to a database, a table or the cluster - like > statistics for example or any operational data state or data (think rolling > backup) - fall in this use-case. > Typically, accommodating such a feature requires modifying the Metastore > database schema by adding or altering a table. It also usually implies > modifying the thrift APIs to expose such meta-data to consumers. > The proposed feature wants to solve the persistence and query/transport for > these types of use-cases by exposing a 'key/(meta)value' store exposed as a > property system. > HOW > A property-value model is the simple and generic exposed API. > To provision for several usage scenarios, the model entry point is a > 'namespace' that qualifies the feature-component property manager. For > example, 'stats' could be the namespace for all properties related to the > 'statistics' feature. > The namespace identifies a manager that handles property-groups persisted as > property-maps. For instance, all statistics pertaining to a given table would > be collocated in the same property-group. As such, all properties (say number > o
[jira] [Work logged] (HIVE-27165) PART_COL_STATS metastore query not hitting the index
[ https://issues.apache.org/jira/browse/HIVE-27165?focusedWorklogId=854590&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854590 ] ASF GitHub Bot logged work on HIVE-27165: - Author: ASF GitHub Bot Created on: 03/Apr/23 17:07 Start Date: 03/Apr/23 17:07 Worklog Time Spent: 10m Work Description: DanielZhu58 commented on PR #4141: URL: https://github.com/apache/hive/pull/4141#issuecomment-1494683235 @nrg4878 @saihemanth-cloudera @dengzhhu653 Could you guys help to review this patch? Thank you very much. Issue Time Tracking --- Worklog Id: (was: 854590) Time Spent: 1h 20m (was: 1h 10m) > PART_COL_STATS metastore query not hitting the index > > > Key: HIVE-27165 > URL: https://issues.apache.org/jira/browse/HIVE-27165 > Project: Hive > Issue Type: Improvement >Reporter: Hongdan Zhu >Priority: Major > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > > The query located here: > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/CompactionTxnHandler.java#L1029-L1032] > is not hitting an index. The index contains CAT_NAME whereas this query does > not. This was a change made in Hive 3.0, I think. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27180) Remove JsonSerde from hcatalog, Upgrade should update changed FQN for JsonSerDe in HMS DB
[
https://issues.apache.org/jira/browse/HIVE-27180?focusedWorklogId=854587&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854587
]
ASF GitHub Bot logged work on HIVE-27180:
-
Author: ASF GitHub Bot
Created on: 03/Apr/23 16:57
Start Date: 03/Apr/23 16:57
Worklog Time Spent: 10m
Work Description: rtrivedi12 commented on code in PR #4159:
URL: https://github.com/apache/hive/pull/4159#discussion_r1156220405
##
hcatalog/core/src/test/java/org/apache/hive/hcatalog/data/TestJsonSerDe.java:
##
@@ -1,358 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-package org.apache.hive.hcatalog.data;
-
-import java.io.UnsupportedEncodingException;
-import java.math.BigDecimal;
-import java.util.ArrayList;
-import java.util.Arrays;
-import java.util.HashMap;
-import java.util.List;
-import java.util.Map;
-import java.util.Properties;
-
-import org.apache.hadoop.conf.Configuration;
-import org.apache.hadoop.hive.common.type.Date;
-import org.apache.hadoop.hive.common.type.HiveChar;
-import org.apache.hadoop.hive.common.type.HiveDecimal;
-import org.apache.hadoop.hive.common.type.HiveVarchar;
-import org.apache.hadoop.hive.common.type.Timestamp;
-import org.apache.hadoop.hive.conf.HiveConf;
-import org.apache.hadoop.hive.serde.serdeConstants;
-import org.apache.hadoop.io.Text;
-import org.apache.hadoop.io.Writable;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-
-import static org.junit.Assert.assertTrue;
-import org.junit.Test;
-
-/**
- * TestJsonSerDe.
- */
-public class TestJsonSerDe {
Review Comment:
Added Hcatalog Test suite to new serde2 TestJsonSerDe class. Replaced
Hcatalog classes with Gson classes for Json comparison.
Issue Time Tracking
---
Worklog Id: (was: 854587)
Time Spent: 1h 50m (was: 1h 40m)
> Remove JsonSerde from hcatalog, Upgrade should update changed FQN for
> JsonSerDe in HMS DB
> --
>
> Key: HIVE-27180
> URL: https://issues.apache.org/jira/browse/HIVE-27180
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Riju Trivedi
>Assignee: Riju Trivedi
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> As Hcatalog JsonSerDe uses the "serde2" version as a back end, Remove
> o{*}rg.apache.hive.hcatalog.data.JsonSerDe{*} from hive-hcatalog. Fix tests
> to use the new Serde class org.apache.hadoop.hive.serde2.JsonSerDe. Hive
> Upgrade schema script can update the SERDES table to alter the class name to
> the new class name, the old tables would work automatically.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27180) Remove JsonSerde from hcatalog, Upgrade should update changed FQN for JsonSerDe in HMS DB
[
https://issues.apache.org/jira/browse/HIVE-27180?focusedWorklogId=854586&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854586
]
ASF GitHub Bot logged work on HIVE-27180:
-
Author: ASF GitHub Bot
Created on: 03/Apr/23 16:56
Start Date: 03/Apr/23 16:56
Worklog Time Spent: 10m
Work Description: github-actions[bot] commented on PR #4159:
URL: https://github.com/apache/hive/pull/4159#issuecomment-1494667397
# @check-spelling-bot Report
### :red_circle: Please review
See the [files](4159/files/) view or the [action
log](https://github.com/apache/hive/actions/runs/4599267748) for details.
Unrecognized words (13)
```
bbb
empid
hcatalog
hcatandhadoop
hivechar
hivevarchar
nlist
retval
rjsd
rlist
wjsd
yyy
zzz
```
Previously acknowledged words that are now absent
aarry timestamplocal
To accept these unrecognized words as correct (and remove
the previously acknowledged and now absent words),
run the following commands
... in a clone of the
[g...@github.com:rtrivedi12/hive.git](https://github.com/rtrivedi12/hive.git)
repository
on the `HIVE-27180` branch:
```
update_files() {
perl -e '
my @expect_files=qw('".github/actions/spelling/expect.txt"');
@ARGV=@expect_files;
my @stale=qw('"$patch_remove"');
my $re=join "|", @stale;
my $suffix=".".time();
my $previous="";
sub maybe_unlink { unlink($_[0]) if $_[0]; }
while (<>) {
if ($ARGV ne $old_argv) { maybe_unlink($previous); $previous="$ARGV$suffix";
rename($ARGV, $previous); open(ARGV_OUT, ">$ARGV"); select(ARGV_OUT); $old_argv
= $ARGV; }
next if /^(?:$re)(?:(?:\r|\n)*$| .*)/; print;
}; maybe_unlink($previous);'
perl -e '
my $new_expect_file=".github/actions/spelling/expect.txt";
use File::Path qw(make_path);
use File::Basename qw(dirname);
make_path (dirname($new_expect_file));
open FILE, q{<}, $new_expect_file; chomp(my @words = ); close FILE;
my @add=qw('"$patch_add"');
my %items; @items{@words} = @words x (1); @items{@add} = @add x (1);
@words = sort {lc($a)."-".$a cmp lc($b)."-".$b} keys %items;
open FILE, q{>}, $new_expect_file; for my $word (@words) { print FILE
"$word\n" if $word =~ /\w/; };
close FILE;
system("git", "add", $new_expect_file);
'
}
comment_json=$(mktemp)
curl -L -s -S \
-H "Content-Type: application/json" \
"COMMENT_URL" > "$comment_json"
comment_body=$(mktemp)
jq -r ".body // empty" "$comment_json" > $comment_body
rm $comment_json
patch_remove=$(perl -ne 'next unless s{^(.*)$}{$1};
print' < "$comment_body")
patch_add=$(perl -e '$/=undef; $_=<>; if (m{Unrecognized
words[^<]*\n*```\n*([^<]*)```\n*$}m) { print "$1" } elsif
(m{Unrecognized words[^<]*\n\n((?:\w.*\n)+)\n}m) { print "$1" };' <
"$comment_body")
update_files
rm $comment_body
git add -u
```
Remove JsonSerde from hcatalog, Upgrade should update changed FQN for
> JsonSerDe in HMS DB
> --
>
> Key: HIVE-27180
> URL: https://issues.apache.org/jira/browse/HIVE-27180
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Riju Trivedi
>Assignee: Riju Trivedi
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> As Hcatalog JsonSerDe uses the "serde2" version as a back end, Remove
> o{*}rg.apache.hive.hcatalog.data.JsonSerDe{*} from hive-hcatalog. Fix tests
> to use the new Serde class org.apache.hadoop.hive.serde2.JsonSerDe. Hive
> Upgrade schema script can update the SERDES table to alter the class name to
> the new class name, the old tables would work automatically.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27180) Remove JsonSerde from hcatalog, Upgrade should update changed FQN for JsonSerDe in HMS DB
[
https://issues.apache.org/jira/browse/HIVE-27180?focusedWorklogId=854585&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854585
]
ASF GitHub Bot logged work on HIVE-27180:
-
Author: ASF GitHub Bot
Created on: 03/Apr/23 16:38
Start Date: 03/Apr/23 16:38
Worklog Time Spent: 10m
Work Description: github-actions[bot] commented on PR #4159:
URL: https://github.com/apache/hive/pull/4159#issuecomment-1494644600
# @check-spelling-bot Report
### :red_circle: Please review
See the [files](4159/files/) view or the [action
log](https://github.com/apache/hive/actions/runs/4599111515) for details.
Unrecognized words (13)
```
bbb
empid
hcatalog
hcatandhadoop
hivechar
hivevarchar
nlist
retval
rjsd
rlist
wjsd
yyy
zzz
```
Previously acknowledged words that are now absent
aarry timestamplocal
To accept these unrecognized words as correct (and remove
the previously acknowledged and now absent words),
run the following commands
... in a clone of the
[g...@github.com:rtrivedi12/hive.git](https://github.com/rtrivedi12/hive.git)
repository
on the `HIVE-27180` branch:
```
update_files() {
perl -e '
my @expect_files=qw('".github/actions/spelling/expect.txt"');
@ARGV=@expect_files;
my @stale=qw('"$patch_remove"');
my $re=join "|", @stale;
my $suffix=".".time();
my $previous="";
sub maybe_unlink { unlink($_[0]) if $_[0]; }
while (<>) {
if ($ARGV ne $old_argv) { maybe_unlink($previous); $previous="$ARGV$suffix";
rename($ARGV, $previous); open(ARGV_OUT, ">$ARGV"); select(ARGV_OUT); $old_argv
= $ARGV; }
next if /^(?:$re)(?:(?:\r|\n)*$| .*)/; print;
}; maybe_unlink($previous);'
perl -e '
my $new_expect_file=".github/actions/spelling/expect.txt";
use File::Path qw(make_path);
use File::Basename qw(dirname);
make_path (dirname($new_expect_file));
open FILE, q{<}, $new_expect_file; chomp(my @words = ); close FILE;
my @add=qw('"$patch_add"');
my %items; @items{@words} = @words x (1); @items{@add} = @add x (1);
@words = sort {lc($a)."-".$a cmp lc($b)."-".$b} keys %items;
open FILE, q{>}, $new_expect_file; for my $word (@words) { print FILE
"$word\n" if $word =~ /\w/; };
close FILE;
system("git", "add", $new_expect_file);
'
}
comment_json=$(mktemp)
curl -L -s -S \
-H "Content-Type: application/json" \
"COMMENT_URL" > "$comment_json"
comment_body=$(mktemp)
jq -r ".body // empty" "$comment_json" > $comment_body
rm $comment_json
patch_remove=$(perl -ne 'next unless s{^(.*)$}{$1};
print' < "$comment_body")
patch_add=$(perl -e '$/=undef; $_=<>; if (m{Unrecognized
words[^<]*\n*```\n*([^<]*)```\n*$}m) { print "$1" } elsif
(m{Unrecognized words[^<]*\n\n((?:\w.*\n)+)\n}m) { print "$1" };' <
"$comment_body")
update_files
rm $comment_body
git add -u
```
Remove JsonSerde from hcatalog, Upgrade should update changed FQN for
> JsonSerDe in HMS DB
> --
>
> Key: HIVE-27180
> URL: https://issues.apache.org/jira/browse/HIVE-27180
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Riju Trivedi
>Assignee: Riju Trivedi
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> As Hcatalog JsonSerDe uses the "serde2" version as a back end, Remove
> o{*}rg.apache.hive.hcatalog.data.JsonSerDe{*} from hive-hcatalog. Fix tests
> to use the new Serde class org.apache.hadoop.hive.serde2.JsonSerDe. Hive
> Upgrade schema script can update the SERDES table to alter the class name to
> the new class name, the old tables would work automatically.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27205) Update jackson-databind for CVE fix for CVE-2022-42003
[ https://issues.apache.org/jira/browse/HIVE-27205?focusedWorklogId=854582&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854582 ] ASF GitHub Bot logged work on HIVE-27205: - Author: ASF GitHub Bot Created on: 03/Apr/23 16:23 Start Date: 03/Apr/23 16:23 Worklog Time Spent: 10m Work Description: TuroczyX commented on PR #4184: URL: https://github.com/apache/hive/pull/4184#issuecomment-1494623970 > @zabetak , @abstractdog , @ayushtkn can one of you please approve the workflow builds for this PR ? It is failing :( Non-resolvable import POM: Could not find artifact com.fasterxml.jackson:jackson-bom:pom:2.12.7.1 in central (https://repo.maven.apache.org/maven2) @ line 733, column 19 -> [Help 2] Issue Time Tracking --- Worklog Id: (was: 854582) Time Spent: 20m (was: 10m) > Update jackson-databind for CVE fix for CVE-2022-42003 > -- > > Key: HIVE-27205 > URL: https://issues.apache.org/jira/browse/HIVE-27205 > Project: Hive > Issue Type: Task >Reporter: Diksha >Assignee: Diksha >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Update jackson-databind for CVE fix for CVE-2022-42003 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26809) Upgrade ORC to 1.8.3
[ https://issues.apache.org/jira/browse/HIVE-26809?focusedWorklogId=854580&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854580 ] ASF GitHub Bot logged work on HIVE-26809: - Author: ASF GitHub Bot Created on: 03/Apr/23 16:01 Start Date: 03/Apr/23 16:01 Worklog Time Spent: 10m Work Description: zhangbutao commented on PR #4121: URL: https://github.com/apache/hive/pull/4121#issuecomment-1494589561 Sure, i am also ok with this change. Maybe we can create a following ticket to mask orc file size, then we can easily upgrade orc version next time. Issue Time Tracking --- Worklog Id: (was: 854580) Time Spent: 11h 20m (was: 11h 10m) > Upgrade ORC to 1.8.3 > > > Key: HIVE-26809 > URL: https://issues.apache.org/jira/browse/HIVE-26809 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Dmitriy Fingerman >Assignee: Zoltán Rátkai >Priority: Major > Labels: pull-request-available > Time Spent: 11h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27187) Incremental rebuild of materialized view having aggregate and stored by iceberg
[ https://issues.apache.org/jira/browse/HIVE-27187?focusedWorklogId=854574&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854574 ] ASF GitHub Bot logged work on HIVE-27187: - Author: ASF GitHub Bot Created on: 03/Apr/23 15:53 Start Date: 03/Apr/23 15:53 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4166: URL: https://github.com/apache/hive/pull/4166#issuecomment-1494575502 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4166) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4166&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4166&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4166&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4166&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4166&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4166&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4166&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4166&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4166&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4166&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4166&resolved=false&types=CODE_SMELL) [3 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4166&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4166&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4166&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 854574) Time Spent: 2.5h (was: 2h 20m) > Incremental rebuild of materialized view having aggregate and stored by > iceberg > --- > > Key: HIVE-27187 > URL: https://issues.apache.org/jira/browse/HIVE-27187 > Project: Hive > Issue Type: Improvement > Components: Iceberg integration, Materialized views >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Time Spent: 2.5h > Remaining Estimate: 0h > > Currently incremental rebuild of materialized view stored by iceberg which > definition query contains aggregate operator is transformed to an insert > overwrite statement which contains a union operator if the source tables > contains insert operations only. One branch of the union scans the view the > other produces the delta. > This can be improved further: transform the statement to a multi insert > statement representing a merge statement to insert new aggregations and >
[jira] [Work logged] (HIVE-26809) Upgrade ORC to 1.8.3
[ https://issues.apache.org/jira/browse/HIVE-26809?focusedWorklogId=854573&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854573 ] ASF GitHub Bot logged work on HIVE-26809: - Author: ASF GitHub Bot Created on: 03/Apr/23 15:47 Start Date: 03/Apr/23 15:47 Worklog Time Spent: 10m Work Description: TuroczyX commented on PR #4121: URL: https://github.com/apache/hive/pull/4121#issuecomment-1494565865 > I think in a long term we should. Now it is fixed is fine. But you right pretty over checked. I guess we can create a following ticket. Because this sizes check does not protect the code, and It does not create to much value. Also with new ORC's the files would be always different and check manually is just beh. For the first upgrade I would be OK with these if it is already there. (The last upgrade was in 1972 :) Finally the Hive will have the latest one) Issue Time Tracking --- Worklog Id: (was: 854573) Time Spent: 11h 10m (was: 11h) > Upgrade ORC to 1.8.3 > > > Key: HIVE-26809 > URL: https://issues.apache.org/jira/browse/HIVE-26809 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Dmitriy Fingerman >Assignee: Zoltán Rátkai >Priority: Major > Labels: pull-request-available > Time Spent: 11h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27201) Inconsistency between session Hive and thread-local Hive may cause HS2 deadlock
[ https://issues.apache.org/jira/browse/HIVE-27201?focusedWorklogId=854565&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854565 ] ASF GitHub Bot logged work on HIVE-27201: - Author: ASF GitHub Bot Created on: 03/Apr/23 14:53 Start Date: 03/Apr/23 14:53 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4180: URL: https://github.com/apache/hive/pull/4180#issuecomment-1494475508 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4180) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4180&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4180&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4180&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4180&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4180&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4180&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4180&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4180&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4180&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4180&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4180&resolved=false&types=CODE_SMELL) [1 Code Smell](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4180&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4180&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4180&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 854565) Time Spent: 2h 40m (was: 2.5h) > Inconsistency between session Hive and thread-local Hive may cause HS2 > deadlock > --- > > Key: HIVE-27201 > URL: https://issues.apache.org/jira/browse/HIVE-27201 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Major > Labels: pull-request-available > Time Spent: 2h 40m > Remaining Estimate: 0h > > The HiveServer2’s server handler can switch to process the operation from > other session, in such case, the Hive cached in ThreadLocal is not the same > as the Hive in SessionState, and can be referenced by another session. > If the two handlers swap their sessions to process the DatabaseMetaData > request, and the HiveMetastoreClientFactory obtains the Hive via Hive.get(), > then there is a chance that the deadlock can happen. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27032) Introduce liquibase for HMS schema evolution
[ https://issues.apache.org/jira/browse/HIVE-27032?focusedWorklogId=854563&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854563 ] ASF GitHub Bot logged work on HIVE-27032: - Author: ASF GitHub Bot Created on: 03/Apr/23 14:37 Start Date: 03/Apr/23 14:37 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4060: URL: https://github.com/apache/hive/pull/4060#issuecomment-1494447775 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4060) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=BUG) [12 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4060&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4060&resolved=false&types=SECURITY_HOTSPOT) [4 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4060&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=CODE_SMELL) [207 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4060&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4060&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4060&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 854563) Time Spent: 1h 10m (was: 1h) > Introduce liquibase for HMS schema evolution > > > Key: HIVE-27032 > URL: https://issues.apache.org/jira/browse/HIVE-27032 > Project: Hive > Issue Type: Improvement >Reporter: László Végh >Assignee: László Végh >Priority: Major > Labels: pull-request-available > Time Spent: 1h 10m > Remaining Estimate: 0h > > Introduce liquibase, and replace current upgrade procedure with it. > The Schematool CLI API should remain untouched, while under the hood, > liquibase should be used for HMS schema evolution. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26809) Upgrade ORC to 1.8.3
[ https://issues.apache.org/jira/browse/HIVE-26809?focusedWorklogId=854558&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854558 ] ASF GitHub Bot logged work on HIVE-26809: - Author: ASF GitHub Bot Created on: 03/Apr/23 14:07 Start Date: 03/Apr/23 14:07 Worklog Time Spent: 10m Work Description: zhangbutao commented on PR #4121: URL: https://github.com/apache/hive/pull/4121#issuecomment-1494391817 I am thinking if we can mask or remove the ORC file sizes related qtest items. In this pr, It is so wasting time to fix qtests about orc file size when upgrading orc version. Also, other OSS eg. Spark seems not treat orc file size as a test item. Issue Time Tracking --- Worklog Id: (was: 854558) Time Spent: 11h (was: 10h 50m) > Upgrade ORC to 1.8.3 > > > Key: HIVE-26809 > URL: https://issues.apache.org/jira/browse/HIVE-26809 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Dmitriy Fingerman >Assignee: Zoltán Rátkai >Priority: Major > Labels: pull-request-available > Time Spent: 11h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26997) Iceberg: Vectorization gets disabled at runtime in merge-into statements
[
https://issues.apache.org/jira/browse/HIVE-26997?focusedWorklogId=854556&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854556
]
ASF GitHub Bot logged work on HIVE-26997:
-
Author: ASF GitHub Bot
Created on: 03/Apr/23 13:54
Start Date: 03/Apr/23 13:54
Worklog Time Spent: 10m
Work Description: kasakrisz commented on code in PR #4162:
URL: https://github.com/apache/hive/pull/4162#discussion_r1155899237
##
ql/src/java/org/apache/hadoop/hive/ql/parse/RewriteSemanticAnalyzer.java:
##
@@ -670,19 +670,25 @@ public List getSortKeys() {
}
protected static class NonNativeAcidColumnAppender extends ColumnAppender {
+private String deletePrefix;
Review Comment:
nit.: can this be final?
Issue Time Tracking
---
Worklog Id: (was: 854556)
Time Spent: 2.5h (was: 2h 20m)
> Iceberg: Vectorization gets disabled at runtime in merge-into statements
>
>
> Key: HIVE-26997
> URL: https://issues.apache.org/jira/browse/HIVE-26997
> Project: Hive
> Issue Type: Improvement
> Components: Iceberg integration
>Reporter: Rajesh Balamohan
>Assignee: Zsolt Miskolczi
>Priority: Major
> Labels: pull-request-available
> Attachments: explain_merge_into.txt
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> *Query:*
> Think of "ssv" table as a table containing trickle feed data in the following
> query. "store_sales_delete_1" is the destination table.
>
> {noformat}
> MERGE INTO tpcds_1000_iceberg_mor_v4.store_sales_delete_1 t USING
> tpcds_1000_update.ssv s ON (t.ss_item_sk = s.ss_item_sk
>
> AND t.ss_customer_sk=s.ss_customer_sk
>
> AND t.ss_sold_date_sk = "2451181"
>
> AND ((Floor((s.ss_item_sk) / 1000) * 1000) BETWEEN 1000 AND
> 2000)
>
> AND s.ss_ext_discount_amt < 0.0) WHEN matched
> AND t.ss_ext_discount_amt IS NULL THEN
> UPDATE
> SET ss_ext_discount_amt = 0.0 WHEN NOT matched THEN
> INSERT (ss_sold_time_sk,
> ss_item_sk,
> ss_customer_sk,
> ss_cdemo_sk,
> ss_hdemo_sk,
> ss_addr_sk,
> ss_store_sk,
> ss_promo_sk,
> ss_ticket_number,
> ss_quantity,
> ss_wholesale_cost,
> ss_list_price,
> ss_sales_price,
> ss_ext_discount_amt,
> ss_ext_sales_price,
> ss_ext_wholesale_cost,
> ss_ext_list_price,
> ss_ext_tax,
> ss_coupon_amt,
> ss_net_paid,
> ss_net_paid_inc_tax,
> ss_net_profit,
> ss_sold_date_sk)
> VALUES (s.ss_sold_time_sk,
> s.ss_item_sk,
> s.ss_customer_sk,
> s.ss_cdemo_sk,
> s.ss_hdemo_sk,
> s.ss_addr_sk,
> s.ss_store_sk,
> s.ss_promo_sk,
> s.ss_ticket_number,
> s.ss_quantity,
> s.ss_wholesale_cost,
> s.ss_list_price,
> s.ss_sales_price,
> s.ss_ext_discount_amt,
> s.ss_ext_sales_price,
> s.ss_ext_wholesale_cost,
> s.ss_ext_list_price,
> s.ss_ext_tax,
> s.ss_coupon_amt,
> s.ss_net_paid,
> s.ss_net_paid_inc_tax,
> s.ss_net_profit,
> "2451181")
> {noformat}
>
>
> *Issue:*
> # Map phase is not getting vectorized due to "PARTITION_{_}SPEC{_}_ID" column
> {noformat}
> Map notVectorizedReason: Select expression for SELECT operator: Virtual
> column PARTITION__SPEC__ID is not supported {noformat}
>
> 2. "Reducer 2" stage isn't vectorized.
> {noformat}
> Reduce notVectorizedReason: exception: java.lang.RuntimeException: Full Outer
> Small Table Key Mapping duplicate column 0 in ordered column map {0=(value
> column: 30, type info: int), 1=(value column: 31, type info: int)} when
> adding value column 53, type into int stack trace:
> org.apache.hadoop.hive.ql.exec.vector.VectorColumnOrderedMap.add(VectorColumnOrderedMap.java:102),
>
> org.apache.hadoop.hive.ql.exec.vector.VectorColumnSourceMapping.add(VectorColumnSourceMapping.java:41),
>
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.canSpecializeMapJoin(Vectorizer.java:3865),
>
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.validateAndVectorizeOperator(Vectorizer.java:5246),
>
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.doProcessChild(Vectorizer.java:988),
>
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer
[jira] [Work logged] (HIVE-27209) Backport HIVE-24569: LLAP daemon leaks file descriptors/log4j appenders
[ https://issues.apache.org/jira/browse/HIVE-27209?focusedWorklogId=854554&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854554 ] ASF GitHub Bot logged work on HIVE-27209: - Author: ASF GitHub Bot Created on: 03/Apr/23 13:51 Start Date: 03/Apr/23 13:51 Worklog Time Spent: 10m Work Description: TuroczyX commented on PR #4193: URL: https://github.com/apache/hive/pull/4193#issuecomment-1494360123 +1 Issue Time Tracking --- Worklog Id: (was: 854554) Time Spent: 40m (was: 0.5h) > Backport HIVE-24569: LLAP daemon leaks file descriptors/log4j appenders > --- > > Key: HIVE-27209 > URL: https://issues.apache.org/jira/browse/HIVE-27209 > Project: Hive > Issue Type: Sub-task > Components: llap >Affects Versions: 2.2.0 >Reporter: Nikhil Gupta >Assignee: Nikhil Gupta >Priority: Major > Labels: pull-request-available > Fix For: 3.2.0 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26809) Upgrade ORC to 1.8.3
[ https://issues.apache.org/jira/browse/HIVE-26809?focusedWorklogId=854552&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854552 ] ASF GitHub Bot logged work on HIVE-26809: - Author: ASF GitHub Bot Created on: 03/Apr/23 13:48 Start Date: 03/Apr/23 13:48 Worklog Time Spent: 10m Work Description: TuroczyX commented on PR #4121: URL: https://github.com/apache/hive/pull/4121#issuecomment-1494356696 As I see this change mainly contains the new ORC file sizes changes in the tests, and minor code modifications on the hive itself. I do not see to many critical changes here. Issue Time Tracking --- Worklog Id: (was: 854552) Time Spent: 10h 50m (was: 10h 40m) > Upgrade ORC to 1.8.3 > > > Key: HIVE-26809 > URL: https://issues.apache.org/jira/browse/HIVE-26809 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Dmitriy Fingerman >Assignee: Zoltán Rátkai >Priority: Major > Labels: pull-request-available > Time Spent: 10h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26809) Upgrade ORC to 1.8.3
[ https://issues.apache.org/jira/browse/HIVE-26809?focusedWorklogId=854551&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854551 ] ASF GitHub Bot logged work on HIVE-26809: - Author: ASF GitHub Bot Created on: 03/Apr/23 13:44 Start Date: 03/Apr/23 13:44 Worklog Time Spent: 10m Work Description: TuroczyX commented on PR #4121: URL: https://github.com/apache/hive/pull/4121#issuecomment-1494350810 @difin @akshat0395 pls also check this pr. This change is one of the most interesting change since the Spotify is invented! cc: @nrg4878 Issue Time Tracking --- Worklog Id: (was: 854551) Time Spent: 10h 40m (was: 10.5h) > Upgrade ORC to 1.8.3 > > > Key: HIVE-26809 > URL: https://issues.apache.org/jira/browse/HIVE-26809 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Dmitriy Fingerman >Assignee: Zoltán Rátkai >Priority: Major > Labels: pull-request-available > Time Spent: 10h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26809) Upgrade ORC to 1.8.3
[ https://issues.apache.org/jira/browse/HIVE-26809?focusedWorklogId=854547&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854547 ] ASF GitHub Bot logged work on HIVE-26809: - Author: ASF GitHub Bot Created on: 03/Apr/23 13:07 Start Date: 03/Apr/23 13:07 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4121: URL: https://github.com/apache/hive/pull/4121#issuecomment-1494292924 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4121) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4121&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4121&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4121&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4121&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4121&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4121&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4121&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4121&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4121&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4121&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4121&resolved=false&types=CODE_SMELL) [13 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4121&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4121&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4121&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 854547) Time Spent: 10.5h (was: 10h 20m) > Upgrade ORC to 1.8.3 > > > Key: HIVE-26809 > URL: https://issues.apache.org/jira/browse/HIVE-26809 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Dmitriy Fingerman >Assignee: Zoltán Rátkai >Priority: Major > Labels: pull-request-available > Time Spent: 10.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26997) Iceberg: Vectorization gets disabled at runtime in merge-into statements
[
https://issues.apache.org/jira/browse/HIVE-26997?focusedWorklogId=854546&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854546
]
ASF GitHub Bot logged work on HIVE-26997:
-
Author: ASF GitHub Bot
Created on: 03/Apr/23 13:03
Start Date: 03/Apr/23 13:03
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #4162:
URL: https://github.com/apache/hive/pull/4162#discussion_r1155935183
##
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/IcebergAcidUtil.java:
##
@@ -93,10 +95,16 @@ public static Schema
createFileReadSchemaWithVirtualColums(List
dataCols) {
-List cols =
Lists.newArrayListWithCapacity(dataCols.size() + SERDE_META_COLS.size());
+ public static Schema createSerdeSchemaForDelete(List
dataCols, boolean partitioned,
+ Properties serDeProperties) {
+boolean skipRowData =
Boolean.parseBoolean(serDeProperties.getProperty(WriterBuilder.ICEBERG_DELETE_SKIPROWDATA,
+WriterBuilder.ICEBERG_DELETE_SKIPROWDATA_DEFAULT));
+List cols = Lists.newArrayListWithCapacity(
+SERDE_META_COLS.size() + (skipRowData || partitioned ? 0 :
dataCols.size()));
Review Comment:
completely changed the approach, not valid anymore
Issue Time Tracking
---
Worklog Id: (was: 854546)
Time Spent: 2h 20m (was: 2h 10m)
> Iceberg: Vectorization gets disabled at runtime in merge-into statements
>
>
> Key: HIVE-26997
> URL: https://issues.apache.org/jira/browse/HIVE-26997
> Project: Hive
> Issue Type: Improvement
> Components: Iceberg integration
>Reporter: Rajesh Balamohan
>Assignee: Zsolt Miskolczi
>Priority: Major
> Labels: pull-request-available
> Attachments: explain_merge_into.txt
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> *Query:*
> Think of "ssv" table as a table containing trickle feed data in the following
> query. "store_sales_delete_1" is the destination table.
>
> {noformat}
> MERGE INTO tpcds_1000_iceberg_mor_v4.store_sales_delete_1 t USING
> tpcds_1000_update.ssv s ON (t.ss_item_sk = s.ss_item_sk
>
> AND t.ss_customer_sk=s.ss_customer_sk
>
> AND t.ss_sold_date_sk = "2451181"
>
> AND ((Floor((s.ss_item_sk) / 1000) * 1000) BETWEEN 1000 AND
> 2000)
>
> AND s.ss_ext_discount_amt < 0.0) WHEN matched
> AND t.ss_ext_discount_amt IS NULL THEN
> UPDATE
> SET ss_ext_discount_amt = 0.0 WHEN NOT matched THEN
> INSERT (ss_sold_time_sk,
> ss_item_sk,
> ss_customer_sk,
> ss_cdemo_sk,
> ss_hdemo_sk,
> ss_addr_sk,
> ss_store_sk,
> ss_promo_sk,
> ss_ticket_number,
> ss_quantity,
> ss_wholesale_cost,
> ss_list_price,
> ss_sales_price,
> ss_ext_discount_amt,
> ss_ext_sales_price,
> ss_ext_wholesale_cost,
> ss_ext_list_price,
> ss_ext_tax,
> ss_coupon_amt,
> ss_net_paid,
> ss_net_paid_inc_tax,
> ss_net_profit,
> ss_sold_date_sk)
> VALUES (s.ss_sold_time_sk,
> s.ss_item_sk,
> s.ss_customer_sk,
> s.ss_cdemo_sk,
> s.ss_hdemo_sk,
> s.ss_addr_sk,
> s.ss_store_sk,
> s.ss_promo_sk,
> s.ss_ticket_number,
> s.ss_quantity,
> s.ss_wholesale_cost,
> s.ss_list_price,
> s.ss_sales_price,
> s.ss_ext_discount_amt,
> s.ss_ext_sales_price,
> s.ss_ext_wholesale_cost,
> s.ss_ext_list_price,
> s.ss_ext_tax,
> s.ss_coupon_amt,
> s.ss_net_paid,
> s.ss_net_paid_inc_tax,
> s.ss_net_profit,
> "2451181")
> {noformat}
>
>
> *Issue:*
> # Map phase is not getting vectorized due to "PARTITION_{_}SPEC{_}_ID" column
> {noformat}
> Map notVectorizedReason: Select expression for SELECT operator: Virtual
> column PARTITION__SPEC__ID is not supported {noformat}
>
> 2. "Reducer 2" stage isn't vectorized.
> {noformat}
> Reduce notVectorizedReason: exception: java.lang.RuntimeException: Full Outer
> Small Table Key Mapping duplicate column 0 in ordered column map {0=(value
> column: 30, type info: int), 1=(value column: 31, type info: int)} when
> adding value column 53, type into int stack trace:
> org.apache.hadoop.hive.ql.exec.vector.VectorColumnOrderedMap.add(VectorColumnOrdered
[jira] [Work logged] (HIVE-26997) Iceberg: Vectorization gets disabled at runtime in merge-into statements
[
https://issues.apache.org/jira/browse/HIVE-26997?focusedWorklogId=854545&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854545
]
ASF GitHub Bot logged work on HIVE-26997:
-
Author: ASF GitHub Bot
Created on: 03/Apr/23 13:03
Start Date: 03/Apr/23 13:03
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #4162:
URL: https://github.com/apache/hive/pull/4162#discussion_r1155934442
##
iceberg/iceberg-handler/src/test/queries/positive/vectorized_iceberg_merge_mixed.q:
##
@@ -0,0 +1,197 @@
+
Issue Time Tracking
---
Worklog Id: (was: 854545)
Time Spent: 2h 10m (was: 2h)
> Iceberg: Vectorization gets disabled at runtime in merge-into statements
>
>
> Key: HIVE-26997
> URL: https://issues.apache.org/jira/browse/HIVE-26997
> Project: Hive
> Issue Type: Improvement
> Components: Iceberg integration
>Reporter: Rajesh Balamohan
>Assignee: Zsolt Miskolczi
>Priority: Major
> Labels: pull-request-available
> Attachments: explain_merge_into.txt
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> *Query:*
> Think of "ssv" table as a table containing trickle feed data in the following
> query. "store_sales_delete_1" is the destination table.
>
> {noformat}
> MERGE INTO tpcds_1000_iceberg_mor_v4.store_sales_delete_1 t USING
> tpcds_1000_update.ssv s ON (t.ss_item_sk = s.ss_item_sk
>
> AND t.ss_customer_sk=s.ss_customer_sk
>
> AND t.ss_sold_date_sk = "2451181"
>
> AND ((Floor((s.ss_item_sk) / 1000) * 1000) BETWEEN 1000 AND
> 2000)
>
> AND s.ss_ext_discount_amt < 0.0) WHEN matched
> AND t.ss_ext_discount_amt IS NULL THEN
> UPDATE
> SET ss_ext_discount_amt = 0.0 WHEN NOT matched THEN
> INSERT (ss_sold_time_sk,
> ss_item_sk,
> ss_customer_sk,
> ss_cdemo_sk,
> ss_hdemo_sk,
> ss_addr_sk,
> ss_store_sk,
> ss_promo_sk,
> ss_ticket_number,
> ss_quantity,
> ss_wholesale_cost,
> ss_list_price,
> ss_sales_price,
> ss_ext_discount_amt,
> ss_ext_sales_price,
> ss_ext_wholesale_cost,
> ss_ext_list_price,
> ss_ext_tax,
> ss_coupon_amt,
> ss_net_paid,
> ss_net_paid_inc_tax,
> ss_net_profit,
> ss_sold_date_sk)
> VALUES (s.ss_sold_time_sk,
> s.ss_item_sk,
> s.ss_customer_sk,
> s.ss_cdemo_sk,
> s.ss_hdemo_sk,
> s.ss_addr_sk,
> s.ss_store_sk,
> s.ss_promo_sk,
> s.ss_ticket_number,
> s.ss_quantity,
> s.ss_wholesale_cost,
> s.ss_list_price,
> s.ss_sales_price,
> s.ss_ext_discount_amt,
> s.ss_ext_sales_price,
> s.ss_ext_wholesale_cost,
> s.ss_ext_list_price,
> s.ss_ext_tax,
> s.ss_coupon_amt,
> s.ss_net_paid,
> s.ss_net_paid_inc_tax,
> s.ss_net_profit,
> "2451181")
> {noformat}
>
>
> *Issue:*
> # Map phase is not getting vectorized due to "PARTITION_{_}SPEC{_}_ID" column
> {noformat}
> Map notVectorizedReason: Select expression for SELECT operator: Virtual
> column PARTITION__SPEC__ID is not supported {noformat}
>
> 2. "Reducer 2" stage isn't vectorized.
> {noformat}
> Reduce notVectorizedReason: exception: java.lang.RuntimeException: Full Outer
> Small Table Key Mapping duplicate column 0 in ordered column map {0=(value
> column: 30, type info: int), 1=(value column: 31, type info: int)} when
> adding value column 53, type into int stack trace:
> org.apache.hadoop.hive.ql.exec.vector.VectorColumnOrderedMap.add(VectorColumnOrderedMap.java:102),
>
> org.apache.hadoop.hive.ql.exec.vector.VectorColumnSourceMapping.add(VectorColumnSourceMapping.java:41),
>
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.canSpecializeMapJoin(Vectorizer.java:3865),
>
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.validateAndVectorizeOperator(Vectorizer.java:5246),
>
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.doProcessChild(Vectorizer.java:988),
>
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.doProcessChildren(Vectorizer.java:874),
>
> org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.validateAndVectorizeOperatorTree(Vectorizer.java:841),
>
> org.apache.hadoop.hive.ql
[jira] [Work logged] (HIVE-27187) Incremental rebuild of materialized view having aggregate and stored by iceberg
[
https://issues.apache.org/jira/browse/HIVE-27187?focusedWorklogId=854530&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854530
]
ASF GitHub Bot logged work on HIVE-27187:
-
Author: ASF GitHub Bot
Created on: 03/Apr/23 11:52
Start Date: 03/Apr/23 11:52
Worklog Time Spent: 10m
Work Description: kasakrisz commented on code in PR #4166:
URL: https://github.com/apache/hive/pull/4166#discussion_r1155857426
##
ql/src/java/org/apache/hadoop/hive/ql/ddl/view/materialized/alter/rebuild/AlterMaterializedViewRebuildAnalyzer.java:
##
@@ -564,30 +582,38 @@ private void fixUpASTAggregateIncrementalRebuild(
HiveParser.TOK_INSERT, HiveParser.TOK_SELECT);
ASTNode selectExprNodeInputROJ = (ASTNode) ParseDriver.adaptor.create(
HiveParser.TOK_SELEXPR, "TOK_SELEXPR");
-ASTNode tableName = createRowIdNode(TableName.getDbTable(
+astBuilder.createAcidSortNodes(TableName.getDbTable(
materializationNode.getChild(0).getText(),
-materializationNode.getChild(1).getText()));
-ParseDriver.adaptor.addChild(selectExprNodeInputROJ, tableName);
-ParseDriver.adaptor.addChild(selectNodeInputROJ, selectExprNodeInputROJ);
+materializationNode.getChild(1).getText()))
+.forEach(astNode ->
ParseDriver.adaptor.addChild(selectNodeInputROJ, astNode));
// 4) Transform first INSERT branch into an UPDATE
-// 4.1) Adding ROW__ID field
-ASTNode selectNodeInUpdate = (ASTNode) updateNode.getChild(1);
-if (selectNodeInUpdate.getType() != HiveParser.TOK_SELECT) {
- throw new SemanticException("TOK_SELECT expected in incremental
rewriting");
-}
-ASTNode selectExprNodeInUpdate = (ASTNode)
ParseDriver.adaptor.dupNode(selectExprNodeInputROJ);
-ParseDriver.adaptor.addChild(selectExprNodeInUpdate,
createRowIdNode((ASTNode) subqueryNodeInputROJ.getChild(1)));
-selectNodeInUpdate.insertChild(0, selectExprNodeInUpdate);
-// 4.2) Modifying filter condition.
+// 4.1) Modifying filter condition.
ASTNode whereClauseInUpdate = findWhereClause(updateNode);
if (whereClauseInUpdate.getChild(0).getType() != HiveParser.KW_OR) {
throw new SemanticException("OR clause expected below TOK_WHERE in
incremental rewriting");
}
// We bypass the OR clause and select the first disjunct for the Update
branch
ParseDriver.adaptor.setChild(whereClauseInUpdate, 0,
disjuncts.get(Context.DestClausePrefix.UPDATE));
+ASTNode updateDeleteNode = (ASTNode)
ParseDriver.adaptor.dupTree(updateNode);
+// 4.2) Adding ROW__ID field
+ASTNode selectNodeInUpdateDelete = (ASTNode) updateDeleteNode.getChild(1);
+if (selectNodeInUpdateDelete.getType() != HiveParser.TOK_SELECT) {
+ throw new SemanticException("TOK_SELECT expected in incremental
rewriting");
+}
+// Remove children
+while (selectNodeInUpdateDelete.getChildCount() > 0) {
+ selectNodeInUpdateDelete.deleteChild(0);
Review Comment:
`org.antlr.runtime.tree.BaseTree` doesn't support removing all children in
one call.
Issue Time Tracking
---
Worklog Id: (was: 854530)
Time Spent: 2h 20m (was: 2h 10m)
> Incremental rebuild of materialized view having aggregate and stored by
> iceberg
> ---
>
> Key: HIVE-27187
> URL: https://issues.apache.org/jira/browse/HIVE-27187
> Project: Hive
> Issue Type: Improvement
> Components: Iceberg integration, Materialized views
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> Currently incremental rebuild of materialized view stored by iceberg which
> definition query contains aggregate operator is transformed to an insert
> overwrite statement which contains a union operator if the source tables
> contains insert operations only. One branch of the union scans the view the
> other produces the delta.
> This can be improved further: transform the statement to a multi insert
> statement representing a merge statement to insert new aggregations and
> update existing.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27187) Incremental rebuild of materialized view having aggregate and stored by iceberg
[
https://issues.apache.org/jira/browse/HIVE-27187?focusedWorklogId=854527&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854527
]
ASF GitHub Bot logged work on HIVE-27187:
-
Author: ASF GitHub Bot
Created on: 03/Apr/23 11:47
Start Date: 03/Apr/23 11:47
Worklog Time Spent: 10m
Work Description: kasakrisz commented on code in PR #4166:
URL: https://github.com/apache/hive/pull/4166#discussion_r1155853269
##
ql/src/java/org/apache/hadoop/hive/ql/ddl/view/materialized/alter/rebuild/AlterMaterializedViewRebuildAnalyzer.java:
##
@@ -564,30 +582,38 @@ private void fixUpASTAggregateIncrementalRebuild(
HiveParser.TOK_INSERT, HiveParser.TOK_SELECT);
ASTNode selectExprNodeInputROJ = (ASTNode) ParseDriver.adaptor.create(
HiveParser.TOK_SELEXPR, "TOK_SELEXPR");
-ASTNode tableName = createRowIdNode(TableName.getDbTable(
+astBuilder.createAcidSortNodes(TableName.getDbTable(
materializationNode.getChild(0).getText(),
-materializationNode.getChild(1).getText()));
-ParseDriver.adaptor.addChild(selectExprNodeInputROJ, tableName);
-ParseDriver.adaptor.addChild(selectNodeInputROJ, selectExprNodeInputROJ);
+materializationNode.getChild(1).getText()))
+.forEach(astNode ->
ParseDriver.adaptor.addChild(selectNodeInputROJ, astNode));
// 4) Transform first INSERT branch into an UPDATE
-// 4.1) Adding ROW__ID field
-ASTNode selectNodeInUpdate = (ASTNode) updateNode.getChild(1);
-if (selectNodeInUpdate.getType() != HiveParser.TOK_SELECT) {
- throw new SemanticException("TOK_SELECT expected in incremental
rewriting");
-}
-ASTNode selectExprNodeInUpdate = (ASTNode)
ParseDriver.adaptor.dupNode(selectExprNodeInputROJ);
-ParseDriver.adaptor.addChild(selectExprNodeInUpdate,
createRowIdNode((ASTNode) subqueryNodeInputROJ.getChild(1)));
-selectNodeInUpdate.insertChild(0, selectExprNodeInUpdate);
-// 4.2) Modifying filter condition.
+// 4.1) Modifying filter condition.
ASTNode whereClauseInUpdate = findWhereClause(updateNode);
if (whereClauseInUpdate.getChild(0).getType() != HiveParser.KW_OR) {
throw new SemanticException("OR clause expected below TOK_WHERE in
incremental rewriting");
}
// We bypass the OR clause and select the first disjunct for the Update
branch
ParseDriver.adaptor.setChild(whereClauseInUpdate, 0,
disjuncts.get(Context.DestClausePrefix.UPDATE));
+ASTNode updateDeleteNode = (ASTNode)
ParseDriver.adaptor.dupTree(updateNode);
+// 4.2) Adding ROW__ID field
+ASTNode selectNodeInUpdateDelete = (ASTNode) updateDeleteNode.getChild(1);
+if (selectNodeInUpdateDelete.getType() != HiveParser.TOK_SELECT) {
+ throw new SemanticException("TOK_SELECT expected in incremental
rewriting");
Review Comment:
I added printing `selectNodeInUpdateDelete.getType()` which is the int
representation if the token.
Issue Time Tracking
---
Worklog Id: (was: 854527)
Time Spent: 2h 10m (was: 2h)
> Incremental rebuild of materialized view having aggregate and stored by
> iceberg
> ---
>
> Key: HIVE-27187
> URL: https://issues.apache.org/jira/browse/HIVE-27187
> Project: Hive
> Issue Type: Improvement
> Components: Iceberg integration, Materialized views
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Currently incremental rebuild of materialized view stored by iceberg which
> definition query contains aggregate operator is transformed to an insert
> overwrite statement which contains a union operator if the source tables
> contains insert operations only. One branch of the union scans the view the
> other produces the delta.
> This can be improved further: transform the statement to a multi insert
> statement representing a merge statement to insert new aggregations and
> update existing.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27187) Incremental rebuild of materialized view having aggregate and stored by iceberg
[
https://issues.apache.org/jira/browse/HIVE-27187?focusedWorklogId=854523&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854523
]
ASF GitHub Bot logged work on HIVE-27187:
-
Author: ASF GitHub Bot
Created on: 03/Apr/23 11:46
Start Date: 03/Apr/23 11:46
Worklog Time Spent: 10m
Work Description: kasakrisz commented on code in PR #4166:
URL: https://github.com/apache/hive/pull/4166#discussion_r1155851502
##
ql/src/java/org/apache/hadoop/hive/ql/ddl/view/materialized/alter/rebuild/AlterMaterializedViewRebuildAnalyzer.java:
##
@@ -465,21 +467,37 @@ private RelNode applyIncrementalRebuild(RelNode basePlan,
RelMetadataProvider md
protected ASTNode fixUpAfterCbo(ASTNode originalAst, ASTNode newAst,
CalcitePlanner.PreCboCtx cboCtx)
throws SemanticException {
ASTNode fixedAST = super.fixUpAfterCbo(originalAst, newAst, cboCtx);
+if (mvRebuildMode == MaterializationRebuildMode.INSERT_OVERWRITE_REBUILD) {
+ return fixedAST;
+} else if (mvRebuildMode ==
MaterializationRebuildMode.JOIN_INSERT_REBUILD) {
+ fixUpASTJoinInsertIncrementalRebuild(fixedAST);
+ return fixedAST;
+}
+
+MaterializedViewASTBuilder astBuilder;
+if (AcidUtils.isFullAcidTable(mvTable.getTTable())) {
Review Comment:
done
Issue Time Tracking
---
Worklog Id: (was: 854523)
Time Spent: 2h (was: 1h 50m)
> Incremental rebuild of materialized view having aggregate and stored by
> iceberg
> ---
>
> Key: HIVE-27187
> URL: https://issues.apache.org/jira/browse/HIVE-27187
> Project: Hive
> Issue Type: Improvement
> Components: Iceberg integration, Materialized views
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h
> Remaining Estimate: 0h
>
> Currently incremental rebuild of materialized view stored by iceberg which
> definition query contains aggregate operator is transformed to an insert
> overwrite statement which contains a union operator if the source tables
> contains insert operations only. One branch of the union scans the view the
> other produces the delta.
> This can be improved further: transform the statement to a multi insert
> statement representing a merge statement to insert new aggregations and
> update existing.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27164) Create Temp Txn Table As Select is failing at tablePath validation
[
https://issues.apache.org/jira/browse/HIVE-27164?focusedWorklogId=854524&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854524
]
ASF GitHub Bot logged work on HIVE-27164:
-
Author: ASF GitHub Bot
Created on: 03/Apr/23 11:46
Start Date: 03/Apr/23 11:46
Worklog Time Spent: 10m
Work Description: dengzhhu653 commented on PR #4176:
URL: https://github.com/apache/hive/pull/4176#issuecomment-1494172948
The test failure seems related, can you look into that?
Issue Time Tracking
---
Worklog Id: (was: 854524)
Time Spent: 2h 10m (was: 2h)
> Create Temp Txn Table As Select is failing at tablePath validation
> --
>
> Key: HIVE-27164
> URL: https://issues.apache.org/jira/browse/HIVE-27164
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, Metastore
>Reporter: Naresh P R
>Assignee: Venugopal Reddy K
>Priority: Major
> Labels: pull-request-available
> Attachments: mm_cttas.q
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> After HIVE-25303, every CTAS goes for
> HiveMetaStore$HMSHandler#translate_table_dryrun() call to fetch table
> location for CTAS queries which fails with following exception for temp
> tables if MetastoreDefaultTransformer is set.
> {code:java}
> 2023-03-17 16:41:23,390 INFO
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer:
> [pool-6-thread-196]: Starting translation for CreateTable for processor
> HMSClient-@localhost with [EXTWRITE, EXTREAD, HIVEBUCKET2, HIVEFULLACIDREAD,
> HIVEFULLACIDWRITE, HIVECACHEINVALIDATE, HIVEMANAGESTATS,
> HIVEMANAGEDINSERTWRITE, HIVEMANAGEDINSERTREAD, HIVESQL, HIVEMQT,
> HIVEONLYMQTWRITE] on table test_temp
> 2023-03-17 16:41:23,392 ERROR
> org.apache.hadoop.hive.metastore.RetryingHMSHandler: [pool-6-thread-196]:
> MetaException(message:Illegal location for managed table, it has to be within
> database's managed location)
> at
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer.validateTablePaths(MetastoreDefaultTransformer.java:886)
> at
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer.transformCreateTable(MetastoreDefaultTransformer.java:666)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.translate_table_dryrun(HiveMetaStore.java:2164)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) {code}
> I am able to repro this issue using attached testcase. [^mm_cttas.q]
>
> There are multiple ways to fix this issue
> * Have temp txn table path under db's managed location path. This will help
> with encryption zone paths as well.
> * Skip location check for temp tables at
> MetastoreDefaultTransformer#validateTablePaths()
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27187) Incremental rebuild of materialized view having aggregate and stored by iceberg
[
https://issues.apache.org/jira/browse/HIVE-27187?focusedWorklogId=854521&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854521
]
ASF GitHub Bot logged work on HIVE-27187:
-
Author: ASF GitHub Bot
Created on: 03/Apr/23 11:45
Start Date: 03/Apr/23 11:45
Worklog Time Spent: 10m
Work Description: kasakrisz commented on code in PR #4166:
URL: https://github.com/apache/hive/pull/4166#discussion_r1155850815
##
ql/src/java/org/apache/hadoop/hive/ql/ddl/view/materialized/alter/rebuild/NonNativeMaterializedViewASTBuilder.java:
##
@@ -0,0 +1,49 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.ddl.view.materialized.alter.rebuild;
+
+import org.apache.hadoop.hive.ql.Context;
+import org.apache.hadoop.hive.ql.metadata.Table;
+import org.apache.hadoop.hive.ql.metadata.VirtualColumn;
+import org.apache.hadoop.hive.ql.parse.ASTNode;
+
+import java.util.List;
+import java.util.stream.Collectors;
+
+import static java.util.Collections.singletonList;
+
+public class NonNativeMaterializedViewASTBuilder extends
MaterializedViewASTBuilder {
Review Comment:
Renamed to `NonNativeAcidMaterializedViewASTBuilder`
Issue Time Tracking
---
Worklog Id: (was: 854521)
Time Spent: 1h 40m (was: 1.5h)
> Incremental rebuild of materialized view having aggregate and stored by
> iceberg
> ---
>
> Key: HIVE-27187
> URL: https://issues.apache.org/jira/browse/HIVE-27187
> Project: Hive
> Issue Type: Improvement
> Components: Iceberg integration, Materialized views
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Currently incremental rebuild of materialized view stored by iceberg which
> definition query contains aggregate operator is transformed to an insert
> overwrite statement which contains a union operator if the source tables
> contains insert operations only. One branch of the union scans the view the
> other produces the delta.
> This can be improved further: transform the statement to a multi insert
> statement representing a merge statement to insert new aggregations and
> update existing.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27187) Incremental rebuild of materialized view having aggregate and stored by iceberg
[
https://issues.apache.org/jira/browse/HIVE-27187?focusedWorklogId=854522&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854522
]
ASF GitHub Bot logged work on HIVE-27187:
-
Author: ASF GitHub Bot
Created on: 03/Apr/23 11:45
Start Date: 03/Apr/23 11:45
Worklog Time Spent: 10m
Work Description: kasakrisz commented on code in PR #4166:
URL: https://github.com/apache/hive/pull/4166#discussion_r1155851179
##
ql/src/java/org/apache/hadoop/hive/ql/ddl/view/materialized/alter/rebuild/AlterMaterializedViewRebuildAnalyzer.java:
##
@@ -299,7 +300,8 @@ private RelNode applyRecordIncrementalRebuildPlan(
// First we need to check if it is valid to convert to MERGE/INSERT INTO.
// If we succeed, we modify the plan and afterwards the AST.
// MV should be an acid table.
- boolean fullAcidView = AcidUtils.isFullAcidTable(mvTable.getTTable());
+ boolean fullAcidView = AcidUtils.isFullAcidTable(mvTable.getTTable())
Review Comment:
renamed to `acidView`
##
ql/src/java/org/apache/hadoop/hive/ql/ddl/view/materialized/alter/rebuild/AlterMaterializedViewRebuildAnalyzer.java:
##
@@ -465,21 +467,37 @@ private RelNode applyIncrementalRebuild(RelNode basePlan,
RelMetadataProvider md
protected ASTNode fixUpAfterCbo(ASTNode originalAst, ASTNode newAst,
CalcitePlanner.PreCboCtx cboCtx)
throws SemanticException {
ASTNode fixedAST = super.fixUpAfterCbo(originalAst, newAst, cboCtx);
+if (mvRebuildMode == MaterializationRebuildMode.INSERT_OVERWRITE_REBUILD) {
Review Comment:
refactored
Issue Time Tracking
---
Worklog Id: (was: 854522)
Time Spent: 1h 50m (was: 1h 40m)
> Incremental rebuild of materialized view having aggregate and stored by
> iceberg
> ---
>
> Key: HIVE-27187
> URL: https://issues.apache.org/jira/browse/HIVE-27187
> Project: Hive
> Issue Type: Improvement
> Components: Iceberg integration, Materialized views
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Currently incremental rebuild of materialized view stored by iceberg which
> definition query contains aggregate operator is transformed to an insert
> overwrite statement which contains a union operator if the source tables
> contains insert operations only. One branch of the union scans the view the
> other produces the delta.
> This can be improved further: transform the statement to a multi insert
> statement representing a merge statement to insert new aggregations and
> update existing.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27164) Create Temp Txn Table As Select is failing at tablePath validation
[
https://issues.apache.org/jira/browse/HIVE-27164?focusedWorklogId=854506&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854506
]
ASF GitHub Bot logged work on HIVE-27164:
-
Author: ASF GitHub Bot
Created on: 03/Apr/23 11:34
Start Date: 03/Apr/23 11:34
Worklog Time Spent: 10m
Work Description: sonarcloud[bot] commented on PR #4176:
URL: https://github.com/apache/hive/pull/4176#issuecomment-1494158065
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4176)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4176&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4176&resolved=false&types=BUG)
[0
Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4176&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4176&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4176&resolved=false&types=VULNERABILITY)
[0
Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4176&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4176&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4176&resolved=false&types=SECURITY_HOTSPOT)
[0 Security
Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4176&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4176&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4176&resolved=false&types=CODE_SMELL)
[4 Code
Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4176&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4176&metric=coverage&view=list)
No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4176&metric=duplicated_lines_density&view=list)
No Duplication information
Issue Time Tracking
---
Worklog Id: (was: 854506)
Time Spent: 2h (was: 1h 50m)
> Create Temp Txn Table As Select is failing at tablePath validation
> --
>
> Key: HIVE-27164
> URL: https://issues.apache.org/jira/browse/HIVE-27164
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, Metastore
>Reporter: Naresh P R
>Assignee: Venugopal Reddy K
>Priority: Major
> Labels: pull-request-available
> Attachments: mm_cttas.q
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> After HIVE-25303, every CTAS goes for
> HiveMetaStore$HMSHandler#translate_table_dryrun() call to fetch table
> location for CTAS queries which fails with following exception for temp
> tables if MetastoreDefaultTransformer is set.
> {code:java}
> 2023-03-17 16:41:23,390 INFO
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer:
> [pool-6-thread-196]: Starting translation for CreateTable for processor
> HMSClient-@localhost with [EXTWRITE, EXTREAD, HIVEBUCKET2, HIVEFULLACIDREAD,
> HIVEF
[jira] [Work logged] (HIVE-27135) AcidUtils#getHdfsDirSnapshots() throws FNFE when a directory is removed in HDFS
[
https://issues.apache.org/jira/browse/HIVE-27135?focusedWorklogId=854505&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854505
]
ASF GitHub Bot logged work on HIVE-27135:
-
Author: ASF GitHub Bot
Created on: 03/Apr/23 11:30
Start Date: 03/Apr/23 11:30
Worklog Time Spent: 10m
Work Description: sonarcloud[bot] commented on PR #4114:
URL: https://github.com/apache/hive/pull/4114#issuecomment-1494153045
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4114)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4114&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4114&resolved=false&types=BUG)
[0
Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4114&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4114&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4114&resolved=false&types=VULNERABILITY)
[0
Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4114&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4114&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4114&resolved=false&types=SECURITY_HOTSPOT)
[0 Security
Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4114&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4114&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4114&resolved=false&types=CODE_SMELL)
[0 Code
Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4114&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4114&metric=coverage&view=list)
No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4114&metric=duplicated_lines_density&view=list)
No Duplication information
Issue Time Tracking
---
Worklog Id: (was: 854505)
Time Spent: 7h 10m (was: 7h)
> AcidUtils#getHdfsDirSnapshots() throws FNFE when a directory is removed in
> HDFS
> ---
>
> Key: HIVE-27135
> URL: https://issues.apache.org/jira/browse/HIVE-27135
> Project: Hive
> Issue Type: Bug
>Reporter: Dayakar M
>Assignee: Dayakar M
>Priority: Major
> Labels: pull-request-available
> Time Spent: 7h 10m
> Remaining Estimate: 0h
>
> AcidUtils#getHdfsDirSnapshots() throws FileNotFoundException when a directory
> is removed in HDFS while fetching HDFS Snapshots.
> Below testcode can be used to reproduce this issue.
> {code:java}
> @Test
> public void
> testShouldNotThrowFNFEWhenHiveStagingDirectoryIsRemovedWhileFetchingHDFSSnapshots()
> throws Exception {
> MockFileSystem fs = new MockFileSystem(new HiveConf(),
> new MockFile("mock:/tbl/part1/.hive-staging_dir/-ext-10002", 500, new
> byte[0]),
> new MockFile("mock:/tbl/part2/.hive-staging_dir", 500, new byte[0]),
>
[jira] [Work logged] (HIVE-27208) Iceberg: Add support for rename table
[ https://issues.apache.org/jira/browse/HIVE-27208?focusedWorklogId=854504&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854504 ] ASF GitHub Bot logged work on HIVE-27208: - Author: ASF GitHub Bot Created on: 03/Apr/23 11:29 Start Date: 03/Apr/23 11:29 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4185: URL: https://github.com/apache/hive/pull/4185#issuecomment-1494151691 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4185) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4185&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4185&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4185&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4185&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4185&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4185&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4185&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4185&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4185&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4185&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4185&resolved=false&types=CODE_SMELL) [4 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4185&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4185&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4185&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 854504) Time Spent: 1h (was: 50m) > Iceberg: Add support for rename table > - > > Key: HIVE-27208 > URL: https://issues.apache.org/jira/browse/HIVE-27208 > Project: Hive > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > Add support for renaming iceberg tables. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27177) Add alter table...Convert to Iceberg command
[
https://issues.apache.org/jira/browse/HIVE-27177?focusedWorklogId=854488&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854488
]
ASF GitHub Bot logged work on HIVE-27177:
-
Author: ASF GitHub Bot
Created on: 03/Apr/23 09:57
Start Date: 03/Apr/23 09:57
Worklog Time Spent: 10m
Work Description: zhangbutao commented on code in PR #4155:
URL: https://github.com/apache/hive/pull/4155#discussion_r1155739197
##
ql/src/java/org/apache/hadoop/hive/ql/ddl/table/convert/AlterTableConvertAnalyzer.java:
##
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.ddl.table.convert;
+
+import org.apache.hadoop.hive.common.TableName;
+import org.apache.hadoop.hive.ql.QueryState;
+import org.apache.hadoop.hive.ql.ddl.DDLSemanticAnalyzerFactory.DDLType;
+import org.apache.hadoop.hive.ql.ddl.DDLWork;
+import org.apache.hadoop.hive.ql.ddl.table.AbstractAlterTableAnalyzer;
+import org.apache.hadoop.hive.ql.ddl.table.AlterTableType;
+import org.apache.hadoop.hive.ql.exec.TaskFactory;
+import org.apache.hadoop.hive.ql.hooks.ReadEntity;
+import org.apache.hadoop.hive.ql.metadata.Table;
+import org.apache.hadoop.hive.ql.parse.ASTNode;
+import org.apache.hadoop.hive.ql.parse.AlterTableConvertSpec;
+import org.apache.hadoop.hive.ql.parse.HiveParser;
+import org.apache.hadoop.hive.ql.parse.SemanticException;
+
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * Analyzer for ALTER TABLE ... CONVERT commands.
+ */
+@DDLType(types = HiveParser.TOK_ALTERTABLE_CONVERT)
+public class AlterTableConvertAnalyzer extends AbstractAlterTableAnalyzer {
+
+ public AlterTableConvertAnalyzer(QueryState queryState) throws
SemanticException {
+super(queryState);
+ }
+
+ @Override
+ protected void analyzeCommand(TableName tableName, Map
partitionSpec, ASTNode command)
+ throws SemanticException {
+Table table = getTable(tableName);
+// the first child must be the execute operation type
Review Comment:
nit: This line should be: `the first child must be the addprops type`
Issue Time Tracking
---
Worklog Id: (was: 854488)
Time Spent: 1h 10m (was: 1h)
> Add alter table...Convert to Iceberg command
>
>
> Key: HIVE-27177
> URL: https://issues.apache.org/jira/browse/HIVE-27177
> Project: Hive
> Issue Type: Improvement
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Add an alter table convert to Iceberg [TBLPROPERTIES('','')] to
> convert exiting external tables to iceberg tables
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27203) Add compaction pending Qtest for Insert-only, Partitioned, Clustered ACID, and combination Tables
[
https://issues.apache.org/jira/browse/HIVE-27203?focusedWorklogId=854477&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854477
]
ASF GitHub Bot logged work on HIVE-27203:
-
Author: ASF GitHub Bot
Created on: 03/Apr/23 09:43
Start Date: 03/Apr/23 09:43
Worklog Time Spent: 10m
Work Description: SourabhBadhya commented on code in PR #4181:
URL: https://github.com/apache/hive/pull/4181#discussion_r1155722596
##
ql/src/test/queries/clientpositive/compaction_query_based_insert_only_clustered.q:
##
@@ -0,0 +1,33 @@
+--! qt:replace:/createTime:(\d+)/#Masked#/
+--! qt:replace:/location:(\S+)/#Masked#/
+--! qt:replace:/lastAccessTime:(\d+)/#Masked#/
+--! qt:replace:/ownerType:(\S*)/#Masked#/
+--! qt:replace:/owner:(\S*)/#Masked#/
+--! qt:replace:/skewedColValueLocationMaps:(\S*)/#Masked#/
+--! qt:replace:/transient_lastDdlTime=(\d+)/#Masked#/
+--! qt:replace:/totalSize=(\d+)/#Masked#/
+--! qt:replace:/rawDataSize=(\d+)/#Masked#/
+--! qt:replace:/writeId:(\d+)/#Masked#/
+--! qt:replace:/bucketing_version=(\d+)/#Masked#/
+--! qt:replace:/id:(\d+)/#Masked#/
+
+drop table orc_bucketed;
+
+create table orc_bucketed (a int, b string) clustered by (a) into 3 buckets
stored as orc TBLPROPERTIES('transactional'='true',
'transactional_properties'='insert_only');
+
+insert into orc_bucketed values('1', 'text1');
+insert into orc_bucketed values('2', 'text2');
+insert into orc_bucketed values('3', 'text3');
+insert into orc_bucketed values('4', 'text4');
+insert into orc_bucketed values('5', 'text5');
+insert into orc_bucketed values('6', 'text6');
+insert into orc_bucketed values('7', 'text7');
+insert into orc_bucketed values('8', 'text8');
+insert into orc_bucketed values('9', 'text9');
+insert into orc_bucketed values('10', 'text10');
+
+describe extended orc_bucketed;
+alter table orc_bucketed compact 'MAJOR' and wait;
+analyze table orc_bucketed compute statistics;
+
+describe extended orc_bucketed;
Review Comment:
nit: Add a new line at the end of file.
##
ql/src/test/queries/clientpositive/compaction_query_based_insert_only_clustered.q:
##
@@ -0,0 +1,33 @@
+--! qt:replace:/createTime:(\d+)/#Masked#/
+--! qt:replace:/location:(\S+)/#Masked#/
+--! qt:replace:/lastAccessTime:(\d+)/#Masked#/
+--! qt:replace:/ownerType:(\S*)/#Masked#/
+--! qt:replace:/owner:(\S*)/#Masked#/
+--! qt:replace:/skewedColValueLocationMaps:(\S*)/#Masked#/
+--! qt:replace:/transient_lastDdlTime=(\d+)/#Masked#/
+--! qt:replace:/totalSize=(\d+)/#Masked#/
+--! qt:replace:/rawDataSize=(\d+)/#Masked#/
+--! qt:replace:/writeId:(\d+)/#Masked#/
+--! qt:replace:/bucketing_version=(\d+)/#Masked#/
+--! qt:replace:/id:(\d+)/#Masked#/
+
+drop table orc_bucketed;
Review Comment:
`drop table if exists`?
##
ql/src/test/queries/clientpositive/compaction_query_based_insert_only_clustered.q:
##
@@ -0,0 +1,33 @@
+--! qt:replace:/createTime:(\d+)/#Masked#/
+--! qt:replace:/location:(\S+)/#Masked#/
+--! qt:replace:/lastAccessTime:(\d+)/#Masked#/
+--! qt:replace:/ownerType:(\S*)/#Masked#/
+--! qt:replace:/owner:(\S*)/#Masked#/
+--! qt:replace:/skewedColValueLocationMaps:(\S*)/#Masked#/
+--! qt:replace:/transient_lastDdlTime=(\d+)/#Masked#/
+--! qt:replace:/totalSize=(\d+)/#Masked#/
+--! qt:replace:/rawDataSize=(\d+)/#Masked#/
+--! qt:replace:/writeId:(\d+)/#Masked#/
+--! qt:replace:/bucketing_version=(\d+)/#Masked#/
+--! qt:replace:/id:(\d+)/#Masked#/
+
+drop table orc_bucketed;
+
+create table orc_bucketed (a int, b string) clustered by (a) into 3 buckets
stored as orc TBLPROPERTIES('transactional'='true',
'transactional_properties'='insert_only');
+
+insert into orc_bucketed values('1', 'text1');
+insert into orc_bucketed values('2', 'text2');
+insert into orc_bucketed values('3', 'text3');
+insert into orc_bucketed values('4', 'text4');
+insert into orc_bucketed values('5', 'text5');
+insert into orc_bucketed values('6', 'text6');
+insert into orc_bucketed values('7', 'text7');
+insert into orc_bucketed values('8', 'text8');
+insert into orc_bucketed values('9', 'text9');
+insert into orc_bucketed values('10', 'text10');
+
+describe extended orc_bucketed;
+alter table orc_bucketed compact 'MAJOR' and wait;
+analyze table orc_bucketed compute statistics;
Review Comment:
Is `analyze table compute statistics` command required here?
Stats update usually happens within the compaction cycle so I think
re-evaluation of stats is extra effort.
The place where Stats update happens in compaction is here -
https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/StatsUpdater.java
Issue Time Tracking
---
Worklog Id: (was: 854477)
Time Spent: 50m (was: 40m)
> Add compaction pending Qtest for Insert-only, Partitioned, Clustered ACID,
> and combination Tables
> -
[jira] [Work logged] (HIVE-26809) Upgrade ORC to 1.8.3
[
https://issues.apache.org/jira/browse/HIVE-26809?focusedWorklogId=854475&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854475
]
ASF GitHub Bot logged work on HIVE-26809:
-
Author: ASF GitHub Bot
Created on: 03/Apr/23 09:41
Start Date: 03/Apr/23 09:41
Worklog Time Spent: 10m
Work Description: zratkai commented on code in PR #4121:
URL: https://github.com/apache/hive/pull/4121#discussion_r1155725610
##
ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java:
##
@@ -298,11 +300,11 @@ public void seek(PositionProvider index) throws
IOException {
}
}
-@Override
+@Override
public void nextVector(
-ColumnVector previousVector, boolean[] isNull, int batchSize) throws
IOException {
+ColumnVector previousVector, boolean[] isNull, int batchSize,
FilterContext filterContext, TypeReader.ReadPhase readPhase) throws IOException
{
Review Comment:
Reformatted to match the style.
##
ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java:
##
@@ -298,11 +300,11 @@ public void seek(PositionProvider index) throws
IOException {
}
}
-@Override
+@Override
Review Comment:
Reformatted to match the style.
Issue Time Tracking
---
Worklog Id: (was: 854475)
Time Spent: 10h 20m (was: 10h 10m)
> Upgrade ORC to 1.8.3
>
>
> Key: HIVE-26809
> URL: https://issues.apache.org/jira/browse/HIVE-26809
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Dmitriy Fingerman
>Assignee: Zoltán Rátkai
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10h 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26809) Upgrade ORC to 1.8.3
[ https://issues.apache.org/jira/browse/HIVE-26809?focusedWorklogId=854473&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854473 ] ASF GitHub Bot logged work on HIVE-26809: - Author: ASF GitHub Bot Created on: 03/Apr/23 09:36 Start Date: 03/Apr/23 09:36 Worklog Time Spent: 10m Work Description: zratkai commented on PR #4121: URL: https://github.com/apache/hive/pull/4121#issuecomment-1493997747 @abstractdog thanks for the review! The issue discussed in the other PR was an ORC related issue, so I fixed it in ORC: https://github.com/apache/orc/pull/1431 Issue Time Tracking --- Worklog Id: (was: 854473) Time Spent: 10h 10m (was: 10h) > Upgrade ORC to 1.8.3 > > > Key: HIVE-26809 > URL: https://issues.apache.org/jira/browse/HIVE-26809 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Dmitriy Fingerman >Assignee: Zoltán Rátkai >Priority: Major > Labels: pull-request-available > Time Spent: 10h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26809) Upgrade ORC to 1.8.3
[ https://issues.apache.org/jira/browse/HIVE-26809?focusedWorklogId=854470&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854470 ] ASF GitHub Bot logged work on HIVE-26809: - Author: ASF GitHub Bot Created on: 03/Apr/23 09:30 Start Date: 03/Apr/23 09:30 Worklog Time Spent: 10m Work Description: abstractdog commented on PR #4121: URL: https://github.com/apache/hive/pull/4121#issuecomment-1493988425 thanks for putting this together @zratkai! can you please elaborate on how you prevented the situation that we were discussing in another PR: https://github.com/apache/hive/pull/3833#discussion_r1068830673 Issue Time Tracking --- Worklog Id: (was: 854470) Time Spent: 10h (was: 9h 50m) > Upgrade ORC to 1.8.3 > > > Key: HIVE-26809 > URL: https://issues.apache.org/jira/browse/HIVE-26809 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Dmitriy Fingerman >Assignee: Zoltán Rátkai >Priority: Major > Labels: pull-request-available > Time Spent: 10h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26809) Upgrade ORC to 1.8.3
[
https://issues.apache.org/jira/browse/HIVE-26809?focusedWorklogId=854465&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854465
]
ASF GitHub Bot logged work on HIVE-26809:
-
Author: ASF GitHub Bot
Created on: 03/Apr/23 09:11
Start Date: 03/Apr/23 09:11
Worklog Time Spent: 10m
Work Description: abstractdog commented on code in PR #4121:
URL: https://github.com/apache/hive/pull/4121#discussion_r1155690043
##
ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java:
##
@@ -298,11 +300,11 @@ public void seek(PositionProvider index) throws
IOException {
}
}
-@Override
+@Override
public void nextVector(
-ColumnVector previousVector, boolean[] isNull, int batchSize) throws
IOException {
+ColumnVector previousVector, boolean[] isNull, int batchSize,
FilterContext filterContext, TypeReader.ReadPhase readPhase) throws IOException
{
Review Comment:
this line looks a bit long :) can you reformat according to
[this](https://github.com/apache/hive/blob/master/dev-support/eclipse-styles.xml)?
please check other occurrences of nextVector too
Issue Time Tracking
---
Worklog Id: (was: 854465)
Time Spent: 9h 50m (was: 9h 40m)
> Upgrade ORC to 1.8.3
>
>
> Key: HIVE-26809
> URL: https://issues.apache.org/jira/browse/HIVE-26809
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Dmitriy Fingerman
>Assignee: Zoltán Rátkai
>Priority: Major
> Labels: pull-request-available
> Time Spent: 9h 50m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26809) Upgrade ORC to 1.8.3
[
https://issues.apache.org/jira/browse/HIVE-26809?focusedWorklogId=854463&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854463
]
ASF GitHub Bot logged work on HIVE-26809:
-
Author: ASF GitHub Bot
Created on: 03/Apr/23 09:10
Start Date: 03/Apr/23 09:10
Worklog Time Spent: 10m
Work Description: abstractdog commented on code in PR #4121:
URL: https://github.com/apache/hive/pull/4121#discussion_r1155690828
##
ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java:
##
@@ -462,10 +464,9 @@ public void seek(PositionProvider index) throws
IOException {
}
@Override
-public void nextVector(
-ColumnVector previousVector, boolean[] isNull, int batchSize) throws
IOException {
+public void nextVector(ColumnVector previousVector, boolean[] isNull,
final int batchSize, FilterContext filterContext, ReadPhase readPhase) throws
IOException {
Review Comment:
long line
Issue Time Tracking
---
Worklog Id: (was: 854463)
Time Spent: 9h 40m (was: 9.5h)
> Upgrade ORC to 1.8.3
>
>
> Key: HIVE-26809
> URL: https://issues.apache.org/jira/browse/HIVE-26809
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Dmitriy Fingerman
>Assignee: Zoltán Rátkai
>Priority: Major
> Labels: pull-request-available
> Time Spent: 9h 40m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26809) Upgrade ORC to 1.8.3
[
https://issues.apache.org/jira/browse/HIVE-26809?focusedWorklogId=854462&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854462
]
ASF GitHub Bot logged work on HIVE-26809:
-
Author: ASF GitHub Bot
Created on: 03/Apr/23 09:10
Start Date: 03/Apr/23 09:10
Worklog Time Spent: 10m
Work Description: abstractdog commented on code in PR #4121:
URL: https://github.com/apache/hive/pull/4121#discussion_r1155690285
##
ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java:
##
@@ -298,11 +300,11 @@ public void seek(PositionProvider index) throws
IOException {
}
}
-@Override
+@Override
Review Comment:
please remove whitespace
Issue Time Tracking
---
Worklog Id: (was: 854462)
Time Spent: 9.5h (was: 9h 20m)
> Upgrade ORC to 1.8.3
>
>
> Key: HIVE-26809
> URL: https://issues.apache.org/jira/browse/HIVE-26809
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Dmitriy Fingerman
>Assignee: Zoltán Rátkai
>Priority: Major
> Labels: pull-request-available
> Time Spent: 9.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26809) Upgrade ORC to 1.8.3
[
https://issues.apache.org/jira/browse/HIVE-26809?focusedWorklogId=854461&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854461
]
ASF GitHub Bot logged work on HIVE-26809:
-
Author: ASF GitHub Bot
Created on: 03/Apr/23 09:09
Start Date: 03/Apr/23 09:09
Worklog Time Spent: 10m
Work Description: abstractdog commented on code in PR #4121:
URL: https://github.com/apache/hive/pull/4121#discussion_r1155690043
##
ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java:
##
@@ -298,11 +300,11 @@ public void seek(PositionProvider index) throws
IOException {
}
}
-@Override
+@Override
public void nextVector(
-ColumnVector previousVector, boolean[] isNull, int batchSize) throws
IOException {
+ColumnVector previousVector, boolean[] isNull, int batchSize,
FilterContext filterContext, TypeReader.ReadPhase readPhase) throws IOException
{
Review Comment:
this line looks a bit long :) can you reformat according to
[this](https://github.com/apache/hive/blob/master/dev-support/eclipse-styles.xml)?
Issue Time Tracking
---
Worklog Id: (was: 854461)
Time Spent: 9h 20m (was: 9h 10m)
> Upgrade ORC to 1.8.3
>
>
> Key: HIVE-26809
> URL: https://issues.apache.org/jira/browse/HIVE-26809
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Dmitriy Fingerman
>Assignee: Zoltán Rátkai
>Priority: Major
> Labels: pull-request-available
> Time Spent: 9h 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27203) Add compaction pending Qtest for Insert-only, Partitioned, Clustered ACID, and combination Tables
[ https://issues.apache.org/jira/browse/HIVE-27203?focusedWorklogId=854434&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854434 ] ASF GitHub Bot logged work on HIVE-27203: - Author: ASF GitHub Bot Created on: 03/Apr/23 07:57 Start Date: 03/Apr/23 07:57 Worklog Time Spent: 10m Work Description: akshat0395 commented on code in PR #4181: URL: https://github.com/apache/hive/pull/4181#discussion_r1155606740 ## ql/src/test/queries/clientpositive/compaction_query_based_insert_only_partitioned_clustered.q: ## Review Comment: Thanks for the comment @rkirtir, These test individually and the reason for having Major and minor in separate tests is to test these compaction in isolation for different scenarios. This patterns has been followed in other compaction related qtests as well. Here are some ref that follows the same pattern: 1. ql/src/test/queries/clientpositive/compaction_query_based.q 2. ql/src/test/queries/clientpositive/compaction_query_based_clustered.q 3. ql/src/test/queries/clientpositive/compaction_query_based_clustered_minor.q 4. ql/src/test/queries/clientpositive/compaction_query_based_insert_only.q 5. ql/src/test/queries/clientpositive/compaction_query_based_insert_only_minor.q 6. ql/src/test/queries/clientpositive/compaction_query_based_minor.q Issue Time Tracking --- Worklog Id: (was: 854434) Time Spent: 40m (was: 0.5h) > Add compaction pending Qtest for Insert-only, Partitioned, Clustered ACID, > and combination Tables > -- > > Key: HIVE-27203 > URL: https://issues.apache.org/jira/browse/HIVE-27203 > Project: Hive > Issue Type: Test >Reporter: Akshat Mathur >Assignee: Akshat Mathur >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > Improve Qtest Coverage for Compaction use cases for ACID Tables: > # Partitioned Tables( Major & Minor ) > # Insert-Only Clustered( Major & Minor ) > # Insert-Only Partitioned( Major & Minor ) > # Insert-Only Clustered and Partitioned( Major & Minor ) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27213) parquet logical decimal type to INT32 is not working while compute statastics
[
https://issues.apache.org/jira/browse/HIVE-27213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
KIRTI RUGE updated HIVE-27213:
--
Description:
test.parquet
Steps to reproduce:
dfs ${system:test.dfs.mkdir} hdfs:///tmp/dwxtest/ws_sold_date_sk=2451825;
dfs -copyFromLocal ../../data/files/dwxtest.parquet
hdfs:///tmp/dwxtest/ws_sold_date_sk=2451825;
dfs -ls hdfs:///tmp/dwxtest/ws_sold_date_sk=2451825/;
CREATE EXTERNAL TABLE `web_sales`(
`ws_sold_time_sk` int,
`ws_ship_date_sk` int,
`ws_item_sk` int,
`ws_bill_customer_sk` int,
`ws_bill_cdemo_sk` int,
`ws_bill_hdemo_sk` int,
`ws_bill_addr_sk` int,
`ws_ship_customer_sk` int,
`ws_ship_cdemo_sk` int,
`ws_ship_hdemo_sk` int,
`ws_ship_addr_sk` int,
`ws_web_page_sk` int,
`ws_web_site_sk` int,
`ws_ship_mode_sk` int,
`ws_warehouse_sk` int,
`ws_promo_sk` int,
`ws_order_number` bigint,
`ws_quantity` int,
`ws_wholesale_cost` decimal(7,2),
`ws_list_price` decimal(7,2),
`ws_sales_price` decimal(7,2),
`ws_ext_discount_amt` decimal(7,2),
`ws_ext_sales_price` decimal(7,2),
`ws_ext_wholesale_cost` decimal(7,2),
`ws_ext_list_price` decimal(7,2),
`ws_ext_tax` decimal(7,2),
`ws_coupon_amt` decimal(7,2),
`ws_ext_ship_cost` decimal(7,2),
`ws_net_paid` decimal(7,2),
`ws_net_paid_inc_tax` decimal(7,2),
`ws_net_paid_inc_ship` decimal(7,2),
`ws_net_paid_inc_ship_tax` decimal(7,2),
`ws_net_profit` decimal(7,2))
PARTITIONED BY (
`ws_sold_date_sk` int)
ROW FORMAT SERDE
'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe'
STORED AS PARQUET LOCATION 'hdfs:///tmp/dwxtest/';
MSCK REPAIR TABLE web_sales;
analyze table web_sales compute statistics for columns;
Error Stack:
analyze table web_sales compute statistics for columns;
], TaskAttempt 3 failed, info=[Error: Error while running task ( failure ) :
attempt_1678779198717__2_00_52_3:java.lang.RuntimeException:
java.lang.RuntimeException: java.io.IOException:
org.apache.parquet.io.ParquetDecodingException: Can not read value at 0 in
block -1 in file
s3a://xx/useDecimal=true,useDate=true,filterNull=false/web_sales/ws_sold_date_sk=2451825/part-00796-788bef86-2748-4e21-a464-b34c7e646c94-cfcafd2c-2abd-4067-8aea-f58cb1021b35.c000.snappy.parquet
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:351)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:280)
at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:84)
at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:70)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899)
at
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:70)
at
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:40)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at
org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:118)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.RuntimeException: java.io.IOException:
org.apache.parquet.io.ParquetDecodingException: Can not read value at 0 in
block -1 in file
s3a://xxx/useDecimal=true,useDate=true,filterNull=false/web_sales/ws_sold_date_sk=2451825/part-00796-788bef86-2748-4e21-a464-b34c7e646c94-cfcafd2c-2abd-4067-8aea-f58cb1021b35.c000.snappy.parquet
at
org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:206)
at
org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.(TezGroupedSplitsInputFormat.java:145)
at
org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat.getRecordReader(TezGroupedSplitsInputFormat.java:111)
at
org.apache.tez.mapreduce.lib.MRReaderMapred.setupOldRecordReader(MRReaderMapred.java:164)
at
org.apache.tez.mapreduce.lib.MRReaderMapred.setSplit(MRReaderMapred.java:83)
at
org.apache.tez.mapreduce.input.MRInput.initFromEventInternal(MRInput.java:704)
at org.apache.tez.mapreduce.input.MRInput.initFromEvent(MRInput.java:663)
at
org.apache.tez.mapreduce.input.MRInputLegacy.checkAndAwaitRecordReaderInitialization(MRInputLegacy.java:150)
at org.apache.tez.mapreduce.input.MRInputLegacy.init(MRInputLegacy.java:114)
at
org.
[jira] [Updated] (HIVE-27213) parquet logical decimal type to INT32 is not working while compute statastics
[
https://issues.apache.org/jira/browse/HIVE-27213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
KIRTI RUGE updated HIVE-27213:
--
Description: test.parquetSteps to reproduce:dfs ${system:test.dfs.mkdir}
hdfs:///tmp/dwxtest/ws_sold_date_sk=2451825; dfs -copyFromLocal
../../data/files/dwxtest.parquet hdfs:///tmp/dwxtest/ws_sold_date_sk=2451825;
dfs -ls hdfs:///tmp/dwxtest/ws_sold_date_sk=2451825/;CREATE EXTERNAL TABLE
`web_sales`( `ws_sold_time_sk` int, `ws_ship_date_sk` int, `ws_item_sk` int,
`ws_bill_customer_sk` int, `ws_bill_cdemo_sk` int, `ws_bill_hdemo_sk` int,
`ws_bill_addr_sk` int, `ws_ship_customer_sk` int, `ws_ship_cdemo_sk` int,
`ws_ship_hdemo_sk` int, `ws_ship_addr_sk` int, `ws_web_page_sk` int,
`ws_web_site_sk` int, `ws_ship_mode_sk` int, `ws_warehouse_sk` int,
`ws_promo_sk` int, `ws_order_number` bigint, `ws_quantity` int,
`ws_wholesale_cost` decimal(7,2), `ws_list_price` decimal(7,2),
`ws_sales_price` decimal(7,2), `ws_ext_discount_amt` decimal(7,2),
`ws_ext_sales_price` decimal(7,2), `ws_ext_wholesale_cost` decimal(7,2),
`ws_ext_list_price` decimal(7,2), `ws_ext_tax` decimal(7,2), `ws_coupon_amt`
decimal(7,2), `ws_ext_ship_cost` decimal(7,2), `ws_net_paid` decimal(7,2),
`ws_net_paid_inc_tax` decimal(7,2), `ws_net_paid_inc_ship` decimal(7,2),
`ws_net_paid_inc_ship_tax` decimal(7,2), `ws_net_profit` decimal(7,2))
PARTITIONED BY ( `ws_sold_date_sk` int) ROW FORMAT SERDE
'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe' STORED AS PARQUET
LOCATION 'hdfs:///tmp/dwxtest/';MSCK REPAIR TABLE web_sales;analyze table
web_sales compute statistics for columns; Error Stack: analyze table web_sales
compute statistics for columns;], TaskAttempt 3 failed, info=[Error: Error
while running task ( failure ) :
attempt_1678779198717__2_00_52_3:java.lang.RuntimeException:
java.lang.RuntimeException: java.io.IOException:
org.apache.parquet.io.ParquetDecodingException: Can not read value at 0 in
block -1 in file
s3a://xx/useDecimal=true,useDate=true,filterNull=false/web_sales/ws_sold_date_sk=2451825/part-00796-788bef86-2748-4e21-a464-b34c7e646c94-cfcafd2c-2abd-4067-8aea-f58cb1021b35.c000.snappy.parquet
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:351)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:280)
at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:84)
at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:70)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/javax.security.auth.Subject.doAs(Subject.java:423) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899)
at
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:70)
at
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:40)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at
org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:118)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829) Caused by:
java.lang.RuntimeException: java.io.IOException:
org.apache.parquet.io.ParquetDecodingException: Can not read value at 0 in
block -1 in file
s3a://xxx/useDecimal=true,useDate=true,filterNull=false/web_sales/ws_sold_date_sk=2451825/part-00796-788bef86-2748-4e21-a464-b34c7e646c94-cfcafd2c-2abd-4067-8aea-f58cb1021b35.c000.snappy.parquet
at
org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:206)
at
org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.(TezGroupedSplitsInputFormat.java:145)
at
org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat.getRecordReader(TezGroupedSplitsInputFormat.java:111)
at
org.apache.tez.mapreduce.lib.MRReaderMapred.setupOldRecordReader(MRReaderMapred.java:164)
at
org.apache.tez.mapreduce.lib.MRReaderMapred.setSplit(MRReaderMapred.java:83)
at
org.apache.tez.mapreduce.input.MRInput.initFromEventInternal(MRInput.java:704)
at org.apache.tez.mapreduce.input.MRInput.initFromEvent(MRInput.java:663)
at
org.apache.tez.mapreduce.input.MRInputLegacy.checkAndAwaitRecordReaderInitialization(MRInputLegacy.java:150)
at
org.apache.tez.mapreduce.input.MRInputLegacy.init(MRInputLegac
[jira] [Work logged] (HIVE-27203) Add compaction pending Qtest for Insert-only, Partitioned, Clustered ACID, and combination Tables
[ https://issues.apache.org/jira/browse/HIVE-27203?focusedWorklogId=854431&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854431 ] ASF GitHub Bot logged work on HIVE-27203: - Author: ASF GitHub Bot Created on: 03/Apr/23 07:41 Start Date: 03/Apr/23 07:41 Worklog Time Spent: 10m Work Description: rkirtir commented on code in PR #4181: URL: https://github.com/apache/hive/pull/4181#discussion_r1155589934 ## ql/src/test/queries/clientpositive/compaction_query_based_insert_only_partitioned_clustered.q: ## Review Comment: Is not it better to have major and minor compactions in one file? Issue Time Tracking --- Worklog Id: (was: 854431) Time Spent: 0.5h (was: 20m) > Add compaction pending Qtest for Insert-only, Partitioned, Clustered ACID, > and combination Tables > -- > > Key: HIVE-27203 > URL: https://issues.apache.org/jira/browse/HIVE-27203 > Project: Hive > Issue Type: Test >Reporter: Akshat Mathur >Assignee: Akshat Mathur >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Improve Qtest Coverage for Compaction use cases for ACID Tables: > # Partitioned Tables( Major & Minor ) > # Insert-Only Clustered( Major & Minor ) > # Insert-Only Partitioned( Major & Minor ) > # Insert-Only Clustered and Partitioned( Major & Minor ) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26900) Error message not representing the correct line number with a syntax error in a HQL File
[ https://issues.apache.org/jira/browse/HIVE-26900?focusedWorklogId=854428&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854428 ] ASF GitHub Bot logged work on HIVE-26900: - Author: ASF GitHub Bot Created on: 03/Apr/23 07:32 Start Date: 03/Apr/23 07:32 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4188: URL: https://github.com/apache/hive/pull/4188#issuecomment-1493821873 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4188) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4188&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4188&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4188&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4188&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4188&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4188&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4188&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4188&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4188&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4188&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4188&resolved=false&types=CODE_SMELL) [3 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4188&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4188&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4188&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 854428) Time Spent: 3h 10m (was: 3h) > Error message not representing the correct line number with a syntax error in > a HQL File > > > Key: HIVE-26900 > URL: https://issues.apache.org/jira/browse/HIVE-26900 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.2, 4.0.0-alpha-1, 4.0.0-alpha-2 >Reporter: Vikram Ahuja >Priority: Minor > Labels: pull-request-available > Time Spent: 3h 10m > Remaining Estimate: 0h > > When a wrong syntax is added in a HQL file, the error thrown by beeline while > running the HQL file is having the wrong line number. The line number and > even the position is incorrect. Seems like parser is not considering spaces > and new lines and always throwing the error on line number 1 irrespective of > what line the error is on in the HQL file > > For instance, consider the following test.hql file: > # --comment > # --comment > # SET hive.server2.logging.operation.enabled=true; > # SET hive.serve
[jira] [Work logged] (HIVE-27209) Backport HIVE-24569: LLAP daemon leaks file descriptors/log4j appenders
[ https://issues.apache.org/jira/browse/HIVE-27209?focusedWorklogId=854420&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854420 ] ASF GitHub Bot logged work on HIVE-27209: - Author: ASF GitHub Bot Created on: 03/Apr/23 07:09 Start Date: 03/Apr/23 07:09 Worklog Time Spent: 10m Work Description: guptanikhil007 commented on PR #4193: URL: https://github.com/apache/hive/pull/4193#issuecomment-1493795899 @vihangk1 and @sankarh Can you review this? Issue Time Tracking --- Worklog Id: (was: 854420) Time Spent: 0.5h (was: 20m) > Backport HIVE-24569: LLAP daemon leaks file descriptors/log4j appenders > --- > > Key: HIVE-27209 > URL: https://issues.apache.org/jira/browse/HIVE-27209 > Project: Hive > Issue Type: Sub-task > Components: llap >Affects Versions: 2.2.0 >Reporter: Nikhil Gupta >Assignee: Nikhil Gupta >Priority: Major > Labels: pull-request-available > Fix For: 3.2.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27211) Backport HIVE-22453: Describe table unnecessarily fetches partitions to branch-3
[ https://issues.apache.org/jira/browse/HIVE-27211?focusedWorklogId=854419&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854419 ] ASF GitHub Bot logged work on HIVE-27211: - Author: ASF GitHub Bot Created on: 03/Apr/23 07:08 Start Date: 03/Apr/23 07:08 Worklog Time Spent: 10m Work Description: guptanikhil007 commented on PR #4191: URL: https://github.com/apache/hive/pull/4191#issuecomment-1493795623 @vihangk1 and @sankarh Can you review this? Issue Time Tracking --- Worklog Id: (was: 854419) Time Spent: 0.5h (was: 20m) > Backport HIVE-22453: Describe table unnecessarily fetches partitions to > branch-3 > > > Key: HIVE-27211 > URL: https://issues.apache.org/jira/browse/HIVE-27211 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.1.2 >Reporter: Nikhil Gupta >Assignee: Nikhil Gupta >Priority: Major > Labels: pull-request-available > Fix For: 3.2.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
