[jira] [Updated] (HIVE-26526) MSCK sync is not removing partitions with special characters
[ https://issues.apache.org/jira/browse/HIVE-26526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naresh P R updated HIVE-26526: -- Attachment: test.q > MSCK sync is not removing partitions with special characters > > > Key: HIVE-26526 > URL: https://issues.apache.org/jira/browse/HIVE-26526 > Project: Hive > Issue Type: Bug >Reporter: Naresh P R >Priority: Major > Attachments: test.q > > > PARTITIONS table were having encoding string & PARTITION_KEY_VALS were having > original string. > {code:java} > hive=> select * from "PARTITION_KEY_VALS" where "PART_ID" IN (46753, 46754, > 46755, 46756); > PART_ID | PART_KEY_VAL | INTEGER_IDX > -+-+- > 46753 | 2022-02-* | 0 > 46754 | 2011-03-01 | 0 > 46755 | 2022-01-* | 0 > 46756 | 2010-01-01 | 0 > > > hive=> select * from "PARTITIONS" where "TBL_ID" = 23567 ; > PART_ID | CREATE_TIME | LAST_ACCESS_TIME | PART_NAME | SD_ID | > TBL_ID | WRITE_ID > -+-+--+---+---++-- > 46753 | 0 | 0 | part_date=2022-02-%2A | 70195 | > 23567 | 0 > 46754 | 0 | 0 | part_date=2011-03-01 | 70196 | > 23567 | 0 > 46755 | 0 | 0 | part_date=2022-01-%2A | 70197 | > 23567 | 0 > 46756 | 0 | 0 | part_date=2010-01-01 | 70198 | > 23567 | 0 > (4 rows){code} > > 1) DirectSQL has a join condition on PARTITION_KEY_VALS.PART_KEY_VAL = > "2022-02-%2A" at here > https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java#L883 > 2) Jdo is having filter condition on PARTITIONS.PART_NAME = > "part_date=2022-02-%252A" (ie., 2 times url encoded) > Once from HS2 > https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreChecker.java#L353 > 2nd from HMS > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/parser/ExpressionTree.java#L365] > Above conditions returns 0 partitions, so those are not removed from HMS > metadata. > > Attaching repro q file -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-26526) MSCK sync is not removing partitions with special characters
[ https://issues.apache.org/jira/browse/HIVE-26526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naresh P R updated HIVE-26526: -- Issue Type: Bug (was: New Feature) > MSCK sync is not removing partitions with special characters > > > Key: HIVE-26526 > URL: https://issues.apache.org/jira/browse/HIVE-26526 > Project: Hive > Issue Type: Bug >Reporter: Naresh P R >Priority: Major > > PARTITIONS table were having encoding string & PARTITION_KEY_VALS were having > original string. > {code:java} > hive=> select * from "PARTITION_KEY_VALS" where "PART_ID" IN (46753, 46754, > 46755, 46756); > PART_ID | PART_KEY_VAL | INTEGER_IDX > -+-+- > 46753 | 2022-02-* | 0 > 46754 | 2011-03-01 | 0 > 46755 | 2022-01-* | 0 > 46756 | 2010-01-01 | 0 > > > hive=> select * from "PARTITIONS" where "TBL_ID" = 23567 ; > PART_ID | CREATE_TIME | LAST_ACCESS_TIME | PART_NAME | SD_ID | > TBL_ID | WRITE_ID > -+-+--+---+---++-- > 46753 | 0 | 0 | part_date=2022-02-%2A | 70195 | > 23567 | 0 > 46754 | 0 | 0 | part_date=2011-03-01 | 70196 | > 23567 | 0 > 46755 | 0 | 0 | part_date=2022-01-%2A | 70197 | > 23567 | 0 > 46756 | 0 | 0 | part_date=2010-01-01 | 70198 | > 23567 | 0 > (4 rows){code} > > 1) DirectSQL has a join condition on PARTITION_KEY_VALS.PART_KEY_VAL = > "2022-02-%2A" at here > https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java#L883 > 2) Jdo is having filter condition on PARTITIONS.PART_NAME = > "part_date=2022-02-%252A" (ie., 2 times url encoded) > Once from HS2 > https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreChecker.java#L353 > 2nd from HMS > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/parser/ExpressionTree.java#L365] > Above conditions returns 0 partitions, so those are not removed from HMS > metadata. > > Attaching repro q file -- This message was sent by Atlassian Jira (v8.20.10#820010)