VenuReddy2103 opened a new pull request, #3899:
URL: https://github.com/apache/hive/pull/3899
### What changes were proposed in this pull request?
**Root cause:** Current implementation of drop partitions is handled through
direct sql along with the fallback to JDO processing upon any failure.
Direct sql is failing due to following issues:
1. Cast exception(from Long to array of Object) in `dropStorageDescriptors`
method while iterating over result fetched for
`SKEWED_VALUES.STRING_LIST_ID_EID`.
2. Cast exception in `dropDanglingColumnDescriptors` method while iterating
over result fetched for `SDS.CD_ID`(issue applicable with oracle DB).
And there is no rollback of the current transaction upon direct sql
processing failure(i.e., other table entries like PARTITIONS etc are already
deleted in transaction before the exception) before trying with JDO. Thus JDO
datanucleus processing is unable to cascade delete all the dependent entries in
respective tables. SDS, SERDES, SERDE_PARAMS, SKEWED_COL_NAMES,
SKEWED_COL_VALUE_LOC_MAP, SKEWED_VALUES remained dangling.
Following changes where introduced in the PR:
1. Upon drop partitions directsql processing failure due to any reason in
the middle of transaction, current transaction is rolled back and a new
transaction is opened prior to drop partitions JDO processing.
2. Fixed cast exception in `dropStorageDescriptors` method while iterating
over result fetched for `SKEWED_VALUES.STRING_LIST_ID_EID`.
3. Fixed cast exception in `dropDanglingColumnDescriptors` method while
iterating over result fetched for `SDS.CD_ID`.
### Why are the changes needed?
When a partitioned table with skewed columns is dropped, appropriate
entries in HMS tables i.e., SDS, SERDES, SERDE_PARAMS, SKEWED_COL_NAMES,
SKEWED_COL_VALUE_LOC_MAP, SKEWED_VALUES are not deleted.
[HIVE-26860](https://issues.apache.org/jira/browse/HIVE-26860). These entries
will remain in the backing datastore forever.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Manually verified from the database client UI for the entries to be deleted
upon drop table.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
