[
https://issues.apache.org/jira/browse/HIVE-29498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18064373#comment-18064373
]
lei w commented on HIVE-29498:
------------------------------
[~alexdesiqueira] [~gsaihemanth]
> Incomplete partition values retrieved for multi-level partitioned tables
> during DROP TABLE
> ------------------------------------------------------------------------------------------
>
> Key: HIVE-29498
> URL: https://issues.apache.org/jira/browse/HIVE-29498
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Affects Versions: 1.2.1
> Reporter: lei w
> Priority: Major
>
> Currently, we are using Hive version 1.2.1 and have encountered an issue
> where multi-level partitioned tables occasionally fail to be dropped.
> The specific symptom is that the partition retrieved via MetaStoreDirectSql
> contains a value for only one partition key, while the other partition keys
> are null. Typically, only the first-level partition key has a value, and the
> subsequent levels are empty.
> Consequently, this leads to an exception when calling the
> Warehouse#makePartName method to concatenate partition values later in the
> ObjectStore.
> Does anyone have any insights or workarounds regarding this issue? Any help
> would be greatly appreciated. Thank you!
> {code:java}
> private List<Path> dropPartitionsAndGetLocations(RawStore ms, String
> dbName,
> String tableName, Path tablePath, List<FieldSchema> partitionKeys,
> boolean checkLocation)
> throws MetaException, IOException, NoSuchObjectException,
> InvalidObjectException,
> InvalidInputException {
> int partitionBatchSize = HiveConf.getIntVar(hiveConf,
> ConfVars.METASTORE_BATCH_RETRIEVE_MAX);
> Path tableDnsPath = null;
> if (tablePath != null) {
> tableDnsPath = wh.getDnsPath(tablePath);
> }
> List<Path> partPaths = new ArrayList<Path>();
> Table tbl = ms.getTable(dbName, tableName);
> // call dropPartition on each of the table's partitions to follow the
> // procedure for cleanly dropping partitions.
> while (true) {
> List<Partition> partsToDelete = ms.getPartitions(dbName, tableName,
> partitionBatchSize);
> if (partsToDelete == null || partsToDelete.isEmpty()) {
> break;
> }
> List<String> partNames = new ArrayList<String>();
> for (Partition part : partsToDelete) {
> if (checkLocation && part.getSd() != null &&
> part.getSd().getLocation() != null) {
> Path partPath = wh.getDnsPath(new
> Path(part.getSd().getLocation()));
> if (tableDnsPath == null ||
> (partPath != null && !isSubdirectory(tableDnsPath,
> partPath))) {
> if (!wh.isWritable(partPath.getParent())) {
> throw new MetaException("Table metadata not deleted since the
> partition " +
> Warehouse.makePartName(partitionKeys, part.getValues()) +
> " has parent location " + partPath.getParent() + " which
> is not writable " +
> "by " + hiveConf.getUser());
> }
> partPaths.add(partPath);
> }
> }
> //makePartName throw exception
> partNames.add(Warehouse.makePartName(tbl.getPartitionKeys(),
> part.getValues()));
> }
> // drop partiton
> ms.dropPartitions(dbName, tableName, partNames);
> }
> return partPaths;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
