lei w created HIVE-29498:
----------------------------
Summary: Incomplete partition values retrieved for multi-level
partitioned tables during DROP TABLE
Key: HIVE-29498
URL: https://issues.apache.org/jira/browse/HIVE-29498
Project: Hive
Issue Type: Bug
Components: Hive
Affects Versions: 1.2.1
Reporter: lei w
Currently, we are using Hive version 1.2.1 and have encountered an issue where
multi-level partitioned tables occasionally fail to be dropped.
The specific symptom is that the partition retrieved via MetaStoreDirectSql
contains a value for only one partition key, while the other partition keys are
null. Typically, only the first-level partition key has a value, and the
subsequent levels are empty.
Consequently, this leads to an exception when calling the
Warehouse#makePartName method to concatenate partition values later in the
ObjectStore.
Does anyone have any insights or workarounds regarding this issue? Any help
would be greatly appreciated. Thank you!
{code:java}
private List<Path> dropPartitionsAndGetLocations(RawStore ms, String dbName,
String tableName, Path tablePath, List<FieldSchema> partitionKeys,
boolean checkLocation)
throws MetaException, IOException, NoSuchObjectException,
InvalidObjectException,
InvalidInputException {
int partitionBatchSize = HiveConf.getIntVar(hiveConf,
ConfVars.METASTORE_BATCH_RETRIEVE_MAX);
Path tableDnsPath = null;
if (tablePath != null) {
tableDnsPath = wh.getDnsPath(tablePath);
}
List<Path> partPaths = new ArrayList<Path>();
Table tbl = ms.getTable(dbName, tableName);
// call dropPartition on each of the table's partitions to follow the
// procedure for cleanly dropping partitions.
while (true) {
List<Partition> partsToDelete = ms.getPartitions(dbName, tableName,
partitionBatchSize);
if (partsToDelete == null || partsToDelete.isEmpty()) {
break;
}
List<String> partNames = new ArrayList<String>();
for (Partition part : partsToDelete) {
if (checkLocation && part.getSd() != null &&
part.getSd().getLocation() != null) {
Path partPath = wh.getDnsPath(new Path(part.getSd().getLocation()));
if (tableDnsPath == null ||
(partPath != null && !isSubdirectory(tableDnsPath, partPath))) {
if (!wh.isWritable(partPath.getParent())) {
throw new MetaException("Table metadata not deleted since the
partition " +
Warehouse.makePartName(partitionKeys, part.getValues()) +
" has parent location " + partPath.getParent() + " which is
not writable " +
"by " + hiveConf.getUser());
}
partPaths.add(partPath);
}
}
//makePartName throw exception
partNames.add(Warehouse.makePartName(tbl.getPartitionKeys(),
part.getValues()));
}
// drop partiton
ms.dropPartitions(dbName, tableName, partNames);
}
return partPaths;
}
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
