Hi all,
The current behavior of dropping a table with HadoopCatalog looks
inconsistent to me. When stored at the default location under table path,
metadata and data will be deleted regardless of the "purge" flag. When
stored elsewhere, they will be left there if not "purge". Is this by design?
@Override
public boolean dropTable(TableIdentifier identifier, boolean purge) {
if (!isValidIdentifier(identifier)) {
throw new NoSuchTableException("Invalid identifier: %s", identifier);
}
Path tablePath = new Path(defaultWarehouseLocation(identifier));
TableOperations ops = newTableOps(identifier);
TableMetadata lastMetadata = ops.current();
try {
if (lastMetadata == null) {
LOG.debug("Not an iceberg table: {}", identifier);
return false;
} else {
if (purge) {
// Since the data files and the metadata files may store in different
locations,
// so it has to call dropTableData to force delete the data file.
CatalogUtil.dropTableData(ops.io(), lastMetadata);
}
return fs.delete(tablePath, true /* recursive */);
}
} catch (IOException e) {
throw new RuntimeIOException(e, "Failed to delete file: %s", tablePath);
}
}
Thanks,
Manu
