Wang Jiangkun created HIVE-26679:
------------------------------------
Summary: [Hive] Drops archive partitions error
Key: HIVE-26679
URL: https://issues.apache.org/jira/browse/HIVE-26679
Project: Hive
Issue Type: Bug
Reporter: Wang Jiangkun
When the partition table is converted to an archive partition, the drop
partition times are Error. it seems there is an issue occurs, which is
{code:java}
set hive.archive.enabled=true;
alter table tb1 archive partition(city="nanjing");{code}
{code:java}
2022-10-27 16:55:41,872 ERROR hive.ql.exec.DDLTask:
org.apache.hadoop.hive.ql.metadata.HiveException: Got exception:
java.io.IOException Invalid path for the Har Filesystem. No index file in
har://hdfs-nameservice/hive/tb1/city=nanjing/data.har
at
org.apache.hadoop.hive.ql.metadata.Hive.dropPartitions(Hive.java:2364)
at
org.apache.hadoop.hive.ql.metadata.Hive.dropPartitions(Hive.java:2345)
at
org.apache.hadoop.hive.ql.exec.DDLTask.dropPartitions(DDLTask.java:3900)
at
org.apache.hadoop.hive.ql.exec.DDLTask.dropTableOrPartitions(DDLTask.java:3860)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:368)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2130)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1801)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1501)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1206)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1194)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: MetaException(message:Got exception: java.io.IOException Invalid
path for the Har Filesystem. No index file in
har://hdfs-nameservice/hive/tb1/city=nanjing/data.har)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$drop_partitions_req_result$drop_partitions_req_resultStandardScheme.read(ThriftHiveMetastore.java)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$drop_partitions_req_result$drop_partitions_req_resultStandardScheme.read(ThriftHiveMetastore.java)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$drop_partitions_req_result.read(ThriftHiveMetastore.java)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_drop_partitions_req(ThriftHiveMetastore.java:2081)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.drop_partitions_req(ThriftHiveMetastore.java:2068)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.dropPartitions(HiveMetaStoreClient.java:1008)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:173)
at com.sun.proxy.$Proxy29.dropPartitions(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:2332)
at com.sun.proxy.$Proxy29.dropPartitions(Unknown Source)
at
org.apache.hadoop.hive.ql.metadata.Hive.dropPartitions(Hive.java:2358)
... 23 more 2022-10-09T10:36:32,271 INFO [573c4ce0-f73c-439b-829d{code}
After the archiving operation is performed, the corresponding partition
metadata information has changed
{code:java}
Partition Value: [nanjing]
Database: default
Table: tb1
CreateTime: Mon Oct 31 08:53:47 CST 2022
LastAccessTime: UNKNOWN
Location: har://hdfs-nameservice/hive/tb1/city=nanjing/data.har
Partition Parameters:
archiving_level 1
is_archived true
numFiles 2
original_location hdfs://nameservice/hive/tb1/city=nanjing
totalSize 32
transient_lastDdlTime 1667177954
{code}
When the drop partition is performed, an error occurs because the
original_location : hdfs://nameservice/hive/tb1/city=nanjing is deleted first,
and then the directory does not exist when the Location:
har://hdfs-nameservice/hive/tb1/city=nanjing/data.har is deleted.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
