[
https://issues.apache.org/jira/browse/HIVE-29512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Venugopal Reddy K updated HIVE-29512:
-------------------------------------
Description:
*[Description]*
COMPACTION_QUEUE and COMPLETED_COMPACTIONS table entries are not cleared when
an iceberg table is deleted. Hence such entries can keep on accumulating over
time.
Note: For ACID tables, entries are cleared.
*[Steps to reproduce]*
{code:java}
create database mydb;
use mydb;
create table icet1(i int) stored by iceberg
tblproperties('compactor.threshold.target.size'='1000');
insert into icet1 values(1),(2),(3);
insert into icet1 values(1),(2),(3);
# Wait for COMPACTION_QUEUE to have an entry for table.
#show compactions; => check state as initiated
drop table icet1;
#show compactions; => Still entry exists and it is in initiated state.
{code}
1. If the table is recreated again(without adding data) before compaction is
triggered, compaction fails with below exception. Error message can be seen in
show compactions output.
*java.lang.NullPointerException: Cannot invoke
"org.apache.iceberg.Snapshot.snapshotId()" because the return value of
"org.apache.iceberg.Table.currentSnapshot()" is null*
2. If the table is not created before compaction is triggered, compaction fails
with error message. Error message can be seen in show compactions output -
*NoSuchObjectException(message:hive.mydb.icet1 table not found)*
Note: In both the cases, COMPACTION_QUEUE entry for the table is moved to
COMPLETED_COMPACTIONS upon compaction failure.
was:
*[Description]*
COMPACTION_QUEUE and COMPLETED_COMPACTIONS table entries are not cleared when
an iceberg table is deleted. Hence such entries can keep on accumulating over
time.
*[Steps to reproduce]*
{code:java}
create database mydb;
use mydb;
create table icet1(i int) stored by iceberg
tblproperties('compactor.threshold.target.size'='1000');
insert into icet1 values(1),(2),(3);
insert into icet1 values(1),(2),(3);
# Wait for COMPACTION_QUEUE to have an entry for table.
#show compactions; => check state as initiated
drop table icet1;
#show compactions; => Still entry exists and it is in initiated state.
{code}
1. If the table is recreated again(without adding data) before compaction is
triggered, compaction fails with below exception. Error message can be seen in
show compactions output.
*java.lang.NullPointerException: Cannot invoke
"org.apache.iceberg.Snapshot.snapshotId()" because the return value of
"org.apache.iceberg.Table.currentSnapshot()" is null*
2. If the table is not created before compaction is triggered, compaction fails
with error message. Error message can be seen in show compactions output -
*NoSuchObjectException(message:hive.mydb.icet1 table not found)*
Note: In both the cases, COMPACTION_QUEUE entry for the table is moved to
COMPLETED_COMPACTIONS upon compaction failure.
> COMPACTION_QUEUE and COMPLETED_COMPACTIONS table entries are not cleared upon
> iceberg table delete
> --------------------------------------------------------------------------------------------------
>
> Key: HIVE-29512
> URL: https://issues.apache.org/jira/browse/HIVE-29512
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Reporter: Venugopal Reddy K
> Priority: Major
>
> *[Description]*
> COMPACTION_QUEUE and COMPLETED_COMPACTIONS table entries are not cleared when
> an iceberg table is deleted. Hence such entries can keep on accumulating over
> time.
> Note: For ACID tables, entries are cleared.
> *[Steps to reproduce]*
> {code:java}
> create database mydb;
> use mydb;
> create table icet1(i int) stored by iceberg
> tblproperties('compactor.threshold.target.size'='1000');
> insert into icet1 values(1),(2),(3);
> insert into icet1 values(1),(2),(3);
> # Wait for COMPACTION_QUEUE to have an entry for table.
> #show compactions; => check state as initiated
> drop table icet1;
> #show compactions; => Still entry exists and it is in initiated state.
> {code}
>
> 1. If the table is recreated again(without adding data) before compaction is
> triggered, compaction fails with below exception. Error message can be seen
> in show compactions output.
> *java.lang.NullPointerException: Cannot invoke
> "org.apache.iceberg.Snapshot.snapshotId()" because the return value of
> "org.apache.iceberg.Table.currentSnapshot()" is null*
> 2. If the table is not created before compaction is triggered, compaction
> fails with error message. Error message can be seen in show compactions
> output -
> *NoSuchObjectException(message:hive.mydb.icet1 table not found)*
> Note: In both the cases, COMPACTION_QUEUE entry for the table is moved to
> COMPLETED_COMPACTIONS upon compaction failure.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
