liu created KUDU-3370:
-------------------------
Summary: drop table failed
Key: KUDU-3370
URL: https://issues.apache.org/jira/browse/KUDU-3370
Project: Kudu
Issue Type: Bug
Affects Versions: 1.15.0
Environment: hdp :3.2.3
hive :3.1.1
kudu :1.15.0
trino:369-SNAPSHOT
Reporter: liu
Fix For: NA
At present, according to the patch provided by hive-16993, hive 3.1.1 is
recompiled. Now it can successfully synchronize with hive metadata and create
tables. However, when deleting tables, the log shows that the tables have been
successfully deleted, but the tables in kudu still exist and cannot be deleted。
{code:java}
USE
trino:test2> CREATE TABLE users (
-> userid int WITH (primary_key = true),
-> firstname varchar,
-> lastname varchar
-> ) WITH (
-> partition_by_hash_columns = ARRAY['userid'],
-> partition_by_hash_buckets = 2
-> );
CREATE TABLE
trino:test2> show tables;
Table
-------
users
(1 row)
trino:test2> drop table users;
DROP TABLE
trino:test2> show tables;
Table
-------
users
(1 row)Query 20220523_152932_00147_ytvpx, FINISHED, 3 nodes
Splits: 20 total, 20 done (100.00%)
0.23 [1 rows, 20B] [4 rows/s, 86B/s]trino:test2> drop table users;
Query 20220523_152946_00148_ytvpx failed: failed to drop Hive Metastore table:
TException - service has thrown: NoSuchObjectException(message=hive.test2.users
table not found) {code}
The metadata of hive has been successfully deleted when I delete it for the
first time. When I delete it for the second time, it is reported that the
metadata does not exist because it has been deleted, but the table still exists
in kudu. Use the command:
{code:java}
kudu table delete gpnode1 test2.users{code}
error message：
{code:java}
# kudu table delete gpnode1 test2.users
# Not found: failed to drop Hive Metastore table: TException - service has
thrown: NoSuchObjectException(message=hive.test2.users table not found) {code}
log：
{code:java}
W0523 23:21:52.950664 11595 master_path_handlers.cc:837] Unable to fetch master
addresses: OK
W0523 23:22:31.867633 11595 master_path_handlers.cc:837] Unable to fetch master
addresses: OK
I0523 23:28:41.818248 5917 catalog_manager.cc:1681] Servicing CreateTable
request from {username='presto'} at 192.168.2.182:36402:
name: "test2.users"
schema {
columns {
name: "userid"
type: INT32
is_key: true
is_nullable: false
cfile_block_size: 0
}
columns {
name: "firstname"
type: STRING
is_key: false
is_nullable: false
cfile_block_size: 0
}
columns {
name: "lastname"
type: STRING
is_key: false
is_nullable: false
cfile_block_size: 0
}
}
partition_schema {
hash_bucket_schemas {
columns {
name: "userid"
}
num_buckets: 2
seed: 0
}
}
I0523 23:28:41.877279 5917 catalog_manager.cc:1963] created HMS catalog entry
for table [id=3a50aa7d5aa6416f8ab5587ed16bc85a]
I0523 23:28:42.091735 5907 maintenance_manager.cc:382] P
a01491afc9f24dc3adef5d9cbc4ff405: Scheduling
FlushMRSOp(00000000000000000000000000000000): perf score=0.302975
I0523 23:28:42.109527 5906 maintenance_manager.cc:603] P
a01491afc9f24dc3adef5d9cbc4ff405: FlushMRSOp(00000000000000000000000000000000)
complete. Timing: real 0.018s user 0.003s sys 0.003s Metrics:
{"bytes_written":5660,"cfile_init":1,"dirs.queue_time_us":90,"dirs.run_cpu_time_us":112,"dirs.run_wall_time_us":667,"drs_written":1,"fdatasync":13,"fdatasync_us":11633,"lbm_read_time_us":25,"lbm_reads_lt_1ms":4,"lbm_write_time_us":133,"lbm_writes_lt_1ms":27,"rows_written":3,"thread_start_us":59,"threads_started":1}
I0523 23:28:42.109880 5907 maintenance_manager.cc:382] P
a01491afc9f24dc3adef5d9cbc4ff405: Scheduling
CompactRowSetsOp(00000000000000000000000000000000): perf score=0.212344
I0523 23:28:42.133885 5906 maintenance_manager.cc:603] P
a01491afc9f24dc3adef5d9cbc4ff405:
CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real
0.024s user 0.008s sys 0.004s Metrics:
{"bytes_written":213198,"cfile_cache_hit":13,"cfile_cache_hit_bytes":206138,"cfile_cache_miss":35,"cfile_cache_miss_bytes":225520,"cfile_init":10,"delta_iterators_relevant":6,"dirs.queue_time_us":502,"dirs.run_cpu_time_us":810,"dirs.run_wall_time_us":950,"drs_written":1,"fdatasync":15,"fdatasync_us":12534,"lbm_read_time_us":662,"lbm_reads_lt_1ms":75,"lbm_write_time_us":530,"lbm_writes_lt_1ms":32,"num_input_rowsets":2,"rows_written":209,"spinlock_wait_cycles":2944,"thread_start_us":397,"threads_started":4}
I0523 23:28:44.408016 5917 catalog_manager.cc:4733] T
273e6e3223f14ea690621e40c53c8c0e P 18c9a3f5aa314a2b80dfa970003e4350 reported
cstate change: term changed from 0 to 1, leader changed from <none> to
18c9a3f5aa314a2b80dfa970003e4350 (192.168.2.211). New cstate: current_term: 1
leader_uuid: "18c9a3f5aa314a2b80dfa970003e4350" committed_config { opid_index:
-1 OBSOLETE_local: false peers { permanent_uuid:
"524c9034d0b540cab9161d44a5dab7ef" member_type: VOTER last_known_addr { host:
"192.168.2.184" port: 7050 } health_report { overall_health: UNKNOWN } } peers
{ permanent_uuid: "18c9a3f5aa314a2b80dfa970003e4350" member_type: VOTER
last_known_addr { host: "192.168.2.211" port: 7050 } health_report {
overall_health: HEALTHY } } peers { permanent_uuid:
"9ad2b92eba814e78895e645f74618bd2" member_type: VOTER last_known_addr { host:
"192.168.2.182" port: 7050 } health_report { overall_health: UNKNOWN } } }
I0523 23:28:59.145552 5917 catalog_manager.cc:4733] T
e0d2911469d147609abcdcebffacc7f1 P 18c9a3f5aa314a2b80dfa970003e4350 reported
cstate change: term changed from 0 to 1, leader changed from <none> to
18c9a3f5aa314a2b80dfa970003e4350 (192.168.2.211). New cstate: current_term: 1
leader_uuid: "18c9a3f5aa314a2b80dfa970003e4350" committed_config { opid_index:
-1 OBSOLETE_local: false peers { permanent_uuid:
"18c9a3f5aa314a2b80dfa970003e4350" member_type: VOTER last_known_addr { host:
"192.168.2.211" port: 7050 } health_report { overall_health: HEALTHY } } peers
{ permanent_uuid: "9ad2b92eba814e78895e645f74618bd2" member_type: VOTER
last_known_addr { host: "192.168.2.182" port: 7050 } health_report {
overall_health: UNKNOWN } } peers { permanent_uuid:
"524c9034d0b540cab9161d44a5dab7ef" member_type: VOTER last_known_addr { host:
"192.168.2.184" port: 7050 } health_report { overall_health: UNKNOWN } } }
I0523 23:29:28.640434 5917 catalog_manager.cc:2253] Servicing DeleteTable
request from {username='presto'} at 192.168.2.182:36402:
table { table_name: "test2.users" }
I0523 23:29:28.706779 5917 catalog_manager.cc:2293] deleted HMS catalog entry
for table test2.users [id=3a50aa7d5aa6416f8ab5587ed16bc85a]
I0523 23:29:33.955879 5917 catalog_manager.cc:4733] T
e0d2911469d147609abcdcebffacc7f1 P 9ad2b92eba814e78895e645f74618bd2 reported
cstate change: term changed from 1 to 2, leader changed from
18c9a3f5aa314a2b80dfa970003e4350 (192.168.2.211) to
9ad2b92eba814e78895e645f74618bd2 (192.168.2.182). New cstate: current_term: 2
leader_uuid: "9ad2b92eba814e78895e645f74618bd2" committed_config { opid_index:
-1 OBSOLETE_local: false peers { permanent_uuid:
"18c9a3f5aa314a2b80dfa970003e4350" member_type: VOTER last_known_addr { host:
"192.168.2.211" port: 7050 } health_report { overall_health: UNKNOWN } } peers
{ permanent_uuid: "9ad2b92eba814e78895e645f74618bd2" member_type: VOTER
last_known_addr { host: "192.168.2.182" port: 7050 } health_report {
overall_health: HEALTHY } } peers { permanent_uuid:
"524c9034d0b540cab9161d44a5dab7ef" member_type: VOTER last_known_addr { host:
"192.168.2.184" port: 7050 } health_report { overall_health: UNKNOWN } } }
I0523 23:29:46.440908 5917 catalog_manager.cc:2253] Servicing DeleteTable
request from {username='presto'} at 192.168.2.182:36402:
table { table_name: "test2.users" }
W0523 23:29:46.446835 5917 catalog_manager.cc:2296] failed to delete HMS
catalog entry for table test2.users [id=3a50aa7d5aa6416f8ab5587ed16bc85a]: Not
found: failed to drop Hive Metastore table: TException - service has thrown:
NoSuchObjectException(message=hive.test2.users table not found)
I0523 23:30:44.479030 5907 maintenance_manager.cc:382] P
a01491afc9f24dc3adef5d9cbc4ff405: Scheduling
FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033352
I0523 23:30:44.493083 5906 maintenance_manager.cc:603] P
a01491afc9f24dc3adef5d9cbc4ff405:
FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real
0.014s user 0.001s sys 0.000s Metrics:
{"bytes_written":476,"delete_count":0,"fdatasync":3,"fdatasync_us":12119,"lbm_write_time_us":23,"lbm_writes_lt_1ms":4,"reinsert_count":0,"thread_start_us":21,"threads_started":1,"update_count":3,"wal-append.queue_time_us":63}
I0523 23:30:44.493170 5907 maintenance_manager.cc:382] P
a01491afc9f24dc3adef5d9cbc4ff405: Scheduling
MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I0523 23:30:44.498075 5906 maintenance_manager.cc:603] P
a01491afc9f24dc3adef5d9cbc4ff405:
MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real
0.005s user 0.001s sys 0.000s Metrics:
{"cfile_cache_hit":1,"cfile_cache_hit_bytes":29,"cfile_cache_miss":3,"cfile_cache_miss_bytes":1283,"cfile_init":1,"delete_count":146,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"fdatasync":3,"fdatasync_us":3378,"lbm_read_time_us":5,"lbm_reads_lt_1ms":7,"lbm_write_time_us":11,"lbm_writes_lt_1ms":4,"reinsert_count":0,"thread_start_us":16,"threads_started":1,"update_count":3}
I0524 00:00:48.336750 5917 catalog_manager.cc:2253] Servicing DeleteTable
request from {username='root'} at 192.168.2.182:42712:
table { table_name: "test2.users" } modify_external_catalogs: true
W0524 00:00:48.349134 5917 catalog_manager.cc:2296] failed to delete HMS
catalog entry for table test2.users [id=3a50aa7d5aa6416f8ab5587ed16bc85a]: Not
found: failed to drop Hive Metastore table: TException - service has thrown:
NoSuchObjectException(message=hive.test2.users table not found)
{code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
