Zoltán Borók-Nagy created IMPALA-10663:
------------------------------------------
Summary: Coordinator might observe stale metadata in local catalog
mode
Key: IMPALA-10663
URL: https://issues.apache.org/jira/browse/IMPALA-10663
Project: IMPALA
Issue Type: Bug
Components: Catalog, Frontend
Reporter: Zoltán Borók-Nagy
Coordinator might observe stale metadata in local catalog mode, even if the DDL
operation was executed on the same coordinator.
The issue is reproducible in the dev environment using local catalog mode:
{noformat}
bin/start-impala-cluster.py --impalad_args --use_local_catalog=true
--catalogd_args --catalog_topic_mode=minimal --catalogd_args
--hms_event_polling_interval_s=1
{noformat}
{noformat}
CREATE TABLE doub (d double);
{noformat}
Renaming the table back and forth and querying from it reproduces the issue:
bin/impala-shell.sh -q "alter table doub rename to doub_tmp; select * from
doub_tmp; alter table doub_tmp rename to doub; select * from doub; alter table
doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to
doub; select * from doub; alter table doub rename to doub_tmp; select * from
doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table
doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to
doub; select * from doub; alter table doub rename to doub_tmp; select * from
doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table
doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to
doub; select * from doub; alter table doub rename to doub_tmp; select * from
doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table
doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to
doub; select * from doub; alter table doub rename to doub_tmp; select * from
doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table
doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to
doub; select * from doub; alter table doub rename to doub_tmp; select * from
doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table
doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to
doub; select * from doub; alter table doub rename to doub_tmp; select * from
doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table
doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to
doub; select * from doub; alter table doub rename to doub_tmp; select * from
doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table
doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to
doub; select * from doub; alter table doub rename to doub_tmp; select * from
doub_tmp;alter table doub_tmp rename to doub; select * from doub;"
{noformat}
...
... then, after a while ...
...
Query: alter table doub rename to doub_tmp
+--------------------------+
| summary |
+--------------------------+
| Renaming was successful. |
+--------------------------+
Fetched 1 row(s) in 0.02s
Query: select * from doub_tmp
Query submitted at: 2021-04-20 14:49:52 (Coordinator: http://localhost:25000)
ERROR: AnalysisException: Could not resolve table reference: 'doub_tmp'
Could not execute command: select * from doub_tmp
{noformat}
Then, if I execute the same SELECT a bit later:
{noformat}
$ bin/impala-shell.sh -q "select * from doub_tmp"
Fetched 0 row(s) in 0.12s
{noformat}
I think if the DDL operation succeeds, then the following operations on the
same coordinator should see its effect.
The current behavior makes tests flaky. The workaround is to set sync_ddl=true,
but it makes these operations very slow.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
