Qifan Chen created IMPALA-11190:
-----------------------------------
Summary: Test failing insert tests are broken in local catalog mode
Key: IMPALA-11190
URL: https://issues.apache.org/jira/browse/IMPALA-11190
Project: IMPALA
Issue Type: Bug
Components: Catalog
Reporter: Qifan Chen
Stress test test_failing_inserts (in tests/stress/test_acid_stress.py) fail
repeatedly in local catalog mode. The concurrent checker query (select * from
<table>) can return rows of value -1 which should not happen.
This is reproducible by commenting out test TestFailingAcidInserts in
tests/stress/test_acid_stress.py.
Setup:
1. Build the impala and clear HMS in case in a bad state:
$IMPALA_HOME/buildall.sh -format_metastore -notests
2. Start the cluster in local catalog mode:
$IMPALA_HOME/bin/start-impala-cluster.py --impalad_args
--use_local_catalog=true --catalogd_args --catalog_topic_mode=minimal
--catalogd_args --hms_event_polling_interval_s=1
3. Run the modified stress test: $IMPALA_HOME/bin/impala-py.test
$IMPALA_TESTS/stress/test_acid_stress.py
Error reported:
Main branch failed on test est_failing_inserts
[11:48:38 qchen@qifan-10229: Impala.03112022] test_acid_stress
rootLoggerLevel = INFO
================================================== test session starts
===================================================
platform linux2 -- Python 2.7.16, pytest-2.9.2, py-1.4.32, pluggy-0.3.1 --
/home/qchen/Impala.03112022/infra/python/env-gcc7.5.0/bin/python
cachedir: tests/.cache
rootdir: /home/qchen/Impala.03112022/tests, inifile: pytest.ini
plugins: xdist-1.17.1, timeout-1.2.1, random-0.2, forked-0.2
timeout: 7200s method: signal
collected 4 items
tests/stress/test_acid_stress.py::TestAcidInsertsBasic::test_read_hive_inserts
PASSED
tests/stress/test_acid_stress.py::TestAcidInsertsBasic::test_read_impala_inserts
PASSED
tests/stress/test_acid_stress.py::TestAcidInsertsBasic::test_partitioned_inserts[unique_database0]
PASSED
tests/stress/test_acid_stress.py::TestFailingAcidInserts::test_failing_inserts[unique_database0]
FAILED
================================================ short test summary info
=================================================
FAIL
tests/stress/test_acid_stress.py::TestFailingAcidInserts::()::test_failing_inserts[unique_database0]
======================================================== FAILURES
========================================================
_____________________________
TestFailingAcidInserts.test_failing_inserts[unique_database0]
______________________________
tests/stress/test_acid_stress.py:387: in test_failing_inserts
self._run_test_failing_inserts(unique_database, is_partitioned)
tests/stress/test_acid_stress.py:376: in _run_test_failing_inserts
run_tasks(writers + checkers)
tests/stress/stress_util.py:45: in run_tasks
pool.map_async(Task.run, tasks).get(timeout_seconds)
../Impala.03082022/toolchain/toolchain-packages-gcc7.5.0/python-2.7.16/lib/python2.7/multiprocessing/pool.py:572:
in get
raise self._value
E assert 1 == 0
E + where 1 = len(['-1'])
E + where ['-1'] = <tests.beeswax.impala_beeswax.ImpalaBeeswaxResult
object at 0x7f62144b6890>.data
------------------------------------------------- Captured stderr setup
--------------------------------------------------
SET
client_identifier=stress/test_acid_stress.py::TestFailingAcidInserts::()::test_failing_inserts[unique_database0];
-- connecting to: localhost:21000
-- connecting to localhost:21050 with impyla
-- 2022-03-16 12:03:05,065 INFO MainThread: Closing active operation
-- connecting to localhost:28000 with impyla
-- 2022-03-16 12:03:05,077 INFO MainThread: Closing active operation
SET
client_identifier=stress/test_acid_stress.py::TestFailingAcidInserts::()::test_failing_inserts[unique_database0];
SET sync_ddl=True;
-- executing against localhost:21000
DROP DATABASE IF EXISTS `test_failing_inserts_b980fc6` CASCADE;
-- 2022-03-16 12:03:05,084 INFO MainThread: Started query
ee4cd7bba1374e44:5133f3bb00000000
SET
client_identifier=stress/test_acid_stress.py::TestFailingAcidInserts::()::test_failing_inserts[unique_database0];
SET sync_ddl=True;
-- executing against localhost:21000
CREATE DATABASE `test_failing_inserts_b980fc6`;
-- 2022-03-16 12:03:05,179 INFO MainThread: Started query
d949fdad1d1e9d19:430dd1ca00000000
-- 2022-03-16 12:03:11,071 INFO MainThread: Created database
"test_failing_inserts_b980fc6" for test ID
"stress/test_acid_stress.py::TestFailingAcidInserts::()::test_failing_inserts[unique_database0]"
-------------------------------------------------- Captured stderr call
--------------------------------------------------
SET SYNC_DDL=true;
-- executing against localhost:21000
drop table if exists test_failing_inserts_b980fc6.test_inserts_fail;
-- 2022-03-16 12:03:11,073 INFO MainThread: Started query
1742bdbc8e07861b:6a973ebe00000000
-- executing against localhost:21000
create table test_failing_inserts_b980fc6.test_inserts_fail (i int)
TBLPROPERTIES (
'transactional_properties' = 'insert_only', 'transactional' = 'true')
;
-- 2022-03-16 12:03:11,136 INFO MainThread: Started query
0d4433060875cadd:04a0ccd500000000
SET
client_identifier=stress/test_acid_stress.py::TestFailingAcidInserts::()::test_failing_inserts[unique_database0];
SET
client_identifier=stress/test_acid_stress.py::TestFailingAcidInserts::()::test_failing_inserts[unique_database0];
-- connecting to: localhost:21000
SET
client_identifier=stress/test_acid_stress.py::TestFailingAcidInserts::()::test_failing_inserts[unique_database0];
SET
client_identifier=stress/test_acid_stress.py::TestFailingAcidInserts::()::test_failing_inserts[unique_database0];
-- connecting to: localhost:21001
SET
client_identifier=stress/test_acid_stress.py::TestFailingAcidInserts::()::test_failing_inserts[unique_database0];
SET
client_identifier=stress/test_acid_stress.py::TestFailingAcidInserts::()::test_failing_inserts[unique_database0];
-- connecting to: localhost:21002
-- connecting to: localhost:21000
SET DEBUG_ACTION=CLIENT_REQUEST_UPDATE_CATALOG:FAIL@1.0;
-- connecting to: localhost:21001
-- connecting to: localhost:21002
SET DEBUG_ACTION=CLIENT_REQUEST_UPDATE_CATALOG:FAIL@1.0;
SET DEBUG_ACTION=;
-- executing against localhost:21000
-- executing against localhost:21000
-- executing against localhost:21001
-- executing against localhost:21002
insert into table test_failing_inserts_b980fc6.test_inserts_fail values (-1);
-- executing against localhost:21001
select * from test_failing_inserts_b980fc6.test_inserts_fail where i != 1;
-- executing against localhost:21002
insert into table test_failing_inserts_b980fc6.test_inserts_fail values (-1);
insert into table test_failing_inserts_b980fc6.test_inserts_fail values (1);
select * from test_failing_inserts_b980fc6.test_inserts_fail where i != 1;
select * from test_failing_inserts_b980fc6.test_inserts_fail where i != 1;
-- 2022-03-16 12:03:17,180 INFO Thread-45: Started query
934a794f6d796f8c:eeaca26300000000
-- 2022-03-16 12:03:17,212 INFO Thread-41: Started query
0c4f3d0571723ada:b2f7a9c100000000
SET DEBUG_ACTION=;
-- executing against localhost:21000
insert into table test_failing_inserts_b980fc6.test_inserts_fail values (1);
-- 2022-03-16 12:03:17,378 INFO Thread-41: Started query
874ae51eb56990da:bdfbddce00000000
-- 2022-03-16 12:03:17,479 INFO Thread-46: Started query
f8429461318ea368:2f43151100000000
-- 2022-03-16 12:03:17,480 INFO Thread-42: Started query
ff4f21882092ffa3:aebc4e5c00000000
-- executing against localhost:21000
insert into table test_failing_inserts_b980fc6.test_inserts_fail values (1);
-- 2022-03-16 12:03:17,497 INFO Thread-41: Started query
904706928ea46ba0:d772197700000000
-- 2022-03-16 12:03:17,502 INFO Thread-43: Started query
824472b15fa78a95:584c068300000000
-- 2022-03-16 12:03:17,502 INFO Thread-44: Started query
c34b7c74bdf3104d:d90fe6ad00000000
-- executing against localhost:21000
insert into table test_failing_inserts_b980fc6.test_inserts_fail values (1);
-- 2022-03-16 12:03:17,613 INFO Thread-41: Started query
9f4006339f2bbf0b:2e42159600000000
SET DEBUG_ACTION=;
-- executing against localhost:21001
insert into table test_failing_inserts_b980fc6.test_inserts_fail values (1);
-- 2022-03-16 12:03:17,694 INFO Thread-46: Started query
e5499d3a175a3d92:3360b65900000000
-- executing against localhost:21000
select * from test_failing_inserts_b980fc6.test_inserts_fail where i != 1;
-- 2022-03-16 12:03:17,790 INFO Thread-45: Started query
76411bd974b8ba88:7559ffbd00000000
-- executing against localhost:21001
insert into table test_failing_inserts_b980fc6.test_inserts_fail values (1);
-- 2022-03-16 12:03:17,822 INFO Thread-46: Started query
1b49a91212435590:07ad032900000000
-- closing connection to: localhost:21000
Traceback (most recent call last):
File "/home/qchen/Impala.03112022/tests/stress/stress_util.py", line 35, in
run
return self.func(*self.args, **self.kwargs)
File "/home/qchen/Impala.03112022/tests/stress/test_acid_stress.py", line
353, in _impala_role_checker
assert len(result.data) == 0
AssertionError: assert 1 == 0
+ where 1 = len(['-1'])
+ where ['-1'] = <tests.beeswax.impala_beeswax.ImpalaBeeswaxResult object
at 0x7f62144b6890>.data
========================================== 1 failed, 3 passed in 395.45 seconds
==========================================
[12:03:25 qchen@qifan-10229: Impala.03112022] git branch
IMPALA-10992-auto-scaling-planner-support
* master
[12:07:16 qchen@qifan-10229: Impala.03112022]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
