Paul Rogers created IMPALA-8130:
-----------------------------------
Summary: VERIFY_IF_SUBSET gives false positives on dev box
Key: IMPALA-8130
URL: https://issues.apache.org/jira/browse/IMPALA-8130
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Affects Versions: Impala 3.1.0
Reporter: Paul Rogers
Introduced a change that caused a (yet unknown) change in the behavior for
tables partitioned using a column of type double. Ran the pre-review tests and
got a failure:
{noformat}
00:50:52 TestPartitioning.test_partition_col_types[protocol: beeswax |
exec_option: {'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 5000, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
text/none]
00:50:52 [gw2] linux2 -- Python 2.7.12
/home/ubuntu/Impala/bin/../infra/python/env/bin/python
00:50:52 query_test/test_partitioning.py:45: in test_partition_col_types
00:50:52 use_db=unique_database)
00:50:52 common/impala_test_suite.py:501: in run_test_case
00:50:52 self.__verify_results_and_errors(vector, test_section, result,
use_db)
00:50:52 common/impala_test_suite.py:359: in __verify_results_and_errors
00:50:52 replace_filenames_with_placeholder)
00:50:52 common/test_result_verifier.py:449: in verify_raw_results
00:50:52 VERIFIER_MAP[verifier](expected, actual)
00:50:52 common/test_result_verifier.py:239: in verify_query_result_is_subset
00:50:52 assert expected_literal_strings <= actual_literal_strings
00:50:52 E assert Items in expected results not found in actual results:
00:50:52 E ' partitions=10/11 files=10 size=20B'
{noformat}
Notice the last line: we expect to match 10 partitions. This output does not
show the actual output, nor is the actual output saved in any of the logs that
I could find. (This is another nuisance that should be fixed...)
Ran the same test on the same code on my dev box. This time the test passes:
{noformat}
[gw3] PASSED
query_test/test_partitioning.py::TestPartitioning::test_partition_col_types[protocol:
beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 5000, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
text/none] -- 2019-01-27 11:53:16,531 INFO MainThread: Started query
df42090017b0b652:4647c82500000000
{noformat}
However, if I examine the expected and actual output, in file
{{QueryTest/partition-col-types.test}}, I see that the test actually did fail
locally, but was not reported as a failure. Expected results:
{noformat}
---- QUERY
EXPLAIN
SELECT * from all_insert_partition_col_types
WHERE double_col = 1.1
---- RESULTS: VERIFY_IS_SUBSET
'01:EXCHANGE [UNPARTITIONED]'
'|'
'00:SCAN HDFS [$DATABASE.all_insert_partition_col_types]'
' partitions=10/11 files=10 size=20B'
{noformat}
Notice the last line: it is the one that was expected, but not found, in the
pre-review tests. Actual output:
{noformat}
---- RESULTS: VERIFY_IS_SUBSET
...
'PLAN-ROOT SINK'
'|'
'01:EXCHANGE [UNPARTITIONED]'
'|'
'00:SCAN HDFS
[test_partition_col_types_a253a8a3.all_insert_partition_col_types]'
' partition predicates: double_col = 1.1'
' partitions=0/11 files=0 size=0B'
' row-size=42B cardinality=0'
{noformat}
Notice the second-to-last line. It says that we matched zero partitions, which
is a bug because we expect to match 10 as per the earlier lines.
However, despite the error, and that the expected and actual files should not
match, the test run was reported as a success.
To reproduce this, grab patch set 3 from the following patch:
12265 (Draft): IMPALA-8106: Cleanup of literal expression nodes
This is currently a draft; will be published once the bug has been found and
fixed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
