Qifan Chen has uploaded a new patch set (#10). (
http://gerrit.cloudera.org:8080/17252 )
Change subject: IMPALA-10532:
TestOverlapMinMaxFilters.test_overlap_min_max_filters seems flaky
......................................................................
IMPALA-10532: TestOverlapMinMaxFilters.test_overlap_min_max_filters seems flaky
This change disables the overlap min/max filter test for hdfs in
erasure coding, due to the query plan change (from 3-node scan to
2-node scan) which splits the row groups among scan nodes differently.
The change also improves how a coordinator behaves when a just
arriving min/max filter is the last one to arrive or is always true.
Previously, the coordinator disables the corresponding filter
representation by setting it to Always True, which makes it
impossible to differentiate a true AlwaysTrue filter (say, set in the
hash join building step) from the one being disabled. A dedicated
Boolean variable minmaxDisabled_ is introduced to record the disabled
state. The Always True state of a filter is never altered. The
enhancement improves the display of the min and max column in
"Filter routing table" and "Final filter table" in profile. These two
columns now display the following possible values.
1. 'PartialUpdates' - The min and the max are partially updated;
2. 'AlwaysTrue' - The filter is always true;
3. 'AlwaysFalse' - The filter is always false;
4. Real values - The filter is neither always true or false,
fully updated with the min/max real values.
A third change introduced is to record, in profile for scan node, the
arrival time of min/max filters (in elapsed time since the system is
rebooted obtained by calling MonotonicMillis()). It can help the
diagnosis of late arrival of filters, when compared with the elpased
time when a row group is filtered with these filters.
Testing:
1. Ran unit tests;
2. Ran core tests.
Change-Id: I326317833979efcbe02ce6c95ad80133dd5c7964
---
M be/src/exec/parquet/hdfs-parquet-scanner.cc
M be/src/exec/scan-node.cc
M be/src/runtime/coordinator-filter-state.h
M be/src/runtime/coordinator.cc
M be/src/util/min-max-filter.cc
M be/src/util/min-max-filter.h
M tests/query_test/test_runtime_filters.py
7 files changed, 112 insertions(+), 18 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/52/17252/10
--
To view, visit http://gerrit.cloudera.org:8080/17252
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I326317833979efcbe02ce6c95ad80133dd5c7964
Gerrit-Change-Number: 17252
Gerrit-PatchSet: 10
Gerrit-Owner: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
