Hello Quanlong Huang, Michael Smith, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/24711
to look at the new patch set (#5).
Change subject: IMPALA-15234: Add a way to clear the HBO cache
......................................................................
IMPALA-15234: Add a way to clear the HBO cache
HBO keeps historical runs in a per-coordinator cache inside the
frontend. Once a run that no longer describes the data is stored, for
example after a table is overwritten with a very different
distribution, it keeps influencing plans until it is evicted.
End-to-end tests have the same problem in reverse: a run stored by an
earlier query can satisfy a later assertion.
Add CacheBackend.clear(), reachable from the backend through a new
clearHboStats() JNI entry point, and expose it as the
/clear_hbo_stats WebUI endpoint. As the JIRA suggests, this drops the
cache as a whole: entries are keyed by hash strings, so the ones
belonging to a given table cannot be identified.
The cache lives in the frontend of each coordinator, so the endpoint
clears the coordinator that serves the request. Require POST so the
mutation goes through the existing Webserver CSRF checks. GET returns
an error without changing the cache, and the end-to-end test covers
both paths.
Testing:
- InMemoryCacheBackendTest
- TestHBO::test_clear_hbo_cache
- python3 -m py_compile tests/common/impala_service.py
tests/query_test/test_hbo.py
- git diff --check
Change-Id: Ib58305e96f05ba49eb03d935765620915964308a
Assisted-by: claude-opus-5 (Claude Code)
Assisted-by: gpt-5.6-sol (OpenAI Codex)
---
M be/src/service/frontend.cc
M be/src/service/frontend.h
M be/src/service/impala-http-handler.cc
M be/src/service/impala-http-handler.h
M fe/src/main/java/org/apache/impala/service/CacheBackend.java
M fe/src/main/java/org/apache/impala/service/HistoricalStats.java
M fe/src/main/java/org/apache/impala/service/InMemoryCacheBackend.java
M fe/src/main/java/org/apache/impala/service/JniFrontend.java
A fe/src/test/java/org/apache/impala/service/InMemoryCacheBackendTest.java
M tests/common/impala_service.py
M tests/query_test/test_hbo.py
11 files changed, 181 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/11/24711/5
--
To view, visit http://gerrit.cloudera.org:8080/24711
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ib58305e96f05ba49eb03d935765620915964308a
Gerrit-Change-Number: 24711
Gerrit-PatchSet: 5
Gerrit-Owner: Aleksandr Efimov <[email protected]>
Gerrit-Reviewer: Aleksandr Efimov <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Michael Smith <[email protected]>
Gerrit-Reviewer: Quanlong Huang <[email protected]>
