Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/24822 )
Change subject: IMPALA-14598: Store HBO cache in Redis/Valkey ...................................................................... Patch Set 3: (2 comments) http://gerrit.cloudera.org:8080/#/c/24822/2/fe/src/main/java/org/apache/impala/service/HistoricalStats.java File fe/src/main/java/org/apache/impala/service/HistoricalStats.java: http://gerrit.cloudera.org:8080/#/c/24822/2/fe/src/main/java/org/apache/impala/service/HistoricalStats.java@141 PS2, Line 141: (HistoricalStatsValue<TPlanNodeRun>) cacheBackend_.getIfPresent( > Could we make this update atomic now? Coordinators can still overwrite each It's an existing issue that even in a single coordinator, multi-threads in the unregistration thread pool could write to the same key and overwrite each other's value list. Ensuring atomic needs support from the cache backend. So I plan to do this later after adding enough backends. We might need to change the interface as well. Batch writes (and reads) is a different thing. I meant we can consider this optimization together with atomic support in a follow-up work. Note that atomic support will complicate batch reads & writes which I think is more important than atomic support. Without atomic support, the new writer still has chance to add the previously overwritten value. HBO stats are best effort and we don't want read/write on it significantly slow down normal process (e.g., query planning, flusing profiles). Filed IMPALA-15352 and IMPALA-15353 as follow-up tasks. http://gerrit.cloudera.org:8080/#/c/24822/2/fe/src/main/java/org/apache/impala/service/RedisCacheBackend.java File fe/src/main/java/org/apache/impala/service/RedisCacheBackend.java: http://gerrit.cloudera.org:8080/#/c/24822/2/fe/src/main/java/org/apache/impala/service/RedisCacheBackend.java@81 PS2, Line 81: */ > Timeouts still add up per node and strategy. Could we bound total HBO looku This is originally planned in MPALA-15037 and independent to the cache backend implementation. I'd like to add this in a separate patch to avoid complicating this patch. -- To view, visit http://gerrit.cloudera.org:8080/24822 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I43a171bcd436f57bcff14ceaaaa98c0f7dcec769 Gerrit-Change-Number: 24822 Gerrit-PatchSet: 3 Gerrit-Owner: Quanlong Huang <[email protected]> Gerrit-Reviewer: Aleksandr Efimov <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Comment-Date: Tue, 15 Sep 2026 09:10:48 +0000 Gerrit-HasComments: Yes
