Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/22668 )
Change subject: IMPALA-13901: Make histogram/sample/group_concat ineligible for tuple caching ...................................................................... Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/22668/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/22668/1//COMMIT_MSG@12 PS1, Line 12: ineligible functions to include these variable aggregate functions. > Why are these cases problematic? We should provide a way to enable caching There are two problems: 1. When a non-deterministic result passes through an exchange, it can be an input into cache entries on all the nodes past the exchange. Nodes can evict cache entries at different times, so we could end up having a cache hit on node A but a cache miss on node B. In order to get a sensible result, the recomputed result used on node B needs to match what A had. Non-deterministic functions can't do that. 2. These functions create noise for automated correctness checking. We don't have a good way to handle them other than turning it off for those locations. I will file a JIRA to do more sophisticated handling for non-determinism, so that we can cache in legal locations with a non-deterministic result (i.e. locations that aren't subject to the concern in #1). Until that gets done, the functions should be ineligible to avoid impacts to correctness checking. -- To view, visit http://gerrit.cloudera.org:8080/22668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibb017dd4297a3bb219ffed2bd9abc94e6c5b2605 Gerrit-Change-Number: 22668 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Kurt Deschler <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Yida Wu <[email protected]> Gerrit-Comment-Date: Thu, 22 May 2025 00:25:38 +0000 Gerrit-HasComments: Yes
