mzabaluev opened a new pull request, #26616: URL: https://github.com/apache/datafusion/pull/26616
Stacked on #26589. ## Rationale for this change When evaluating a multi-branch CASE, the remainder of the batch is filtered at each branch, copying the remaining rows of every column the CASE references. A wide column that is only returned by THEN or ELSE expressions, such as a whole struct, was copied over and over. ## What changes are included in this PR? Only the columns read by the WHEN expressions are now filtered along with the remainder. The other columns are gathered from the input with `take` for the branches whose THEN or ELSE expression reads them, so each row is copied at most once. While not needed, a deferred column stands as a zero-copy slice of matching length, keeping column indices valid. Deferral is not used when the body contains lambdas or leaves that are opaque to projection. A field access in a WHEN expression is now also precomputed when its struct is used as a whole only by THEN or ELSE expressions, so the struct is deferred rather than filtered at each branch. ## Are these changes tested? Adds an SQL case, as well as unit tests to verify correct construction of the data structures and expected operation. Adds a benchmark returning a whole struct from several branches: 36-52% faster. `CASE c2 WHEN 0 THEN 0 ... ELSE n + 1` can measure slower, but it is evaluated with the literal lookup table, which this change does not touch. Its hash map is seeded randomly per process. The jitter for small lookup tables is addressed by #26609. ## Are there any user-facing changes? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
