jiwen624 opened a new pull request, #57289:
URL: https://github.com/apache/spark/pull/57289
### What changes were proposed in this pull request?
Fix the MatchError on empty input with changes below:
- Fall back to the statically-known item data type carried by the state
struct (`uncheckedItemDataType`) when no input sketch resolved it, in both
`eval` and `serialize`.
- When combine size is unspecified and no input was consumed,
`maxItemsTracked` is still the internal `VOID` sentinel (-1); resolve it to the
default in `eval` only (a partial buffer must keep `VOID` for the final merge),
so the empty output sketch carries a valid size that downstream functions such
as `approx_top_k_estimate` accept.
### Why are the changes needed?
The query aborts with an internal error instead of returning an empty
result, unlike the sibling functions (`approx_top_k`,
`approx_top_k_accumulate`, `approx_top_k_estimate`) which handle empty input
correctly:
```sql
SELECT approx_top_k_combine(sketch, 5)
FROM (SELECT approx_top_k_accumulate(expr) AS sketch
FROM VALUES (1), (2) AS t(expr))
WHERE false;
-- before: org.apache.spark.SparkException ... scala.MatchError: null
-- after: returns a single empty sketch (estimates to [])
```
### Does this PR introduce _any_ user-facing change?
Yes. Fixed an query failing issue when the input is empty.
### How was this patch tested?
Added UT case
### Was this patch authored or co-authored using generative AI tooling?
Yes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
