eugenegujing opened a new pull request, #7532:
URL: https://github.com/apache/texera/pull/7532
### What changes were proposed in this PR?
One-line fix in `AggregationOperation.scala`: the `max` aggregation starts
its running maximum from the type's minimum value, so its final "was this group
empty?" check must compare against that same minimum value, but it compared
against the type's maximum value instead (the line was copied from `min`, where
that comparison is correct because `min` starts from the maximum).
Because the check looked at the wrong sentinel, two results were silently
wrong while the workflow completed with no error:
- A group with only null values returned the sentinel itself instead of
`null`, e.g. `-2147483648` for INTEGER or `1970-01-01 00:00:00` for TIMESTAMP.
- A true maximum equal to the type's maximum value was mistaken for "no
value seen" and discarded, so `max` over `{1, 5, 2147483647}` reported `5` (the
largest value from the other local aggregation worker) instead of `2147483647`.
#### Before-and-after
Test data: group `g1` has only nulls, `g2` contains `{1, 5, 2147483647}`,
`g3` contains `{10, 42}`.
<img width="1347" height="900" alt="Screenshot 2026-08-07 at 2 27 40 PM"
src="https://github.com/user-attachments/assets/faac65fd-e83f-4952-a0dd-b8109d56334d";
/>
Before the fix, `max(v)` grouped by `k` returns `-2147483648` for `g1`
(expected `null`) and `5` for `g2` (expected `2147483647`):
<img width="1341" height="866" alt="Screenshot 2026-08-07 at 2 27 51 PM"
src="https://github.com/user-attachments/assets/fdf7f045-aa34-44eb-adee-6e722839d7eb";
/>
`min(v)` on the same data is correct (`null` for `g1`, `1` for `g2`, `10`
for `g3`), confirming only `max`'s empty-group check is broken:
<img width="1342" height="868" alt="Screenshot 2026-08-07 at 2 28 00 PM"
src="https://github.com/user-attachments/assets/3df9c390-626f-4f01-9ca6-7e634927386b";
/>
After the fix, `max(v)` returns `null` for `g1`, `2147483647` for `g2`, and
`42` for `g3`:
<img width="1131" height="756" alt="Screenshot 2026-08-07 at 5 54 17 PM"
src="https://github.com/user-attachments/assets/68ba757c-8b4b-4682-b04e-8db740d1583c";
/>
### Any related issues, documentation, discussions?
Closes #7531
The regression was introduced by #1840, which generalised the hard-coded
`Double` sentinels to per-type `minValue`/`maxValue` helpers and updated
`maxAgg`'s initialiser but not its finaliser.
### How was this PR tested?
Added 5 regression tests, all of which fail without the one-line fix and
pass with it (verified in both directions):
- `AggregateOpSpec`: `max` over empty input and over all-null input returns
`null`; `max` keeps a true maximum equal to the type's maximum value, covering
INTEGER, LONG, DOUBLE, and TIMESTAMP.
- `AggregationOperationSpec`: a worker-to-final pipeline via `getFinal`
keeps `Int.MaxValue` when partial results are re-aggregated, reproducing the
two-worker scenario shown above; `max`'s merge stays neutral when one side saw
no values.
Full aggregate suite: `sbt "WorkflowOperator/testOnly
org.apache.texera.amber.operator.aggregate.*"` — 68 tests, all passing.
Also verified end to end in the UI with the workflow shown above
(screenshots are from before and after rebuilding the backend with this fix).
### Was this PR authored or co-authored using generative AI tooling?
Co-authored by: Claude Code (Claude Fable 5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
