zeroshade opened a new pull request, #1430:
URL: https://github.com/apache/arrow-go/pull/1430
### Rationale for this change
The sort kernels order floats the way Arrow C++ does: numerically with `-0
== +0`, and every NaN (whatever its sign) grouped next to the nulls according
to `NullPlacement`, regardless of `Order`. Some consumers need a deterministic
total order over floats that distinguishes `-0`/`+0` and signed NaNs and treats
NaN as an ordinary value, matching Java's `Double.compare` / IEEE 754
`totalOrder`.
### What changes are included in this PR?
- New `SortKey.FloatTotalOrder bool` field:
- `false` (default): existing behavior, unchanged.
- `true`: IEEE 754 totalOrder for float16/float32/float64:
`-NaN < -Inf < negative values < -0 < +0 < positive values < +Inf <
+NaN`.
NaNs sort as values (not null-like), and `Descending` reverses the whole
order including NaNs.
- Nulls are unaffected: they still follow `NullPlacement` in both modes.
- Non-float columns ignore the field.
- In total-order mode the float comparators compare order-preserving integer
keys derived from the raw bits, and report no null-like values so NaNs are not
partitioned out.
- The single-chunk, no-null radix fast path (previously integers only) now
also handles total-order floats by radix sorting the materialized integer keys.
### Are these changes tested?
Yes:
- `TestSortIndicesFloatTotalOrder`: float64/32/16 × ascending/descending ×
nulls first/last, over the single-array, chunked (per-chunk sort + merge) and
multi-key record batch paths, with -NaN, NaN, ±Inf, interleaved ±0 and a null.
- `TestSortIndicesFloatDefaultOrderingUnchanged`: pins the default behavior.
- `TestSortIndicesSingleChunkRadixFloatTotalOrder`: radix fast path (sliced
chunk, 4096 rows) for all three float widths, checked against an independent
reference comparator.
### Are there any user-facing changes?
Yes, a new opt-in `SortKey.FloatTotalOrder` field. The default is unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
