stackedsax commented on issue #47426:
URL: https://github.com/apache/arrow/issues/47426#issuecomment-5981157854
This was fixed by #50183 (GH-50182), released in Arrow 25.0.0.
`DefaultMin()`/`DefaultMax()` in `cpp/src/parquet/statistics.cc` now seed the
running min/max with `+Inf`/`-Inf` for floating-point types (including
`FLOAT16`) instead of `numeric_limits<T>::max()`/`lowest()`, so an all-`-Inf`
column now reports `[-Inf, -Inf]` as expected. `TestFloatStatistics.Infinities`
in `statistics_test.cc` covers FLOAT, DOUBLE and FLOAT16.
Quick check on 25.0.0 or later:
```python
import io, pyarrow as pa, pyarrow.parquet as pq
inf = float("inf")
buf = io.BytesIO()
pq.write_table(pa.table({"c": pa.array([-inf, -inf, -inf])}), buf)
st = pq.ParquetFile(buf).metadata.row_group(0).column(0).statistics
print(st.min, st.max) # -inf -inf
```
Note for anyone arriving from facebookincubator/velox#14603: Velox vendors a
copy of this file under `velox/dwio/parquet/writer/arrow/Statistics.cpp`, so it
needs to pick up the equivalent change separately.
@thisisnic I think this can be closed as a duplicate of #50182
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
