Fly-a-Kite opened a new issue, #23440:
URL: https://github.com/apache/datafusion/issues/23440
### Describe the bug
DataFusion 54.0.0 returns an empty result for a grouped aggregate query
where the aggregate value is `NULL`, followed by `ORDER BY ... NULLS FIRST
LIMIT`.
The grouped aggregate should still produce one output row. The final
`ORDER BY` / `LIMIT` should not remove it.
This looks related to #22190, but I can still reproduce it on the latest
`datafusion==54.0.0` with a smaller reproducer.
### To Reproduce
```python
import datafusion
import pyarrow as pa
from datafusion import SessionContext
print("datafusion", datafusion.__version__)
batch = pa.record_batch(
[
pa.array(["gamma"], type=pa.string()),
pa.array([None], type=pa.float64()),
],
names=["s", "y"],
)
ctx = SessionContext()
ctx.register_record_batches("t0", [[batch]])
query = """
SELECT max_y
FROM (
SELECT s, MAX(y) AS max_y
FROM t0
GROUP BY s
)
ORDER BY max_y DESC NULLS FIRST
LIMIT 3
"""
result = ctx.sql(query).to_pandas()
print(result)
print("row count:", len(result))
### Expected behavior
### Expected behavior
The query should return one row:
max_y
0 NULL
Reason: the input contains one group, s = 'gamma'. MAX(y) over a group
where all y values are NULL should produce one grouped output row with max_y =
NULL.
### Actual behavior
DataFusion 54.0.0 returns zero rows:
Empty DataFrame
Columns: [max_y]
Index: []
row count: 0
### Additional context
### Environment
- datafusion==54.0.0
- pyarrow==24.0.0
- Python 3.12.3
- Linux x86_64
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
