adriangb commented on code in PR #18207:
URL: https://github.com/apache/datafusion/pull/18207#discussion_r2461379187
##########
datafusion/sqllogictest/test_files/repartition.slt:
##########
@@ -146,3 +146,135 @@ statement ok
DROP TABLE t1;
# End repartition on empty columns test
+
+# Start spilling tests
Review Comment:
This test passes on main but fails on `datafusion-cli` v50:
```
❯ datafusion-cli
DataFusion CLI v50.0.0
> CREATE UNBOUNDED EXTERNAL TABLE annotated_data_infinite2 (
a0 INTEGER,
a INTEGER,
b INTEGER,
c INTEGER,
d INTEGER
)
STORED AS CSV
WITH ORDER (a ASC, b ASC, c ASC)
LOCATION 'datafusion/core/tests/data/window_2.csv'
OPTIONS ('format.has_header' 'true');
SET datafusion.runtime.memory_limit = '12K';
EXPLAIN ANALYZE
SELECT SUM(a) OVER(partition by a, b order by c) as sum1,
SUM(a) OVER(partition by b, a order by c) as sum2,
SUM(a) OVER(partition by a, d order by b) as sum3,
SUM(a) OVER(partition by d order by a) as sum4
FROM annotated_data_infinite2;
0 row(s) fetched.
Elapsed 0.026 seconds.
0 row(s) fetched.
Elapsed 0.001 seconds.
Resources exhausted: Additional allocation failed with top memory consumers
(across reservations) as:
RepartitionExec[Merge 11]#85(can spill: false) consumed 1896.0 B, peak
1896.0 B,
RepartitionExec[Merge 5]#73(can spill: false) consumed 1448.0 B, peak
1448.0 B,
RepartitionExec[Merge 3]#59(can spill: false) consumed 1384.0 B, peak
1384.0 B,
RepartitionExec[Merge 1]#56(can spill: false) consumed 1304.0 B, peak
1304.0 B,
RepartitionExec[8]#48(can spill: false) consumed 1216.0 B, peak 1856.0 B.
Error: Failed to allocate additional 240.0 B for RepartitionExec[Merge 6]
with 0.0 B already allocated for this reservation - 8.0 B remain available for
the total pool
```
It's not this PR that enabled it to pass, it was #18014, but worth adding
anyway.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
