dekuu5 opened a new pull request, #20067:
URL: https://github.com/apache/datafusion/pull/20067
## Which issue does this PR close?
- Closes #20027
## Rationale for this change
The SpillPool test spill::spill_pool::channel was failing
indeterministically due to a race condition between SpillFile's coordination
logic and the spawn_buffered background task used in the stream reader.
## What changes are included in this PR?
read_spill_as_stream now returns a normal stream not a buffered stream
## Are these changes tested?
No, the fix is for an existing test mentioned in the issue. I wasn't able to
find the bug initially, but I found it by stress-testing the original test 100
times in parallel.
## Are there any user-facing changes?
no
i still think we should add a buffer stream that is aware of the
coordination of the writer and reader but i can't warp my head around it yet.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
