2010YOUY01 commented on issue #23974:
URL: https://github.com/apache/datafusion/issues/23974#issuecomment-5364547156
> * First separating regular joins from semi/anti/mark joins (preserving the
existing polling behavior/spilling/state machines)
> * Then converting the streams to generators independently
If you decide to split the semi/anti/mark joins, perhaps we can
PR1: Keep existing implementation, and implement a new stream for
semi/anti/mark joins directly with the generator pattern
```
if !standard_join:
SemiAntiStream
else:
NestedLoopStream
```
PR2: Implement the standard join similarly, and delete the legacy
implementation
If the generator pattern produces overall simpler solution, then directly
implement it this way might be easier 🤔, but anyway this is still a full
rewrite, can be a little tricky, we can try and see how to split the work down
to smaller PRs further.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
