jonathanc-n commented on PR #16660:
URL: https://github.com/apache/datafusion/pull/16660#issuecomment-3203939654
@comphead @2010YOUY01 I have added incremental processing, performance cost
docs, and diagrams in the tests to verify correctness easier. I will make the
incremental processing batch limit to be configurable as a runtime config once
planner is implemented.
I will do the following as follow-up PRs + put it in an EPIC issue (this
pull request was getting quite large and cluttered, probably better to merge
this for now if it looks ready):
- Fuzz test
- Planner + AND predicate handling -> SLT tests (i'd like to do the AND
expression + planner together as right now Datafusion doesn't have too good of
a method to separate IE predicates and equal predicates)
- Mark joins
- Serialization
Thank you very much so far for the reviews, really appreciate it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
