mithuncy commented on issue #23823:
URL: https://github.com/apache/datafusion/issues/23823#issuecomment-5173015872
Thanks @saadtajwar! I did some digging and have a local fix for this.
The issue is the large recursive `try_from_logical_plan` match: in debug, it
ends up carrying temporaries from many unrelated arms on every recursive call.
The original repro was around 194 KB per entry on macOS/AArch64.
My debug-side fix is to run each match arm through a small no-inline closure
boundary:
```rust
#[cfg_attr(debug_assertions, inline(never))]
fn serialize_logical_plan_arm<F>(f: F) -> Result<LogicalPlanNode>
where
F: FnOnce() -> Result<LogicalPlanNode>,
{
f()
}
// Each match arm becomes:
$pattern => serialize_logical_plan_arm(|| { /* existing arm body */ })
```
This keeps the large locals for `Join`, `TableScan`, etc. out of the
recursive dispatcher frame. I also added `recursive` stack-growth protection at
`try_from_logical_plan` and a 2 MiB-stack regression test.
I’m planning to put up a patch tomorrow. If you want to take this sooner,
please go ahead — I’m happy to test and review your PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
