sryza opened a new pull request, #51352: URL: https://github.com/apache/spark/pull/51352
### What changes were proposed in this pull request? This PR inverts the relationship between event callbacks and `PipelineRunEventBuffer`. Prior to this PR: - `PipelineUpdateContext` had a `PipelineRunEventBuffer`, that all events from the run were written to. - This `PipelineRunEventBuffer` contained an `eventCallback` val which the Connect backend used to forward events to the client - The only code that reads from the buffer is test code After this PR: - `PipelineUpdateContext` has an `eventCallback`, which is invoked on all events. - The test harness constructs a `PipelineRunEventBuffer` and passes its `addEvent` method to the `eventCallback` ### Why are the changes needed? With the prior code, there was a risk of memory pressure from the event buffer for long-running pipelines. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Existing tests ### Was this patch authored or co-authored using generative AI tooling? No -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
