aglinxinyuan opened a new pull request, #4424: URL: https://github.com/apache/texera/pull/4424
### What changes were proposed in this PR? This PR fixes Python state processing in `MainLoop` / `DataProcessor` so an operator can process multiple input states in sequence. The root cause is that the state-processing path was missing context switches that keep `MainLoop` and `DataProcessor` in sync. Without those handoffs, the first state can be processed, but the second state does not get delivered correctly. This PR adds the missing `_switch_context()` calls: - before `MainLoop` starts processing an input state - before `DataProcessor` invokes `executor.process_state(...)` - before `MainLoop` continues internal-marker handling after ECM processing These changes keep the state-processing handshake aligned across repeated state inputs. ### Any related issues, documentation, discussions? Closes #4421 ### How was this PR tested? Added a regression test in `amber/src/main/python/core/runnables/test_main_loop.py`: - `test_process_state_can_emit_multiple_states` Ran: - `pytest amber/src/main/python/core/runnables/test_main_loop.py -k process_state_can_emit_multiple_states` - `pytest amber/src/main/python/core/runnables/test_main_loop.py` The regression test passes on this branch and fails on `main`, confirming that without the context-switch changes the operator cannot process two states correctly. ### Was this PR authored or co-authored using generative AI tooling? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
