eugenegujing opened a new pull request, #7749:
URL: https://github.com/apache/texera/pull/7749
### What changes were proposed in this PR?
Adds `test_async_rpc_handler_initializer.py` (50 tests) for
`core/architecture/rpc/async_rpc_handler_initializer.py` — the class whose body
is `pass` and whose mixin list is the entire wiring between the generated
`WorkerService` and the Python worker's control handlers. Nothing covered it
before: `test_async_rpc_server.py` deliberately substitutes a stub initializer,
so a proto RPC added without a Python handler was only discovered at runtime.
Expectations are derived from the generated code rather than hand-listed, so
a newly generated RPC joins the parametrization on its own and fails until a
handler exists. The spec pins:
- every declared RPC resolves to an async coroutine on a `ControlHandler`
mixin, taking exactly the request type and returning exactly the reply type the
generated mapping declares;
- the deliberately-unimplemented set is exactly `prepare_checkpoint`,
`finalize_checkpoint`, `retrieve_state` and `flush_network_buffer`, as a set
equality in both directions — implementing one without updating the set fails
too. (One correction to the issue text: `flush_network_buffer` is a Scala-side
network concern, not checkpoint-related, so the set is four RPCs rather than
"the checkpoint-related ones".)
- no RPC is defined by more than one mixin — an MRO-shadowed duplicate is
dead code invisible to every other check;
- every `ControlHandler` subclass on disk is wired into the initializer's
bases — the one check not derived from the MRO, so a handler that was written
but never wired is reported by name;
- every request type is registered in `ControlRequest`'s oneof, and the
reply types missing from `ControlReturn`'s oneof are exactly
`evaluate_python_expression`. That last one is a real, currently-broken shape:
the worker service declares `EvaluatedValue` as its reply but the oneof only
registers the coordinator-side wrapper, so `set_one_of` packs an empty
`ControlReturn` and the answer is silently dropped. The fix is a `.proto`
change and out of scope here; pinning the current shape forces whoever fixes it
to clean up the exception set in the same change.
- the real `AsyncRPCServer` registers one collision-free lowercased lookup
key per RPC, each bound to the real initializer, `NoOperation` round-trips end
to end through the unmodified production wiring, and each unimplemented RPC
replies with an `UNIMPLEMENTED` `ControlError` instead of taking the worker
down.
Not redundant with `test_async_rpc_server.py`: that spec pins the server's
dispatch and packing mechanics against a stub (including `set_one_of`
swallowing a wrong-typed reply, exercised with a synthetic type); this one pins
the real wiring and names which actual RPC hits that mechanism. The Scala twin
of this file gained its spec in #7729; this is the Python side.
### Any related issues, documentation, discussions?
Closes #7747.
### How was this PR tested?
50 new tests, all green locally:
```
cd amber
pytest
src/test/python/core/architecture/rpc/test_async_rpc_handler_initializer.py -q
# 50 passed
pytest src/test/python/core/architecture/ -q
# 341 passed
pytest -m "not integration" -q
# 1106 passed
ruff check src/main/python src/test/python && ruff format --check
src/main/python src/test/python
```
The only full-suite failures are the pre-existing
`core/storage/iceberg/test_iceberg_document.py` items that need a local catalog
stack; they fail identically without this PR.
The tests were also mutation-checked: removing a mixin from the bases,
renaming a handler method away from the proto name, turning a handler `async
def` into a plain `def`, adding a new RPC to the generated base without a
handler, implementing one of the acknowledged-unimplemented RPCs, duplicating
an RPC on a second mixin, hiding an unwired handler in a subpackage, and
deleting a oneof registration each turn the expected tests red.
### Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
