aglinxinyuan commented on code in PR #5900:
URL: https://github.com/apache/texera/pull/5900#discussion_r3488229451
##########
amber/src/main/python/core/models/state.py:
##########
@@ -25,13 +25,41 @@
class State(dict):
CONTENT = "content"
- SCHEMA = Schema(raw_schema={CONTENT: "STRING"})
+ # Loop-control bookkeeping owned by the worker runtime, NOT user state --
it
+ # never appears in the content JSON. In memory it rides on the StateFrame
+ # envelope; it is materialized/serialized as its own column (parallel to
+ # content) by to_tuple(...). from_tuple() returns the bare State; callers
+ # that need these values read the corresponding columns off the tuple.
+ LOOP_COUNTER = "loop_counter"
+ LOOP_START_ID = "loop_start_id"
+ LOOP_START_STATE_URI = "loop_start_state_uri"
Review Comment:
Good question — and you're right it differs from the offline sketch. As
currently wired in the loop branch, the URI isn't derivable from
`loop_start_id` alone:
- `loop_start_id` is just LoopStart's *logical* op id — it's what
`JumpToOperatorRegionRequest(OperatorIdentity(loop_start_id))` jumps to.
- `loop_start_state_uri` is `VFSURIFactory.state_uri(readers[0].uri)` — the
*physical* iceberg URI of LoopStart's **input port**, which also encodes the
execution id, the physical layer name, and which input port/reader. LoopEnd
writes the next iteration's state straight into that URI, so it needs the full
path, not just the logical id.
Basically, the LOOP_START_STATE_URI is the storage URI for the operator
right before the LoopStart.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
