The GitHub Actions job "Build and push images" on texera.git/main has failed. Run started by GitHub user bobbai00 (triggered by bobbai00).
Head commit for run: 878eb8a38b720dfd1131fa6f04f863a813f2bb53 / Xinyuan Lin <[email protected]> feat(amber): add loop-bookkeeping columns to materialized State (dormant) (#5900) ### What changes were proposed in this PR? Extends the cross-region **State materialization** format from a single `content` column to **3 columns** — `content`, `loop_counter`, `loop_start_id` — promoting loop bookkeeping to first-class columns (never inside the content JSON). The transport carries them end to end: the `OutputManager` state writer + `emit_state`, the Python network sender/receiver, the materialization reader, and the Scala `state.toTuple` call sites. In memory the two loop fields ride on the `StateFrame` envelope; they are materialized/serialized as their own columns (parallel to `content`), and `from_tuple` / `fromTuple` read only `content` back into the `State`. The loop-back write address (LoopStart's input-port URI) is **intentionally not** carried in State. It's constant per-execution config, not per-iteration data, so it will be delivered to Loop End at **setup** in the loop PR rather than round-tripping through every State row. (An earlier revision of this PR carried a `loop_start_state_uri` column; it was dropped after review — better than shipping a dormant column and removing it later.) On the Python side the column-name → value mapping is defined once in `State.to_columns` and reused by both `to_tuple` (iceberg) and the network sender's `StateFrame` branch, so adding a column later is a single-line change in one place rather than an edit in every serializer. **Dormant on `main`** — nothing observable changes without the loop operators: - `to_tuple()` / `toTuple()` and `OutputManager.save_state_to_storage_if_needed` / `emit_state` default the two loop columns to `0` / `""`, so every existing non-loop caller is unchanged. - `from_tuple` / `fromTuple` read only the `content` column, so round-trip identity is preserved and the extra columns are inert. No backward-compatible read of old 1-column State is needed: State materialization is **intra-execution only** — the iceberg state-document URI is execution-scoped (`…/eid/{executionId}/`) and recreated fresh each run, and State tuples are never replayed across executions or engine versions, so a 1-column tuple can never reach the 3-column reader. This is the state-format prerequisite the loop operators build on; the columns carry non-default values only once Loop Start/End set them (follow-up PR). ### Any related issues, documentation, discussions? Extracted from #5700 (loop operators) per @Xiao-zhen-Liu's split request; part of #4442 ("Introduce for loop"). ### How was this PR tested? - **Format / round-trip:** `test_state.py` (loop columns are their own columns, never in content JSON, default to `0` / `""`), Scala `StateSpec` (both loop columns round-trip through a tuple with non-default values, not just `content`), `ArrowUtilsSpec` (3-column Arrow vector round-trip), `IcebergDocumentSpec` (iceberg state-doc round-trip). - **Transport:** `test_network_receiver.py`, `test_input_port_materialization_reader_runnable.py`, and `test_state_materialization_e2e.py` — the e2e (hermetic sqlite catalog) writes non-default values for both loop columns end-to-end and asserts they replay both on the `StateFrame` and on the raw iceberg row, exercising the real Tuple/Schema/iceberg path. - **Dormancy:** `test_output_manager.py::test_defaults_loop_columns_when_omitted` pins that a no-loop caller (no `loop_counter`) still produces a valid 3-column tuple with the loop columns at `0` / `""`. - Local: `workflow-core` + `amber` compile; `StateSpec` + `ArrowUtilsSpec` pass; Python state + transport + e2e tests pass; scalafmt + scalafix + black clean. (`IcebergDocumentSpec` needs the iceberg catalog backend, so it runs in CI.) ### Was this PR authored or co-authored using generative AI tooling? Co-authored with Claude Opus 4.8 in compliance with ASF. Report URL: https://github.com/apache/texera/actions/runs/28562403263 With regards, GitHub Actions via GitBox
