GitHub user carloea2 created a discussion: Loop Start requires exactly one
materialized reader on its input port, is lifting this planned?
## Context
I build Texera `WorkflowContent` programmatically and run it on a current dev
build. Some of the
plans I generate have a Loop Start whose single input port is fed by **two
upstream operators**
(two different producers each contribute values the loop body reads). Those
plans are rejected at
`StartWorkflow`:
```
java.lang.IllegalArgumentException: requirement failed:
Loop Start input port GlobalPortIdentity(
PhysicalOpIdentity(OperatorIdentity(LoopStart-operator-...),main),
PortIdentity(0,false), true)
expected exactly one reader URI, got 2
at
WorkflowExecutionManager.$anonfun$loopStartStateUris$3(WorkflowExecutionManager.scala:86)
at
WorkflowExecutionManager.loopStartStateUris(WorkflowExecutionManager.scala:69)
at
WorkflowExecutionManager.advanceRegionExecutions(WorkflowExecutionManager.scala:131)
at StartWorkflowHandler.startWorkflow(StartWorkflowHandler.scala:45)
```
## Restriction
The check is in `loopStartStateUris`
(`amber/.../scheduling/WorkflowExecutionManager.scala`):
```scala
require(
cfg.storagePairs.size == 1,
s"Loop Start input port $gpid expected exactly one reader URI, got
${cfg.storagePairs.size}"
)
op.id.logicalOpId.id ->
VFSURIFactory.stateURI(cfg.storagePairs.head._1).toString
```
## Questions
1. **Is the single-reader restriction intended to be permanent**, or a
placeholder until loop input
fan-in is supported?
2. **Is per-port channel accounting on the roadmap** completing a port only
when all of its
channels have ended, and firing `produce_state_on_finish` / `on_finish` /
`PortCompletedRequest`
once at that point? That looks like the change that would make loop fan-in
fall out for free, but
it also moves when regions advance, so we assume it is deliberate that it
works the way it does.
GitHub link: https://github.com/apache/texera/discussions/6966
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
