kz930 commented on PR #6436:
URL: https://github.com/apache/texera/pull/6436#issuecomment-4985442120
Really clean V1 — the hardcoded/strategy-discriminator split and the
catalog validation are a nice way to ship the whole pipeline at zero API cost.
Two forward-looking notes, both about what the endpoint
knows before V2 lands:
1. Validation checks existence, but not connectability.
operatorTypeExists guarantees the ghost is a real operator, which is great
for not suggesting broken types. It doesn't yet check that the suggested
operator can actually be wired to the just-added operator's
output. Two levels here:
- Arity is checkable in V1 today: catalog metadata knows a target's
input-port count, so an operator that needs two inputs (e.g. a join) can't be
single-wired as a "next" step. Right now this is avoided by
hand-curating the successor lists (no joins/unions appear as successors),
but nothing enforces it — a future rule edit could reintroduce an
un-connectable suggestion.
- Schema/type compatibility can't be done in V1, since the request only
carries the operator type, not the output schema. Worth flagging as a V2 need
(and it ties into the next point).
2. The reserved context field may be thinner than its stated purpose.
Nice call reserving existingOperatorTypes so V2 can add graph context
without a request-shape change. One thing to reconsider before V2 builds on it:
a flat list of operator types loses the topology (what's
actually upstream of the added operator) and the output-port schema —
which is most of what an LLM would need to "bias with graph context." When V2
lands, capturing the upstream path / operator id + the
output schema (rather than just the set of types present) would likely
serve the ranker better — and it's the same information point 1's schema check
would want, so the two converge.
Neither blocks V1 — just wanted to surface them while the request shape is
still easy to evolve.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
