PG1204 opened a new pull request, #6972:
URL: https://github.com/apache/texera/pull/6972
### What changes were proposed in this PR?
Fixes two related bugs in the HuggingFace inference operator's
provider-fallback loop (`_post_with_fallback` in the generated Python, emitted
by `PythonCodegenBase.scala`) that break the `text-generation` /
`image-text-to-text` path, which is the operator's default task.
1. **Wrong model id on provider-scoped chat routes.** The chat branch posted
the same payload, carrying `"model": self.MODEL_ID` (the HF Hub ID), to every
provider's route (`router.huggingface.co/{provider}/...`). Provider-scoped
routes require the provider's *own* model name (`providerId`, e.g. the Hub's
`Qwen/Qwen2.5-72B-Instruct` is a different string on Groq or Fireworks), which
`_resolve_providers` already fetches and `_call_provider` already uses for the
other task families. Providers whose internal name differs from the Hub ID
rejected every request with 400/404, so fallback only succeeded when the names
happened to match. The chat branch now posts a per-attempt copy with the model
overridden:
chat_payload = {**pipeline_payload, "model": provider_id}
(a copy, not in-place mutation, pipeline_payload is reused for the next
provider
attempt).
2. **Malformed hf-inference chat URL.** The chat URL for hf-inference was
built as `hf-inference/v1/chat/completions`, but hf-inference expects the model
in the URL path: `hf-inference/models/{model-id}/v1/chat/completions` (the form
this file already uses for pipeline tasks). Since hf-inference sorts first in
`PROVIDER_COST_PRIORITY`, the cheapest provider failed on every row, wasting a
doomed request (up to the 120 s timeout) per row before falling through.
Net effect: text-gen rows now succeed on the cheapest live provider instead
of failing with "All inference providers failed" or silently drifting to
pricier providers. No behavior change for other task families -
`_call_provider` already handled them correctly.
### Any related issues, documentation, discussions?
Closes #6965
### How was this PR tested?
Added a generated-code test to HuggingFaceInferenceOpDescSpec ("send the
provider-specific model id on provider-scoped chat routes") pinning both fixes:
the corrected hf-inference chat URL, and the chat branch posting the
per-provider payload copy (asserted as an anchored two-line block so pipeline
routes that legitimately post pipeline_payload directly stay unaffected).
Run:
sbt "WorkflowOperator/testOnly
org.apache.texera.amber.operator.huggingFace.*"
Full HF package passes (123 tests, 12 suites), including
PythonCodeRawInvalidTextSpec, which py_compiles the generated Python of all 117
Python operators, verifying the edited template still emits syntactically valid
Python. scalafmt clean.
### Was this PR authored or co-authored using generative AI tooling?
Co-authored with Claude Fable 5 in compliance with ASF.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
