This is an automated email from the ASF dual-hosted git repository.
github-merge-queue[bot] pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/texera.git
The following commit(s) were added to refs/heads/main by this push:
new 25e40ecad6 fix(workflow-operator): validate base64 in HF binary-column
fallback … (#7243)
25e40ecad6 is described below
commit 25e40ecad607b09de44cedfc78b68df31a34ea3b
Author: Prateek Ganigi <[email protected]>
AuthorDate: Tue Aug 4 17:09:24 2026 -0700
fix(workflow-operator): validate base64 in HF binary-column fallback …
(#7243)
### What changes were proposed in this PR?
In the HuggingFace operator's `_read_binary_value`, the final fallback
decoded a column value with `base64.b64decode(val)`. Without
`validate=True`, `b64decode` silently ignores characters outside the
base64 alphabet, so ordinary text "successfully" decodes into garbage
bytes and the intended `val.encode("utf-8")` fallback rarely fires and a
plain-text cell could be turned into garbage and sent as image/audio
bytes.
Adding `validate=True` makes `b64decode` reject non-base64 input
(spaces, punctuation, newlines), so real text now correctly falls
through to UTF-8. Only this fallback call changes; the `data:`-URL
decodes are left as-is (genuine base64 that can legitimately contain
whitespace).
### Any related issues, documentation, discussions?
Closes #7200.
### How was this PR tested?
`sbt "WorkflowOperator/testOnly
org.apache.texera.amber.operator.huggingFace.*
org.apache.texera.amber.util.PythonCodeRawInvalidTextSpec"`: passes (125
tests). Added a test asserting the fallback uses `validate=True`;
`PythonCodeRawInvalidTextSpec` py-compiles the generated Python.
scalafmt clean.
### Was this PR authored or co-authored using generative AI tooling?
This PR was co-authored with Claude Opus 4.8 in compliance with ASF
policy.
---
.../operator/huggingFace/codegen/HuggingFaceCodegenBase.scala | 2 +-
.../operator/huggingFace/HuggingFaceInferenceOpDescSpec.scala | 7 +++++++
2 files changed, 8 insertions(+), 1 deletion(-)
diff --git
a/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/HuggingFaceCodegenBase.scala
b/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/HuggingFaceCodegenBase.scala
index a4dfc7ca3f..7db809a07a 100644
---
a/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/HuggingFaceCodegenBase.scala
+++
b/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/HuggingFaceCodegenBase.scala
@@ -888,7 +888,7 @@ object HuggingFaceCodegenBase {
| # base64-encoded bytes. Anything else is treated as raw bytes,
never
| # as a path to open.
| try:
- | return base64.b64decode(val)
+ | return base64.b64decode(val, validate=True)
| except Exception:
| return val.encode("utf-8")
|
diff --git
a/common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/huggingFace/HuggingFaceInferenceOpDescSpec.scala
b/common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/huggingFace/HuggingFaceInferenceOpDescSpec.scala
index 85e38c08e2..d34a6e8fea 100644
---
a/common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/huggingFace/HuggingFaceInferenceOpDescSpec.scala
+++
b/common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/huggingFace/HuggingFaceInferenceOpDescSpec.scala
@@ -649,6 +649,13 @@ class HuggingFaceInferenceOpDescSpec extends AnyFlatSpec
with Matchers {
outSchema.getAttributeNames.contains("hf_response") shouldBe true
}
+ it should "validate base64 in the binary-column fallback so plain text isn't
decoded to garbage" in {
+ val code = makeDesc().generatePythonCode()
+ // validate=True makes b64decode reject non-base64 input, so real text
falls
+ // through to utf-8 instead of decoding to garbage bytes.
+ code should include("base64.b64decode(val, validate=True)")
+ }
+
it should "treat a 401 as retryable so one provider's auth failure doesn't
abort the fallback" in {
val code = makeDesc().generatePythonCode()
// 401 is in the retryable set -> the loop tries the next provider instead
of bailing.
