aoelvp94 opened a new pull request, #77249:
URL: https://github.com/apache/airflow/pull/77249

   `Frame` encodes a `SetXCom` / `SetTaskStateStore` request by running Jackson 
`convertValue` on the whole message object. A Scala `Map`, `Seq`, `Option` or 
case class (none of which are `java.util.Map` / `java.util.Collection`) was 
stored as that object's bean properties, or as `{}` when it had none, and the 
task succeeded. `packAny` never saw the original type.
   
   This inspects the original value in `setXCom` and `taskStateStore.set` 
before it is placed on the message, and throws `IllegalArgumentException` 
naming the type, with a hint to convert Scala collections via 
`scala.jdk.CollectionConverters` (`.asJava`) and a case class to a `Map` or 
list.
   
   The new check is deliberately narrow, so existing Java and Kotlin callers 
keep the previous encoding:
   
   - **Rejected:** Scala types, detected by a `scala.` prefix on the class, any 
superclass or any interface (`scala.collection.*`, `scala.Option` / `Some` / 
`None`, `scala.Product` for case classes, `scala.Tuple*`). No Scala dependency 
is added to the SDK.
   - **Rejected:** any other object that the same Jackson mapper would encode 
as `{}` (no serializable properties). An empty `Map` is still valid JSON and is 
left alone.
   - **Unchanged:** JSON scalars, maps and lists; Java POJOs, records and 
Kotlin data classes with properties; enums; `java.time` / `Date`; and other 
types Jackson already encoded. The original value is passed through, so the 
wire shape is the same as before.
   - Nested values in a `Map`, `Collection` or array are checked the same way. 
Byte arrays keep the existing Base64 error.
   
   The Java SDK guide and `Client` / `TaskStateStore` KDoc now state this 
contract.
   
   A follow-up docs PR will document how to use the Java SDK from Scala (sbt 
bundle, `TaskInput`, XCom conversions) and describe this error.
   
   related: #66329
   
   No newsfragment is included. `java-sdk/DEVELOPMENT.md` asks for an 
`airflow-core/newsfragments/` entry for user-visible changes to `task-sdk/` 
(the Python Task SDK). This is a Java SDK behaviour change. Happy to add 
`{pr_number}.bugfix.rst` (or `.significant.rst`) if reviewers want one after 
the PR number is known.
   
   ## How to test
   
   Unit tests (from a checkout with this patch applied; the Scala docs patch 
may also be applied — they do not conflict):
   
   ```bash
   cd java-sdk
   ./gradlew :sdk:check
   ```
   
   Expected: 426 tests, 0 failures, including `JsonValueTest` and the updated 
`CommTest`, plus ktlint and spotless.
   
   End-to-end against Airflow 3.4.0 with real Scala (a self-contained reviewer 
demo; overlay the fix onto official `airflow-sdk` 1.0.0 because 3.4.0 rejects 
the `1.1.0-SNAPSHOT` supervisor schema `2027-02-28`):
   
   Demo (step-by-step README, expected baseline vs patched output, exact error 
text): 
https://github.com/aoelvp94/airflow-scala-spark-demo/tree/setxcom-loud-failure-demo/setxcom-demo
   
   ```bash
   git clone -b setxcom-loud-failure-demo 
https://github.com/aoelvp94/airflow-scala-spark-demo.git && cd 
airflow-scala-spark-demo/setxcom-demo
   docker build -t airflow-scala-docs-e2e:3.4.0 -f Dockerfile .
   export AIRFLOW_SRC=/path/to/this/airflow/checkout
   make bundle
   make overlay
   make baseline   # DAG succeeds; Scala types silently stored as bean 
properties / {}
   make patched    # DAG fails; Scala tasks raise IllegalArgumentException and 
write no XCom; Java controls unchanged
   make evidence
   ```
   
   See the demo README for the exact error text and the per-task expected 
values.
   
   ---
   
   ##### Was generative AI tooling used to co-author this PR?
   
   - [x] Yes (please specify the tool below)
   
   Generated-by: Grok Bot following [the 
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
   
   ---
   
   * Read the **[Pull Request 
Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)**
 for more information. Note: commit author/co-author name and email in commits 
become permanently public when merged.
   * For fundamental code changes, an Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
 is needed.
   * When adding dependency, check compliance with the [ASF 3rd Party License 
Policy](https://www.apache.org/legal/resolved.html#category-x).
   * For significant user-facing changes create newsfragment: 
`{pr_number}.significant.rst`, in 
[airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
 You can add this file in a follow-up commit after the PR is created so you 
know the PR number.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to