aoelvp94 opened a new pull request, #77250:
URL: https://github.com/apache/airflow/pull/77250
The Java SDK guide already says you can write Dags and tasks in Scala, but
the samples are Java and "Building a bundle" only covers Gradle and Maven. This
adds a "Using the Java SDK from Scala" section to the Java SDK guide, and a
short pointer from `java-sdk/scala_spark_example/README.md`.
The new section covers:
- The interface API (`Task`, `DagDef`, `Bundle.register`), which is what the
in-tree Scala Spark example uses
- Why `@Airflow.Dag` / `@Airflow.Deps` do not compile under `scalac` (the
javac annotation processor does not run)
- That `TaskInput` must be a Java class with a public no-arg constructor and
public non-final fields, with the errors a Scala `var` class and a case class
produce
- Converting Scala collections with `.asJava` before `setXCom` /
`DagDef.config`. A Scala `Map`, `Seq`, `Option` or case class is rejected with
`IllegalArgumentException` rather than being stored as bean properties or as
`{}` (this describes the behaviour from #77249)
- An sbt recipe for a thin bundle (`Main-Class`, copy runtime JARs),
`main_class` when other JARs also set `Main-Class`, and aligning
`jackson-module-scala` to the `jackson-databind` version `airflow-sdk` uses
(related: #66329)
`sbt` is added to `docs/spelling_wordlist.txt` (it is not a default enchant
word).
Docs only. No code or behaviour change. No newsfragment.
Depends on: #77249
related: #66329
## How to test
Docs only; there is no runtime change in this PR. The XCom paragraph
describes the error added in #77249, so this should be merged after (or
together with) that PR.
```bash
# From this branch (it also applies cleanly together with #77249).
uv run --group docs build-docs --one-pass-only apache-airflow
```
Confirm the new section `Using the Java SDK from Scala` in
`authoring-and-scheduling/language-sdks/java.html`
(`id="using-the-java-sdk-from-scala"`),
and that `sbt` does not appear as a spelling error
(`docs/spelling_wordlist.txt`).
The Scala / sbt snippets in that section were compiled against `airflow-sdk`
1.0.0
(Scala 2.13 and Scala 3). The `.asJava`
conversion and the rejected Scala types can be exercised end-to-end with the
reviewer demo from #77249:
Demo (step-by-step README):
https://github.com/aoelvp94/airflow-scala-spark-demo/tree/setxcom-loud-failure-demo/setxcom-demo
---
##### Was generative AI tooling used to co-author this PR?
- [x] Yes (please specify the tool below)
Generated-by: Grok Bot following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
---
* Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)**
for more information. Note: commit author/co-author name and email in commits
become permanently public when merged.
* For fundamental code changes, an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
is needed.
* When adding dependency, check compliance with the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
* For significant user-facing changes create newsfragment:
`{pr_number}.significant.rst`, in
[airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
You can add this file in a follow-up commit after the PR is created so you
know the PR number.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]