jason810496 opened a new pull request, #71213: URL: https://github.com/apache/airflow/pull/71213
- related: [AIP-108](https://cwiki.apache.org/confluence/x/pY4mGQ), [AIP-85](https://cwiki.apache.org/confluence/x/_Q7OEg) ## Why A Lang-SDK artifact can play two roles that a Dag processor must treat oppositely, and nothing in the serialized Dag tells them apart: | Role | Where the Dag is defined | What the artifact contributes | |---|---|---| | Mixed-language Dag | A Python file whose tasks are `@task.stub` | Only the task implementations | | Native Lang-SDK Dag | The Go/Java source itself | The whole Dag | For a native Dag the processor parses the artifact and the result is persisted. For a mixed-language Dag it must persist nothing: the Python Dag carrying the `@task.stub` tasks already owns that `dag_id`, so persisting the Lang-SDK side too would give conflicts. ## How The new `is_mixed_language_dag` flag at Dag level solves the problem. Only the Lang SDK Dag can set the `is_mixed_language_dag` flag. - The further mixed language Dag processing flow will be like: `JavaDagImporter` -> for all Dags who `is_mixed_language_dag=True` -> validate the corresponding Python Dag structure -> raise import error if unmatch and **don't persist the Java side Dag structure**. - The Task SDK `DAG` gains nothing, so a Python Dag can never set the `is_mixed_language_dag` flag. - Plumbing only. This PR ships **no producer and no behaviour change**: --- ##### Was generative AI tooling used to co-author this PR? - [x] Yes, with help of Claude Code Opus 5 following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) Generated-by: Claude Code (Opus 5) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
