dheerajturaga opened a new pull request, #75046: URL: https://github.com/apache/airflow/pull/75046
### Stack Stacks on #75045. Stack index: 14 of 20. A PR from a fork cannot target a fork branch, so each PR in the stack is based on `main` and contains every commit below it; only the top commit is new. Review only the commit "AIP-125: Add WebhookTopic to the Task SDK"; the commits below it belong to earlier PRs in the stack. ### Status Draft. [AIP-125 (Inbound Webhooks)](https://lists.apache.org/thread/p3dpnor4zwko8tldpncj6ftj1bpp18q7) is under discussion on dev@ and not yet voted; this stack is its proof of concept so that voters can run it, and nothing should merge before the vote. The feature is opt-in (only `airflow api-server --apps webhooks` serves deliveries) and experimental; no unauthenticated route is reachable until stack PR 18, which merges last. ### Why Dags subscribe to inbound webhook deliveries by scheduling on the topic names the API server builds for each delivery; `WebhookTopic` builds them with the same shared grammar. Topics should only receive events from their webhook, so a topic refuses `extra` and `watchers`, and a task cannot use one as an inlet or outlet. A `PartitionedAssetTimetable` cannot schedule on one, as topic events carry no partition key. ### What to look at - `WebhookTopic(webhook=, path=, event=, action=, detail=)` or `WebhookTopic(name=)`: an `Asset` in the `webhook` group that accepts `access_control`. A plain `Asset(name)` receives the same events. - Refusals: `extra`, `watchers`, another group, a `uri` other than the name, and use as an inlet or outlet, at Dag validation (mapped tasks included) and at run time for `@task` arguments and return values. - Exported from `airflow.sdk` as experimental. It lands before stack PR 15, whose tests use it. ### Testing - The task-sdk tests (333) and core Dag bag and serialization tests passed under breeze on SQLite, Postgres and MySQL. After the last fixes, the SDK tests (338) were rerun under breeze on SQLite, and the core Dag bag and serialization tests that cover topics (`-k 'webhook or topic'`) on SQLite and Postgres, and the final head passed again on MySQL. - In the live demo, a `PartitionedAssetTimetable` over a topic is a Dag import error. ### Known limitations - `test_public_api.py::test_airflow_sdk_exports_exist` fails on upstream `main` on its own (`lineage`), unrelated to this PR. --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Claude Code (Opus 5.5) Generated-by: Claude Code (Opus 5.5) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) --- * Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged. * For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed. * When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). * For significant user-facing changes create newsfragment: `{pr_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments). You can add this file in a follow-up commit after the PR is created so you know the PR number. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
