dheerajturaga opened a new pull request, #75337: URL: https://github.com/apache/airflow/pull/75337
Part of [AIP-125 (Inbound Webhooks)](https://lists.apache.org/thread/p3dpnor4zwko8tldpncj6ftj1bpp18q7), which lets external services such as GitHub trigger Dags by sending webhooks to Airflow. This is part of Phase 2 (webhook waits and asset routes). It targets Airflow 3.5 if it merges before the 3.5 freeze, and 3.6 otherwise. This adds asset routes. A delivery is often about a data asset rather than a topic, as when an ETL service reports that it has loaded a table, and until now starting the Dags scheduled on that table needed a Dag in between. A route, in `asset_routes` of the webhook connection's extra, names one of the webhook's topic names and an asset. A delivery that offers the topic also records an event on the asset, in the same transaction as its record and with the extra that topic events carry. A route of any other shape leaves the webhook misconfigured, so its deliveries are answered 503. The connection form gains the field in {{PR-23}}. - A route records an event only when a Dag that can run schedules on the asset, as topic events do. The asset is looked up by URI and then by name among active assets, so deliveries never create assets. A route to a webhook topic records nothing, so a topic still receives events only from its own webhook. Under multi-team, neither does a route whose asset any Dag of a team schedules on, paused or not, since webhooks have no team in 3.5. This is the strictest rule, so team support can relax it later; the planned exception for an asset whose `access_control` allows the webhook's team comes with team support. - `partition_key_field` reads the partition key from a dotted path in the payload, with no template engine. A keyed event queues only partition-aware Dags, and an event without a key only the others. The key is kept whatever `store_payload` says. A missing value, or one that is not a non-blank string of at most 250 characters that every database can store, skips the route instead of failing every retry. - The delivery's `actions` gain `asset_routes`, the events that routes recorded, and `skipped_asset_routes`, the routes whose topic the delivery offered but that recorded nothing, each with a reason. The details panel of the deliveries page shows both and explains each reason. A redelivery of a delivery that recorded nothing now rewrites its record even when the outcome is unchanged, as a skip reason can change between copies. - A route needs only edit rights on `webhook_<name>`, while `POST /api/v2/assets/events` needs permission to create asset events. `security_model.rst` lists this as a residual risk, and the webhooks page says to give edit rights on webhook connections only to users who may also record events on the assets a route could name. The webhooks page also lists the extra database privileges that the webhooks process needs for routes with a partition key. - On SQLite, the asset manager can roll back the delivery's transaction when the database is locked while it queues partition-aware Dags. The delivery is then answered 503 with nothing recorded, rather than committing a record whose earlier events were lost. --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Claude Code (Opus 5.5) Generated-by: Claude Code (Opus 5.5) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) --- * Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged. * For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed. * When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). * For significant user-facing changes create newsfragment: `{pr_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments). You can add this file in a follow-up commit after the PR is created so you know the PR number. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
