dheerajturaga opened a new pull request, #75156: URL: https://github.com/apache/airflow/pull/75156
### Stack Stacks on #75155. Stack index: 8 of 20. Its base is the branch of #75155, so the diff shows only this layer's commit, "AIP-125: Rate-limit verified webhook deliveries and emit metrics". ### Status Draft. [AIP-125 (Inbound Webhooks)](https://lists.apache.org/thread/p3dpnor4zwko8tldpncj6ftj1bpp18q7) is under discussion on dev@ and not yet voted; this stack is its proof of concept so that voters can run it, and nothing should merge before the vote. The feature is opt-in (only `airflow api-server --apps webhooks` serves deliveries) and experimental; no unauthenticated route is reachable until stack PR 18, which merges last. ### Why Each verified delivery costs a database transaction and, once Dags can subscribe, asset events the scheduler loads on every loop, so the webhooks app has to shed a flood itself. Only verified deliveries count, so unsigned requests cannot use up a sender's limit, though a replayed capture can. Metrics show deliveries, refusals and recording time per webhook, without letting anonymous clients add series. ### What to look at - `[webhooks] rate_limit_per_minute`: a token bucket per webhook and process, checked after verification and before parsing, answered 429 with Retry-After. It defaults to 0 (off); reviewers may prefer a non-zero default. - Metrics: `webhooks.deliveries`, `webhooks.rejections` and `webhooks.delivery_duration`. Unlisted names and unrouted paths share the tag value `other`, now a reserved webhook name. - The router's 404 and 405 count as rejections; 500s are logged, not counted. ### Testing - `test_app.py`, `test_registry.py`, `test_connections.py`, `test_inbound_delivery.py` and `test_pipeline_concurrency.py` passed under breeze on SQLite, Postgres and MySQL. After the last fixes, the tests were rerun on SQLite and Postgres, and the final head passed again on MySQL. ### Known limitations - The `team_name` metric tag waits for team-qualified names in 3.6. --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Claude Code (Opus 5.5) Generated-by: Claude Code (Opus 5.5) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) --- * Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged. * For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed. * When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). * For significant user-facing changes create newsfragment: `{pr_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments). You can add this file in a follow-up commit after the PR is created so you know the PR number. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
