hussein-awala opened a new pull request, #64610:
URL: https://github.com/apache/airflow/pull/64610
## Summary
- Add regex-based `partition_key` filtering to asset events across the full
stack:
REST API (`partition_key_pattern` query param), Execution API, Task SDK
client,
supervisor, and `InletEventsAccessor` (`.partition_key(pattern)` chaining
method).
- Add composite database index `(asset_id, partition_key)` on `asset_event`
table
with Alembic migration for query performance.
- Update documentation in `assets.rst` with usage examples for both the Task
SDK
`inlet_events` accessor and the REST API.
## Details
The filtering uses database-native regex (`PostgreSQL ~`, `MySQL REGEXP`),
which
supports matching combined/composite partition keys (e.g. `^us\|2024-`). A
new
`_RegexParam` class and `regex_param_factory` were added to the common
FastAPI
parameters module for reuse.
### Files changed
**Core API layer:**
- `parameters.py` — new `_RegexParam`, `regex_param_factory`,
`QueryAssetEventPartitionKeyRegex`
- `assets.py` — accept `partition_key_pattern` filter in `GET
/api/v2/assets/events`
**Execution API layer:**
- `asset_events.py` — accept `partition_key` regex param in both endpoints
**Task SDK:**
- `client.py` — forward `partition_key` param in `AssetEventOperations.get()`
- `comms.py` — add `partition_key` field to request models
- `supervisor.py` — forward `partition_key` to API client
- `context.py` — add `.partition_key(pattern)` chaining method to
`InletEventsAccessor`
**Database:**
- `asset.py` — add composite index `idx_asset_event_asset_id_partition_key`
- New migration `0111_3_3_0_add_idx_asset_event_partition_key.py`
**Tests:**
- Core API, Execution API, SDK client, and supervisor tests with
postgres-only markers
**Docs:**
- `assets.rst` — usage examples for Task SDK and REST API filtering
## Test plan
- [x] Unit tests for Core REST API partition key regex filtering
(parametrized patterns + combined filters)
- [x] Unit tests for Execution API partition key filtering (both by-asset
and by-alias endpoints)
- [x] Unit tests for Task SDK client parameter forwarding
- [x] Unit tests for supervisor message forwarding with partition_key
- [x] All new regex tests marked `@pytest.mark.backend("postgres")` (regex
not supported on SQLite)
- [x] prek checks pass (except unrelated `uv.lock` regeneration in
`generate-tasksdk-datamodels`)
Made with [Cursor](https://cursor.com)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
