Aditya-XR commented on PR #76085:
URL: https://github.com/apache/airflow/pull/76085#issuecomment-6914040897
Thanks for testing this on macOS and for putting together the comparison
table, @bingqin2!
On the numbering difference in rows 2 and 3 (`2 / 2` and `3 / 2`):
1. **Why `main` gave `eu.ingest__2` / `eu.ingest__3` there:**
On `main`, `_check_for_group_id_collisions` searched `used_group_ids`
with the unprefixed pattern `^ingest__\d+$`. That pattern accidentally matched
top-level `ingest__1` (while ignoring sibling groups like `us.ingest__1`),
which caused the second call inside `eu` to skip `eu.ingest__1` and jump
straight to `eu.ingest__2`.
2. **Consistency with `@task` (`get_unique_task_id`) and isolation between
scopes:**
For decorated tasks, `get_unique_task_id` in
`task-sdk/src/airflow/sdk/bases/decorator.py` already builds the suffix regex
from the prefixed `tg_task_id` (`prefix = re.split(r"__\d+$", tg_task_id)[0]`),
so repeated `@task` calls inside `eu` are always numbered `eu.t`, `eu.t__1`,
`eu.t__2` regardless of how many times `t()` is called at the top level.
With the bare-id lookup + `while self.group_id in self.used_group_ids:`
fallback:
- Calling `ingest()` inside `eu` when top-level `ingest()` was called
twice produces `eu.ingest`, `eu.ingest__2`, `eu.ingest__3` (skipping
`eu.ingest__1`).
- Adding or removing a top-level `ingest()` call outside `eu` shifts all
the suffixed group IDs *inside* `eu` (e.g., `eu.ingest__2` becomes
`eu.ingest__3`), which would rename `eu`'s tasks whenever top-level calls
change.
Scoping the suffix lookup to `prefixed_base` keeps each parent `TaskGroup`'s
numbering self-contained (`eu.ingest`, `eu.ingest__1`, `eu.ingest__2`) and
consistent with `get_unique_task_id`. That said, if maintainers prefer
preserving the top-level offset for existing 2-call Dags via the `while` loop
fallback, I am happy to switch to that approach!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]