sortega opened a new pull request, #69964:
URL: https://github.com/apache/airflow/pull/69964
### Why
`DagBundlesManager.sync_bundles_to_db()` marks every bundle stored in the DB
that is not present in the calling process's config as inactive ("no longer
found in config and has been disabled"). This implicitly assumes the calling
process sees the *complete* bundle configuration.
When multiple `dag-processor` processes are each configured with a *partial*
bundle config — one bundle per processor, a natural way to shard DAG parsing —
each processor treats every other processor's bundle as removed and deactivates
it. Processor A disables B's bundle, B then disables A's, and they flip
`dag_bundle.active` on every parse cycle. The deployment never converges and
continuously logs the disable warning for bundles that are actively configured
on another processor.
Minimal reproduction of the flip-flop (two managers with disjoint config, no
error path involved):
```
AFTER B syncs: [('bundle-a', False), ('bundle-b', True)] # B
disabled A
AFTER A syncs again: [('bundle-a', True), ('bundle-b', False)] # A
disabled B
```
### How
- Add a `deactivate_missing` parameter to `sync_bundles_to_db()`, defaulting
to `True` so existing single-processor / full-config behavior (including
cleanup of genuinely-removed bundles) is unchanged.
- When `deactivate_missing=False`, the method upserts the bundles it knows
about and returns without touching the rest.
- `DagFileProcessorManager.sync_bundles()` now passes
`deactivate_missing=not self.bundle_names_to_parse`: a processor started with
`--bundle-name` (a partial owner) will not deactivate bundles it does not own,
while an unfiltered processor keeps deactivating removed bundles as before.
### Tests
- `test_sync_bundles_to_db_partial_config_does_not_disable_other_bundles` —
two processors with disjoint configs; asserts both bundles stay active across
sync cycles (fails without the fix).
- `test_sync_bundles_deactivates_missing_when_owning_all_bundles` /
`test_sync_bundles_does_not_deactivate_missing_when_filtered` — assert
`sync_bundles()` forwards the right `deactivate_missing` value based on
`bundle_names_to_parse`.
- Existing `test_sync_bundles_to_db` (full-config deactivation) still passes.
closes: #69963
related: #69698
<!-- SPDX-License-Identifier: Apache-2.0
https://www.apache.org/licenses/LICENSE-2.0 -->
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes - Claude Code
<!--
Generated-by: Claude Code following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
