So the behavior before would be that it would return all xcom values that were emitted from the present run?
On Tue, Apr 14, 2026 at 12:28 AM Foldvari, Gyorgy via dev < [email protected]> wrote: > Dear Airflow community, > > I'd like to raise a discussion about the behavior change of > `xcom_pull(task_ids=None)` between Airflow 2.x and 3.x, and whether it > should be treated as an unintended behavior change (a bug to fix). > (Related to https://github.com/apache/airflow/issues/51821) > > Background > > In Airflow 2.x (and in Airflow 1.x as well), the documentation explicitly > states that passing `task_ids=None` removes the filter. In practice, this > resulted in returning the most recent XCom for a given key within the DAG. > In Airflow 3, this changed so that `task_ids=None` defaults to the current > task's `task_id`. > > Why this might be considered an unintended change > > > * The previous behavior was documented, and users have relied on it. > * The new default does not appear to provide meaningful value in most > real scenarios, since tasks typically do not consume their own XCom outputs > within the same run. > * Effectively, a broadly useful default was replaced with behavior > that is often a no-op. > > Practical impact > > The change introduces several usability regressions: > > * Tighter coupling: downstream tasks must explicitly reference > upstream `task_id`s, increasing fragility during refactoring. > * Reduced modularity: reusable DAG components now require knowledge of > upstream internals. > * Lost pattern: it removes a simple and widely used > approach-retrieving the latest value for a key regardless of which task > produced it. > > On the "ambiguity" concern > > A reasonable and consistent interpretation of the previous behavior is: > > return the most recent XCom for the given key within the DAG scope > While there are edge cases (e.g., multiple producers), these are already > part of DAG design considerations and not unique to this behavior. > > Workarounds in Airflow 3 > > It is possible to replicate the old behavior via the public API, but with > notable drawbacks: > > * Additional overhead (e.g., REST calls) > * No support for Jinja templating, which limits usage in classic > operators > > Trade-offs > > Restoring the previous behavior would technically introduce a breaking > change for users already on Airflow 3.0-3.2 who may have adapted to the new > semantics. > However, the practical impact of this is likely limited: > > * The current default behavior (falling back to the current task's > `task_id`) has very low usefulness in real scenarios, so the likelihood > that users intentionally rely on it is low. > * If someone does rely on it, the behavior is trivial to preserve by > explicitly passing the current task's `task_id`. > * A behavior change would only affect cases where: > * an upstream task pushed an XCom with the same key, and > * the current task did not produce a value for that key > > Given this, the risk appears constrained and manageable, while restoring > the previous behavior would re-enable a documented and widely useful > pattern. > > So the question becomes whether: > > * the benefits of restoring a documented and broadly useful behavior > outweighs > > * the relatively low and mitigatable risk for current Airflow 3 users > > Question to the community > > Do we consider this change: > > * an intended semantic correction, or > * an unintended behavior change that should be treated as a bug fix? > > If there is agreement on the latter, I'm happy to contribute to the fix. > > Thanks, > Gyorgy Foldvari > > > ________________________________ > > Information regarding MSCI's processing of personal data may be found at > www.msci.com/privacy-pledge. This email message and any attachments are > for the sole use of the intended recipients and may contain proprietary > and/or confidential information which may be privileged or otherwise > protected from disclosure. Any unauthorized review, use, disclosure or > distribution is prohibited. All rights and remedies are reserved. If you > are not an intended recipient, please contact the sender by reply email and > destroy the original message and any copies of the message as well as any > attachments to the original message. Local registered entity information: > https://www.msci.com/local-registered-entities > > > Information Classification: GENERAL >
