If xcom_pull(task_ids=XCom.ANY, key=”sample”) would return with a list, that would not be the same result what was produced by Airflow 2. Airflow 2 returns with the most recent value and not with a list. I am wondering what would make more sense:
1. Return with one value, the most recent one (on None) instead of a list, copying Airflow 2 behavior. 2. Return with a list but also defining a new optional parameter to override this behavior (XCom.FIRST_OLDEST, XCom.FIRST_NEWEST). I am struggling with coming up a valid use case where getting back a list of values without any further information would make sense. So probably we should go with the simpler first option. Information Classification: GENERAL On 2026/04/15 14:35:35 Jarek Potiuk wrote: > > What do you think about this solution? > > ti.xcom_pull accepting ‘*’ or rather a sentinel value as task_ids to > explicitly say that ANY task_id should be considered. The result would be > all the values pushed to xcom by any task in that dag run as a list, sorted > by XCOM timestamp. > > ti.xcom_pull(task_ids= ANY, key=”example”) > > > I'd rather (if others see a value for it) have some explicit sentinel > (XCom.FiRST_OLDEST, XCom.FIRST_NEWEST - or whatever makes sense for those). > > > The only challenge is how to use this in classic Operators’ templated > fields. > > Those sentinels could be easily exposed to JINJA and predefined there. > > It might of course be over-the-top, and some might say "ugly" - but if it > makes it easier for people to migrate to 3 and would not make it more > difficult for maintenance, I am all for it. > > J. > > > On Wed, Apr 15, 2026 at 4:22 PM Foldvari, Gyorgy via dev < > [email protected]> wrote: > > > What do you think about this solution? > > ti.xcom_pull accepting ‘*’ or rather a sentinel value as task_ids to > > explicitly say that ANY task_id should be considered. The result would be > > all the values pushed to xcom by any task in that dag run as a list, sorted > > by XCOM timestamp. > > ti.xcom_pull(task_ids= ANY, key=”example”) > > The only challenge is how to use this in classic Operators’ templated > > fields. > > > > > > Information Classification: GENERAL > > > > On 2026/04/15 13:34:58 Jarek Potiuk wrote: > > > > This was ambiguous regarding the value you wanted to retrieve and the > > > intended change. > > > > > > While I see why we cannot (should not) bring back the "exact" behaviour, > > I > > > think we could - potentially at least - make it non-ambiguous and > > > predictable by defining the semantics and ordering and selecting the > > first > > > one. And - I think - we do not **have** to bring it back in this > > particular > > > method. Maybe we can just a new method call with defined ordering and > > > semantics - behaving predictably, similar to Airflow 2—and clearly > > defined > > > semantically. That would at least give people an easier way to migrate? > > > > > > While I think the cat if out-of-the bag and we cannot truly revert the > > > change (because that would again potentially affect 3.0 - 3.2 users) - > > but > > > we could at least make it easier for people to cope with it without too > > > much hassle while waiting for the task state to be available in this > > > particular case? > > > > > > Just a thought I had - ... listen to your users and do things easier for > > > them - without breaking our SemVer promises. > > > > > > J. > > > > > > > > > > > > On Wed, Apr 15, 2026 at 3:20 PM Foldvari, Gyorgy via dev < > > > [email protected]> wrote: > > > > > > > I do not want to use XCom for managing task state. It was just a very > > > > simple - and seemingly misleading - example to explain the original > > > > behavior. > > > > > > > > I gave details about valid use cases and issues caused by this change > > in > > > > the behavior, in my original post. Those are not addressed by the AIP > > you > > > > are referring to. > > > > > > > > But that AIP would definitely address another valid use case what I am > > > > missing especially for implementing stateful sensors, so I really hope > > it > > > > goes through and gets implemented. > > > > > > > > > > > > Information Classification: GENERAL > > > > -----Original Message----- > > > > From: Ash Berlin-Taylor <[email protected]> > > > > Sent: Wednesday, April 15, 2026 14:49 > > > > To: [email protected] > > > > Subject: Re: [DISCUSS] Reconsidering `xcom_pull(task_ids=None)` > > behavior > > > > change in Airflow 3 > > > > > > > > [You don't often get email from [email protected]. Learn why this is > > > > important at https://aka.ms/LearnAboutSenderIdentification ] > > > > > > > > CAUTION: This email originated from outside of the organization. Do not > > > > click links or open attachments unless you recognize the sender and > > know > > > > the content is safe. > > > > > > > > > > > > This was ambiguous as to what value you wanted to get, and an intended > > > > change. > > > > > > > > If you want this sort of behaviour, then you probably want to look at > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-103%3A+Task+State+Management > > > > which provides a dedicated way to manage state without many of the > > quirks > > > > of XCom interface as it stands today. Reading between the lines, I > > think > > > > this API describes what you want? > > > > > > > > -ash > > > > > > > > > On 15 Apr 2026, at 12:45, Foldvari, Gyorgy via dev < > > > > [email protected]> wrote: > > > > > > > > > > I see where the confusion is coming from, it is my mistake. Sorry > > about > > > > that. > > > > > > > > > > To clarify, I am taking about the use case where the key parameter is > > > > passed but the task_ids parameter is not or it is None. > > > > > > > > > > -----Original Message----- > > > > > From: Foldvari, Gyorgy via dev <[email protected]> > > > > > Sent: Wednesday, April 15, 2026 13:42 > > > > > To: [email protected] > > > > > Cc: Foldvari, Gyorgy <[email protected]> > > > > > Subject: RE: Re: [DISCUSS] Reconsidering `xcom_pull(task_ids=None)` > > > > > behavior change in Airflow 3 > > > > > > > > > > CAUTION: This email originated from outside of the organization. Do > > not > > > > click links or open attachments unless you recognize the sender and > > know > > > > the content is safe. > > > > > > > > > > > > > > > The original behavior is to return the most recent value put by any > > > > upstream task of the same run. Not all the values, only the recent one. > > > > > Supposing that there are multiple tasks pushing values to XCom in > > this > > > > order: > > > > > Task1: ti.xcom_push(key="example", value=1) > > > > > Task2: ti.xcom_push(key="example", value=2) > > > > > Task3: ti.xcom_push(key="example", value=3) Then in a downstream task > > > > ti.comm_pull(key="example") returns 3. > > > > > I do not propose to change this behavior. > > > > > > > > > > On 2026/04/14 16:09:05 Daniel Standish via dev wrote: > > > > >> So the behavior before would be that it would return all xcom values > > > > >> that were emitted from the present run? > > > > >> > > > > > > > > > > > > > > > ________________________________ > > > > > > > > > > Information regarding MSCI's processing of personal data may be found > > > > > at http://www.msci.com/privacy-pledge. This email message and any > > > > attachments > > > > > are for the sole use of the intended recipients and may contain > > > > > proprietary and/or confidential information which may be privileged > > or > > > > > otherwise protected from disclosure. Any unauthorized review, use, > > > > > disclosure or distribution is prohibited. All rights and remedies are > > > > > reserved. If you are not an intended recipient, please contact the > > > > > sender by reply email and destroy the original message and any copies > > > > > of the message as well as any attachments to the original message. > > > > > Local registered entity information: > > > > > https://www.msci.com/local-registered-entities > > > > > > > > > > --------------------------------------------------------------------- > > > > > To unsubscribe, e-mail: [email protected] > > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > > > > ________________________________ > > > > > > > > Information regarding MSCI’s processing of personal data may be found > > at > > > > http://www.msci.com/privacy-pledge. This email message and any > > > > attachments are for the sole use of the intended recipients and may > > contain > > > > proprietary and/or confidential information which may be privileged or > > > > otherwise protected from disclosure. Any unauthorized review, use, > > > > disclosure or distribution is prohibited. All rights and remedies are > > > > reserved. If you are not an intended recipient, please contact the > > sender > > > > by reply email and destroy the original message and any copies of the > > > > message as well as any attachments to the original message. Local > > > > registered entity information: > > > > https://www.msci.com/local-registered-entities > > > > > > > > > > > ________________________________ > > > > Information regarding MSCI’s processing of personal data may be found at > > www.msci.com/privacy-pledge. This email message and any attachments are > > for the sole use of the intended recipients and may contain proprietary > > and/or confidential information which may be privileged or otherwise > > protected from disclosure. Any unauthorized review, use, disclosure or > > distribution is prohibited. All rights and remedies are reserved. If you > > are not an intended recipient, please contact the sender by reply email and > > destroy the original message and any copies of the message as well as any > > attachments to the original message. Local registered entity information: > > https://www.msci.com/local-registered-entities > > > ________________________________ Information regarding MSCI’s processing of personal data may be found at www.msci.com/privacy-pledge. This email message and any attachments are for the sole use of the intended recipients and may contain proprietary and/or confidential information which may be privileged or otherwise protected from disclosure. Any unauthorized review, use, disclosure or distribution is prohibited. All rights and remedies are reserved. If you are not an intended recipient, please contact the sender by reply email and destroy the original message and any copies of the message as well as any attachments to the original message. Local registered entity information: https://www.msci.com/local-registered-entities
