I'd wait for others to comment. I want to see what others think. On Wed, Apr 15, 2026 at 4:51 PM Foldvari, Gyorgy via dev < [email protected]> wrote:
> If xcom_pull(task_ids=XCom.ANY, key=”sample”) would return with a list, > that would not be the same result what was produced by Airflow 2. Airflow 2 > returns with the most recent value and not with a list. > I am wondering what would make more sense: > > 1. Return with one value, the most recent one (on None) instead of a > list, copying Airflow 2 behavior. > 2. Return with a list but also defining a new optional parameter to > override this behavior (XCom.FIRST_OLDEST, XCom.FIRST_NEWEST). > I am struggling with coming up a valid use case where getting back a list > of values without any further information would make sense. So probably we > should go with the simpler first option. > > > Information Classification: GENERAL > > On 2026/04/15 14:35:35 Jarek Potiuk wrote: > > > What do you think about this solution? > > > ti.xcom_pull accepting ‘*’ or rather a sentinel value as task_ids to > > explicitly say that ANY task_id should be considered. The result would be > > all the values pushed to xcom by any task in that dag run as a list, > sorted > > by XCOM timestamp. > > > ti.xcom_pull(task_ids= ANY, key=”example”) > > > > > > I'd rather (if others see a value for it) have some explicit sentinel > > (XCom.FiRST_OLDEST, XCom.FIRST_NEWEST - or whatever makes sense for > those). > > > > > The only challenge is how to use this in classic Operators’ templated > > fields. > > > > Those sentinels could be easily exposed to JINJA and predefined there. > > > > It might of course be over-the-top, and some might say "ugly" - but if it > > makes it easier for people to migrate to 3 and would not make it more > > difficult for maintenance, I am all for it. > > > > J. > > > > > > On Wed, Apr 15, 2026 at 4:22 PM Foldvari, Gyorgy via dev < > > [email protected]> wrote: > > > > > What do you think about this solution? > > > ti.xcom_pull accepting ‘*’ or rather a sentinel value as task_ids to > > > explicitly say that ANY task_id should be considered. The result would > be > > > all the values pushed to xcom by any task in that dag run as a list, > sorted > > > by XCOM timestamp. > > > ti.xcom_pull(task_ids= ANY, key=”example”) > > > The only challenge is how to use this in classic Operators’ templated > > > fields. > > > > > > > > > Information Classification: GENERAL > > > > > > On 2026/04/15 13:34:58 Jarek Potiuk wrote: > > > > > This was ambiguous regarding the value you wanted to retrieve and > the > > > > intended change. > > > > > > > > While I see why we cannot (should not) bring back the "exact" > behaviour, > > > I > > > > think we could - potentially at least - make it non-ambiguous and > > > > predictable by defining the semantics and ordering and selecting the > > > first > > > > one. And - I think - we do not **have** to bring it back in this > > > particular > > > > method. Maybe we can just a new method call with defined ordering and > > > > semantics - behaving predictably, similar to Airflow 2—and clearly > > > defined > > > > semantically. That would at least give people an easier way to > migrate? > > > > > > > > While I think the cat if out-of-the bag and we cannot truly revert > the > > > > change (because that would again potentially affect 3.0 - 3.2 > users) - > > > but > > > > we could at least make it easier for people to cope with it without > too > > > > much hassle while waiting for the task state to be available in this > > > > particular case? > > > > > > > > Just a thought I had - ... listen to your users and do things easier > for > > > > them - without breaking our SemVer promises. > > > > > > > > J. > > > > > > > > > > > > > > > > On Wed, Apr 15, 2026 at 3:20 PM Foldvari, Gyorgy via dev < > > > > [email protected]> wrote: > > > > > > > > > I do not want to use XCom for managing task state. It was just a > very > > > > > simple - and seemingly misleading - example to explain the original > > > > > behavior. > > > > > > > > > > I gave details about valid use cases and issues caused by this > change > > > in > > > > > the behavior, in my original post. Those are not addressed by the > AIP > > > you > > > > > are referring to. > > > > > > > > > > But that AIP would definitely address another valid use case what > I am > > > > > missing especially for implementing stateful sensors, so I really > hope > > > it > > > > > goes through and gets implemented. > > > > > > > > > > > > > > > Information Classification: GENERAL > > > > > -----Original Message----- > > > > > From: Ash Berlin-Taylor <[email protected]> > > > > > Sent: Wednesday, April 15, 2026 14:49 > > > > > To: [email protected] > > > > > Subject: Re: [DISCUSS] Reconsidering `xcom_pull(task_ids=None)` > > > behavior > > > > > change in Airflow 3 > > > > > > > > > > [You don't often get email from [email protected]. Learn why this is > > > > > important at https://aka.ms/LearnAboutSenderIdentification ] > > > > > > > > > > CAUTION: This email originated from outside of the organization. > Do not > > > > > click links or open attachments unless you recognize the sender and > > > know > > > > > the content is safe. > > > > > > > > > > > > > > > This was ambiguous as to what value you wanted to get, and an > intended > > > > > change. > > > > > > > > > > If you want this sort of behaviour, then you probably want to look > at > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-103%3A+Task+State+Management > > > > > which provides a dedicated way to manage state without many of the > > > quirks > > > > > of XCom interface as it stands today. Reading between the lines, I > > > think > > > > > this API describes what you want? > > > > > > > > > > -ash > > > > > > > > > > > On 15 Apr 2026, at 12:45, Foldvari, Gyorgy via dev < > > > > > [email protected]> wrote: > > > > > > > > > > > > I see where the confusion is coming from, it is my mistake. Sorry > > > about > > > > > that. > > > > > > > > > > > > To clarify, I am taking about the use case where the key > parameter is > > > > > passed but the task_ids parameter is not or it is None. > > > > > > > > > > > > -----Original Message----- > > > > > > From: Foldvari, Gyorgy via dev <[email protected]> > > > > > > Sent: Wednesday, April 15, 2026 13:42 > > > > > > To: [email protected] > > > > > > Cc: Foldvari, Gyorgy <[email protected]> > > > > > > Subject: RE: Re: [DISCUSS] Reconsidering > `xcom_pull(task_ids=None)` > > > > > > behavior change in Airflow 3 > > > > > > > > > > > > CAUTION: This email originated from outside of the organization. > Do > > > not > > > > > click links or open attachments unless you recognize the sender and > > > know > > > > > the content is safe. > > > > > > > > > > > > > > > > > > The original behavior is to return the most recent value put by > any > > > > > upstream task of the same run. Not all the values, only the recent > one. > > > > > > Supposing that there are multiple tasks pushing values to XCom in > > > this > > > > > order: > > > > > > Task1: ti.xcom_push(key="example", value=1) > > > > > > Task2: ti.xcom_push(key="example", value=2) > > > > > > Task3: ti.xcom_push(key="example", value=3) Then in a downstream > task > > > > > ti.comm_pull(key="example") returns 3. > > > > > > I do not propose to change this behavior. > > > > > > > > > > > > On 2026/04/14 16:09:05 Daniel Standish via dev wrote: > > > > > >> So the behavior before would be that it would return all xcom > values > > > > > >> that were emitted from the present run? > > > > > >> > > > > > > > > > > > > > > > > > > ________________________________ > > > > > > > > > > > > Information regarding MSCI's processing of personal data may be > found > > > > > > at http://www.msci.com/privacy-pledge. This email message and > any > > > > > attachments > > > > > > are for the sole use of the intended recipients and may contain > > > > > > proprietary and/or confidential information which may be > privileged > > > or > > > > > > otherwise protected from disclosure. Any unauthorized review, > use, > > > > > > disclosure or distribution is prohibited. All rights and > remedies are > > > > > > reserved. If you are not an intended recipient, please contact > the > > > > > > sender by reply email and destroy the original message and any > copies > > > > > > of the message as well as any attachments to the original > message. > > > > > > Local registered entity information: > > > > > > https://www.msci.com/local-registered-entities > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > To unsubscribe, e-mail: [email protected] > > > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > > > > > > > > ________________________________ > > > > > > > > > > Information regarding MSCI’s processing of personal data may be > found > > > at > > > > > http://www.msci.com/privacy-pledge. This email message and any > > > > > attachments are for the sole use of the intended recipients and may > > > contain > > > > > proprietary and/or confidential information which may be > privileged or > > > > > otherwise protected from disclosure. Any unauthorized review, use, > > > > > disclosure or distribution is prohibited. All rights and remedies > are > > > > > reserved. If you are not an intended recipient, please contact the > > > sender > > > > > by reply email and destroy the original message and any copies of > the > > > > > message as well as any attachments to the original message. Local > > > > > registered entity information: > > > > > https://www.msci.com/local-registered-entities > > > > > > > > > > > > > > > ________________________________ > > > > > > Information regarding MSCI’s processing of personal data may be found > at > > > www.msci.com/privacy-pledge. This email message and any attachments > are > > > for the sole use of the intended recipients and may contain proprietary > > > and/or confidential information which may be privileged or otherwise > > > protected from disclosure. Any unauthorized review, use, disclosure or > > > distribution is prohibited. All rights and remedies are reserved. If > you > > > are not an intended recipient, please contact the sender by reply > email and > > > destroy the original message and any copies of the message as well as > any > > > attachments to the original message. Local registered entity > information: > > > https://www.msci.com/local-registered-entities > > > > > > > ________________________________ > > Information regarding MSCI’s processing of personal data may be found at > www.msci.com/privacy-pledge. This email message and any attachments are > for the sole use of the intended recipients and may contain proprietary > and/or confidential information which may be privileged or otherwise > protected from disclosure. Any unauthorized review, use, disclosure or > distribution is prohibited. All rights and remedies are reserved. If you > are not an intended recipient, please contact the sender by reply email and > destroy the original message and any copies of the message as well as any > attachments to the original message. Local registered entity information: > https://www.msci.com/local-registered-entities >
