Very similar with Kaxil's assessment: - Option A: -1: I think it is against the direction of Airflow architectural changes we work for about 1.5 years. While those aren't completed, it would basically invalidate that work. - Option B: -0.1 (for the reasons Kaxil mentioned - Option C: +1 - and I think this should be addressed quickly with high priority. We already discussed that 3.3 should be released **way faster** than 3.2 and I think this one is yet another reason for itV
J. On Thu, Mar 26, 2026 at 4:28 AM Kaxil Naik <[email protected]> wrote: > (details cut in last email) > > Hi Jens, > > My vote: > > - Option A: -0.5 > - Option B: 0 > - Option C: +1 > > Reasoning: > > Option A has architectural issues. The triggerer today is a client > component running user code; it runs user triggers and reports events back. > PR #63489 turns it into a mini-scheduler by having it instantiate executor > instances > (init_executors()), generate JWT tokens, and dispatch workloads directly to > Celery brokers. That's scheduler territory. > > This moves in the opposite direction from AIP-92, which aims to isolate the > triggerer further from core services (removing even direct DB access in > favor of API-only communication). If we're planning to restrict > what the triggerer can talk to, giving it executor ownership and broker > dispatch now creates tech debt we'd need to undo. > > Option B is a better fit architecturally, but it still adds new surface > area (XCom push from triggerer, changes to TriggerEvent, KPO behavior > changes). For a 3.2 release where we should be focused on stability, > introducing new triggerer capabilities increases what we need to test and > validate. I'd rather not risk 3.2 stability for this. > > Option C is my preference. This is a real problem and worth solving > properly, but 3.2 should be a stability-focused release. An AIP would give > us time to design something that aligns with AIP-92's direction rather > than working against it. Target 3.3. > > While I understand the frustration for waiting for several months, but any > new thing added is a new surface to test, validate and potential for > introducing new bugs. > > Thanks, > Kaxil > > On Thu, 26 Mar 2026 at 03:22, Kaxil Naik <[email protected]> wrote: > > > Option C: 0 > > > > On Wed, 25 Mar 2026 at 23:04, Jens Scheffler <[email protected]> > wrote: > > > >> Hi Airflow Devs, > >> > >> Following-up on the discussions in > >> https://lists.apache.org/thread/6znvd5rtqnxt5r4hys7qn64j5mflr9g1 I'd > >> like to cast a VOTE - even if the discussion is fresh.. but my aim/call > >> would be to improve the problem in the core with Airflow 3.2.0 not to > >> have users stuck in this situation for additional multiple months (as it > >> would be a new feature, not a fix most probably not applicable to a fix > >> release) > >> > >> As we discussed two options we can not have a simple +/- Vote, therefore > >> I call for a VOTE with options, please rate from 0 to +1 for the option > >> and only -1 on an option to call for veto. > >> > >> * OPTION A) Integrate feature to directly queue tasks from Triggerer > >> as of PR https://github.com/apache/airflow/pull/63489 in Airflow > >> 3.2.0 > >> * OPTION B) Extend Triggerer to support XCom, change KPO to end from > >> Triggerer as of PR https://github.com/apache/airflow/pull/64068 > >> (Core) with Target Airflow 3.2.0 and > >> https://github.com/apache/airflow/pull/64069 (KPO Adjustment) > >> * OPTION C) Do not immediately change this for Airflow 3.2.0, further > >> discuss alternatives, e.g. raise an AIP > >> > >> For details of the soluitions please check the discussion thread (might > >> need a bit of reading!) and the linked PRs > >> > >> The VOTE is open for the next 72 hours to be done before RC1, until > >> 2026-03-29 23:59 CET - I will sum-up all vores except any potential veto > >> and will report the summary. > >> > >> My Vote is: > >> > >> * Option A: +0.8 > >> * Option B: +1 > >> * Option C: 0 > >> > >> Thanks, > >> > >> Jens > >> > > >
