t; > > On Thu, 1 Aug 2024 at 01:14, Kaxil Naik <
> > > kaxiln...@gmail.com>
> > > > > > > wrote:
> > > > > > > >> > >
> > > > > > > >> > > > Thanks Vincent, I will take a look again t
gt;> > > >> I updated the AIP-82 given the different comments and
> > > > concerns
> > > > > I
> > > > > > >> > > >> received. I also tried to reply to all comments
> > > > indi
updated this
> > section
> > > > > >> entirely
> > > > > >> > > >> because I received many concerns about the previous
> > proposal.
> > > > The
> > > > > >> > > overall
> > > > >
r
> > > > >> experience
> > > > >> > > >> to use a message queue scenario. I understood that this is
> > > > >> probably the
> > > > >> > > >> main use case we are trying to cover with this AIP, thus I
&g
dated the poll based event-driven scheduling DAG
> author
> > > > >> experience
> > > > >> > > >> to use a message queue scenario. I understood that this is
> > > > >> probably the
> > > > >> > > >> main use case we
> > > >> > > >> Thanks again for your time :)
> > > >> > > >>
> > > >> > > >>
> > > >> > > >>
> > > >> > >
> > > >>
> > >
> >
> https://cwik
4/07/29 15:58:23 Vincent Beck wrote:
> > >> > > >> > Thanks a lot all for the comments, this is very much
> > >> appreciated! I
> > >> > > >> received many comments from this thread and in confluence,
> thanks
> > >> again.
t; this
> >> > > >> thread once done. I will most likely revisit the push-based
> >> approach
> >> > > given
> >> > > >> the number of concerns I received, thanks Jarek for proposing
> >> another
> >> > > >>
t; >> >
>> > > >> > > The bespoke triggerer approach completely makes sense for the
>> long
>> > > >> tail here, but can we do better for the 20% of scenarios which
>> cover
>> > > well
>> > > >> over 80% of
h completely makes sense for the
> long
> > > >> tail here, but can we do better for the 20% of scenarios which cover
> > > well
> > > >> over 80% of usage here is the question in my mind. Or, are you
> thinking
> > > of
> > &
vered in the "push" model?
> > >> >
> > >> > Could you share more details about what is this "20% of scenarios
> > which
> > >> cover well over 80% of usage" please?
> > >> >
> > >> > Vincent
> > >> >
> > >> > On 2024/07/29 15
t is this "20% of scenarios
> which
> >> cover well over 80% of usage" please?
> >> >
> >> > Vincent
> >> >
> >> > On 2024/07/29 15:37:50 Kaxil Naik wrote:
> >> > > Thanks Vincent for driving these, I have added my comments to
> On Fri, 26 Jul 2024 at 20:16, Scheffler Jens (XC-AS/EAE-ADA-T)
>> > > wrote:
>> > >
>> > > > +1 on the comments of Vikram and Jarek, added main points on
>> confluence
>> > > >
>> > > > Sent from Outlook for iOS<htt
t; > Regards,
> > > Kaxil
> > >
> > > On Fri, 26 Jul 2024 at 20:16, Scheffler Jens (XC-AS/EAE-ADA-T)
> > > wrote:
> > >
> > > > +1 on the comments of Vikram and Jarek, added main points on
> confluence
> > > >
>
> Sent from Outlook for iOS<https://aka.ms/o0ukef>
> > > ____________
> > > From: Vikram Koka
> > > Sent: Friday, July 26, 2024 8:46:55 PM
> > > To: dev@airflow.apache.org
> > > Subject: Re: [DISCUSS] External event driven sc
ts on confluence
> >
> > Sent from Outlook for iOS<https://aka.ms/o0ukef>
> >
> > From: Vikram Koka
> > Sent: Friday, July 26, 2024 8:46:55 PM
> > To: dev@airflow.apache.org
> > Subject: Re: [DISCUSS] External event driven schedul
ka.ms/o0ukef>
>
> From: Vikram Koka
> Sent: Friday, July 26, 2024 8:46:55 PM
> To: dev@airflow.apache.org
> Subject: Re: [DISCUSS] External event driven scheduling in Airflow
>
> Vincent,
>
> Thanks for writing this up. The overview looks re
+1 on the comments of Vikram and Jarek, added main points on confluence
Sent from Outlook for iOS<https://aka.ms/o0ukef>
From: Vikram Koka
Sent: Friday, July 26, 2024 8:46:55 PM
To: dev@airflow.apache.org
Subject: Re: [DISCUSS] External event driven sche
Vincent,
Thanks for writing this up. The overview looks really good!
I will leave my comments in the AIP as well, but at a high level they are
both relatively focused on the "how", rather than the "what".
With respect to the pull / polling approach, I completely agree that some
incarnation of thi
Thanks Vincent. I took a look and I have a general comment. I
strongly think external driven scheduling is really needed - especially, it
should be much easier for a user to "plug-in" such an external event to
Airflow. And there are two parts of it - as correctly stated there - pull
and push.
For
Hello everyone,
I created a draft AIP regarding "External event driven scheduling in Airflow".
This proposal is about adding capability in Airflow to schedule DAGs based on
external events. Here are some examples of such external events:
- A user signs up to one of the user pool defined in my cl
21 matches
Mail list logo
