Correction of course.
On Tue, Jul 12, 2022 at 8:01 PM Jarek Potiuk wrote:
>
> I think if we limit it to stress tests, this could be an "extra"
> addition - not even necessarily part of Airflow codebase and adding
> triggers with a script, on a single database, some kind of
> test-harness that y
I think if we limit it to stress tests, this could be an "extra"
addition - not even necessarily part of Airflow codebase and adding
triggers with a script, on a single database, some kind of
test-harness that you always add after you installed airflow in test
environment - for that I have far less
Hi Jarek,
Thanks for the insights and pointing out the potential issues with triggers
in the prod with scheduler HA setup.
The solution that I proposed is mainly for the stress test scheduler before
each airflow release. We can make changes in the airflow codebase to emit
this metric however:
1.
I also think SLAs make sense in airflow and think we should fix the feature.
I agree they SLA as they work now is basically non functional, but strongly
disagree with removing them and think a high level concept such as SLA must
belong in Airflow.
Ultimately what the vast majority of users want to do with Airflow is process
data _and make it available_ in a timely fashi
Can vote for making it as an optional approach for fine-tuning (only for
advance users).
On 12-Jul-2022 at 7:44:35 AM, Jarek Potiuk wrote:
> Not interesting :) ?
>
> On Thu, Jul 7, 2022 at 10:41 AM Jarek Potiuk wrote:
>
>
> Hello everyone,
>
>
> We have just published a blog on our medium -
> h
Thanks for bringing this up. I agree SLAs have been broken forever and try to
stay away from them.
However, I do see people trying to use them (not knowing it's broken). While I
appreciate the effort others have made to build a system around Airflow for
correct SLA alerting, I think Airflow is
Not interesting :) ?
On Thu, Jul 7, 2022 at 10:41 AM Jarek Potiuk wrote:
>
> Hello everyone,
>
> We have just published a blog on our medium -
> https://medium.com/apache-airflow/airflows-magic-loop-ec424b05b629 - that is
> a blog of one of our users Itay Bittan (thanks!) who had been inspired
Hey everyone,
I keep on being involved in discussions where people are complaining
about how bad and useless the SLA feature of Airflow is. And yeah, I
pretty much agree with it.
Without getting into details of why it is bad - should we possibly
just, well, deprecate it? I think that would give a
