Hi Sergio,
We did some benchmarking with Local & K8 Executor Mode. We observed that Each
Airflow Tasks takes ~100 MB of memory in Local Executor Mode.
With 16 GB of RAM we could run ~140 concurrent tasks. After this we started
getting "can not allocate memory error".
With K8 Executor memory
Hello everyone
Wanted to find out what is the transaction isolation level used in
airflow's MySQL meta db in your deployments.
Does anybody use READ COMMITTED instead of the default REPEATABLE READ
transaction isolation level of MySQL? Did you observe any issues?
Thanks,
pala
Hey folks,
Do we have something like airflow benchmarks?
Seems that many people seem to struggle to understand the limitations of
airflow (me included).
Is there some existing work on bechmarking (ie define a few common cases
and measure performance while increase volume of tasks)?
I know it's
Hi again,
After some research on other communities and based on the comments above I
restructure my proposal:
Slack Channels:
* announcements (as is)
- new releases
- community announcements
* general
- events (meetings, webinars, conferences, etc)
- material (tutorials, blog
+1
On Mon, Jul 1, 2019 at 11:20 PM Kaxil Naik wrote:
> +1 We have been using this on some of our Astronomer repositories as well
> and have been happy with it.
>
> Regards,
> Kaxil
>
> On Tue, Jul 2, 2019, 11:46 Jarek Potiuk wrote:
>
> > TL;DR: I would like to make a proposal to add (easily
Great work Jarek. I think the stalebot is a great addition. Even if an
issue gets closed unresolved, it is an indication to me that the issue
might not be relevant. In the end you can always reopen issues again.
Cheers, Fokko
Op di 2 jul. 2019 om 07:41 schreef Jarek Potiuk
> If we finally find
Just to remind - the vote is up till Thursday 7am CEST. W have so far +3
binding +5 non-binding votes so for now the status is PASS.
Thanks for all who participated :). I encourage others as well.
Anticipating positive vote I already implemented the scheme in the upcoming
final commit from
+1 We have been using this on some of our Astronomer repositories as well
and have been happy with it.
Regards,
Kaxil
On Tue, Jul 2, 2019, 11:46 Jarek Potiuk wrote:
> TL;DR: I would like to make a proposal to add (easily managed) pre-commit
> hooks with various lint checkers part of
TL;DR: I would like to make a proposal to add (easily managed) pre-commit
hooks with various lint checkers part of recommended (and encouraged)
development workflow for Airflow. This will help with speeding-up local
development cycle and decrease pressure on Travis CI.
Over the last few months
