Hi Sergio,
We did some benchmarking with Local & K8 Executor Mode. We observed that Each
Airflow Tasks takes ~100 MB of memory in Local Executor Mode.
With 16 GB of RAM we could run ~140 concurrent tasks. After this we started
getting "can not allocate memory error".
With K8 Executor memory foot
Hey folks,
Do we have something like airflow benchmarks?
Seems that many people seem to struggle to understand the limitations of
airflow (me included).
Is there some existing work on bechmarking (ie define a few common cases
and measure performance while increase volume of tasks)?
I know it's
