o-nikolas commented on issue #41055: URL: https://github.com/apache/airflow/issues/41055#issuecomment-2253320678
@ephraimbuddy I was able to reproduce this with another executor (SequentialExecutor) and I get the same behaviour. I looked more closely at our docs and configuration reference and it does not actually say that parallelism=0 means infinite scheduling: > This defines the maximum number of task instances that can run concurrently per scheduler in Airflow, regardless of the worker count. Generally this value, multiplied by the number of schedulers in your cluster, is the maximum number of task instances with the running state in the metadata database. There are no tests that assert this either (at least none that are failing as of now). So I believe this was working as an infinity kind of as a half-baked non-feature previously. So right now the code is behaving as expected. You told it that zero tasks can run concurrently so it is respecting that, and well, running zero tasks. We'll need to decide if we want to support this case more generally/officially. Tagging some folks to join in on the conversation: @eladkal @potiuk -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
