AutomationDev85 commented on PR #43737: URL: https://github.com/apache/airflow/pull/43737#issuecomment-2461447690
> Should not that capacity be a task parameter rather than executor config parameter on DAG level. We have similar concept with `pool_slots` and there they are "per task" - and part of the BaseOperator. It seems to be way more flexible to specify it this way (additionally then this could be renamed as "task_slots" - to be similar to "pool_slots") or maybe even we should combine the two. This way it will also be potentially usable by other executors. I was thinking in the same way, but during coding I saw that the need_capacity parameter is no easy to get into the executor. We have to tough core code like TaskInstanceKey class to get the info into the Executor. My main idea is to tough only Edge package t and then make a later PR which can add this changes into the core because the Edge package is the only which will support this feature for the moment and it is not released yet. So what is your opinion about that? Shall we change this also in this PR or in a separate PR? During writing this lines I have also the feeling to use still the term concurrency instead of capacity. Then it is easier to adapt this to already existing Executor code in the future. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
