wjddn279 commented on PR #64236: URL: https://github.com/apache/airflow/pull/64236#issuecomment-4150134282
@jscheffl Looking at the implementation and comments of BaseExecutor's `_process_workloads`. https://github.com/apache/airflow/blob/main/airflow-core/src/airflow/executors/base_executor.py#L265-L275 It's defined as something subclasses must implement. The reason it's enforced is that all objects inheriting from BaseExecutor are assumed to follow the pattern of queuing tasks into an internal queue via `queue_workload` and processing them in `_process_workloads`. EdgeExecutor is the only case (so far) that doesn't follow this pattern. However, as long as it inherits from BaseExecutor, I believe it should still implement `_process_workloads` and explicitly state that it's an exception case, even if it's not actually used. To be precise, I think BaseExecutor should ideally be split — something like `BaseExecutorWithQueue` and `BaseExecutorWithoutQueue`. But since EdgeExecutor is the only such case, that feels like overkill for now. (Though it should be revisited if more cases come up.) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
