shivaam commented on issue #62247: URL: https://github.com/apache/airflow/issues/62247#issuecomment-3988166952
Tested multi-team AWS Batch executor config isolation with two teams sharing the same AwsBatchExecutor class but with independent configurations. Setup: Airflow running from main (3.2.0dev) on EC2 Two teams: team_alpha and team_beta Each team has its own [team_name=aws_batch_executor] config section pointing to separate Batch job queues and job definitions Config: ``` [core] executor = LocalExecutor;team_alpha=AwsBatchExecutor;team_beta=AwsBatchExecutor [team_alpha=aws_batch_executor] job_name = alpha-airflow-worker job_queue = alpha-batch-queue job_definition = alpha-batch-job-def [team_beta=aws_batch_executor] job_name = beta-airflow-worker job_queue = beta-batch-queue job_definition = beta-batch-job-def ``` Results: - Triggered alpha_simple_dag → Batch job submitted to alpha-batch-queue with alpha-batch-job-def ✅ - Triggered beta_simple_dag → Batch job submitted to beta-batch-queue with beta-batch-job-def ✅ - Scheduler correctly routed alpha to AwsBatchExecutor(team_name='team_alpha') and beta to AwsBatchExecutor(team_name='team_beta') ✅ - No cross-contamination: alpha jobs only appear in alpha-batch-queue, beta jobs only in beta-batch-queue ✅ Bundle routing confirmed from Batch job command payloads: Alpha: bundle_info.name=team_alpha_dags, dag_rel_path=alpha_dag.py ✅ Beta: bundle_info.name=team_beta_dags, dag_rel_path=beta_dag.py ✅ Each team's executor instance correctly reads its own config section and submits jobs to the corresponding Batch infrastructure. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
