pragnareddye commented on code in PR #32310: URL: https://github.com/apache/airflow/pull/32310#discussion_r1252511695
########## docs/apache-airflow/core-concepts/executor/local.rst: ########## @@ -47,3 +47,12 @@ Arguably, :class:`~airflow.executors.sequential_executor.SequentialExecutor` cou parallelism of just 1 worker, i.e. ``self.parallelism = 1``. This option could lead to the unification of the executor implementations, running locally, into just one :class:`~airflow.executors.local_executor.LocalExecutor` with multiple modes. + +.. note:: + + When multiple Schedulers are configured with ``executor = LocalExecutor`` in the ``[core]`` section of your ``airflow.cfg``, each scheduler will run a LocalExecutor. This means tasks would be processed in a distributed fashion across the machines running the Schedulers. + + Some considerations should be taken into account: + + - Restarting a Scheduler: If a Scheduler is restarted, it may take some time for other Schedulers to recognize the orphaned tasks and restart or fail them. + - Sharing of instances: While a single LocalExecutor allows tasks to share the same Python process and exchange information via variables, this is not possible when using multiple Schedulers, as each Scheduler will run in its own separate Python process. Review Comment: You are actually right. xcom is the recommended way and this comment note be a bit misleading. I will remove this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
