Hi Ash, that appears to be OK:
● airflow-scheduler.service - Airflow scheduler daemon Loaded: loaded (/lib/systemd/system/airflow-scheduler.service; enabled; vendor preset: enabled) Active: active (running) since Wed 2020-05-06 10:09:20 UTC; 3s ago Main PID: 10610 (/usr/bin/python) Tasks: 3 (limit: 4667) CGroup: /system.slice/airflow-scheduler.service ├─10610 /usr/bin/python3 /usr/local/bin/airflow scheduler -n 5 --pid /run/airflow/scheduler.pid └─10641 airflow scheduler -- DagFileProcessorManager May 06 10:09:24 ip-10-1-17-115 airflow[10610]: [2020-05-06 10:09:24,119] {scheduler_job.py:1504} DEBUG - Heartbeating the executor May 06 10:09:24 ip-10-1-17-115 airflow[10610]: [2020-05-06 10:09:24,120] {base_executor.py:122} DEBUG - 0 running task instances May 06 10:09:24 ip-10-1-17-115 airflow[10610]: [2020-05-06 10:09:24,120] {base_executor.py:123} DEBUG - 0 in queue May 06 10:09:24 ip-10-1-17-115 airflow[10610]: [2020-05-06 10:09:24,120] {base_executor.py:124} DEBUG - 32 open slots May 06 10:09:24 ip-10-1-17-115 airflow[10610]: [2020-05-06 10:09:24,120] {base_executor.py:133} DEBUG - Calling the <class 'airflow.executors.celery_executor.CeleryExecutor'> sync method May 06 10:09:24 ip-10-1-17-115 airflow[10610]: [2020-05-06 10:09:24,120] {celery_executor.py:240} DEBUG - No task to query celery, skipping sync May 06 10:09:24 ip-10-1-17-115 airflow[10610]: [2020-05-06 10:09:24,120] {scheduler_job.py:1459} DEBUG - Ran scheduling loop in 0.00 seconds May 06 10:09:24 ip-10-1-17-115 airflow[10610]: [2020-05-06 10:09:24,121] {scheduler_job.py:1462} DEBUG - Sleeping for 1.00 seconds May 06 10:09:24 ip-10-1-17-115 airflow[10610]: [2020-05-06 10:09:24,153] {scheduler_job.py:268} DEBUG - Waiting for <Process(DagFileProcessor1-Process, stopped)> May 06 10:09:24 ip-10-1-17-115 airflow[10610]: [2020-05-06 10:09:24,234] {settings.py:278} DEBUG - Disposing DB connection pool (PID 10646) # cat /etc/airflow/airflow.cfg | grep -i execut # The executor class that airflow should use. Choices include # SequentialExecutor, LocalExecutor, CeleryExecutor executor = CeleryExecutor Best, Andreas > On 6. May 2020, at 12:03, Ash Berlin-Taylor <a...@apache.org> wrote: > > Your second point there would lead me to believe that the scheduler is > still actually running with the default SequentialExecutor. > > Which config file did you edit? > > What output is shown when you (re)start the scheduler? > > Thanks, > -ash > > On May 6 2020, at 11:00 am, Andreas Balke <andreas.ba...@now-extern.de> wrote: > >> Dear Airflow community, >> >> not sure if I’m targeting the right audience here. Just trying though >> :) >> >> In a basic setup, like described here: >> https://www.cloudwalker.io/2019/09/30/airflow-scale-out-with-redis-and-celery/ >> <https://www.cloudwalker.io/2019/09/30/airflow-scale-out-with-redis-and-celery/>, >> using `airflow_version: 1.10.10`, I discover two issues: >> >> 1) Even though the `airflow.cfg` is configured to use `CeleryExecutor` >> +Redis and all the logs mention to use it, there is no Task visible in >> Flower. When a DAG is started, it will only be processed via the DB >> >> 2) Many times, especially once a DAG is running, in the UI I see the >> warning "The scheduler does not appear to be running.” >> >> Do you have any advice how to solve those? >> >> Best, Andreas