[
https://issues.apache.org/jira/browse/AIRFLOW-664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556286#comment-16556286
]
Tylar Murray commented on AIRFLOW-664:
--------------------------------------
After running into this myself using airflow v1.9.0 and confirming that the
issue was with the dag table in the database. In general the cause seems to be
that the scheduler and webserver are unable to reconcile differences between
the database and the filesystem. The steps I would suggest to resolve this are:
# check permissions on dag definition files to ensure airflow can read the
files
# restart scheduler and webserver in hopes they can work things out after
restart
# `airflow initdb` to reset the dag table using the filesystemÂ
Anecdotally, `initdb` doesn't affect any dags that still exist, but it will
clear dags not found on the filesystem and add ones on the filesystem but
missing from the database.
> airflow built from master unstable since change to parse DAG files in child
> process
> -----------------------------------------------------------------------------------
>
> Key: AIRFLOW-664
> URL: https://issues.apache.org/jira/browse/AIRFLOW-664
> Project: Apache Airflow
> Issue Type: Bug
> Components: scheduler
> Reporter: Dennis O'Brien
> Priority: Major
> Attachments: airflow_broken_scheduler_screenshot.png
>
>
> Prior to commit
> [fdb7e949140b735b8554ae5b22ad752e86f6ebaf|https://github.com/apache/incubator-airflow/commit/fdb7e949140b735b8554ae5b22ad752e86f6ebaf]
> I was able to pip build, install, and run airflow from the github commit.
> E.g.,
> {code}
> pip install -e
> git://github.com/apache/incubator-airflow.git@835bcb6234956445d819ff39a5790f635c201c5d#egg=airflow[celery,crypto,hive,jdbc,ldap,password,postgres,s3,vertica]
> {code}
> Since this commit, airflow is not functional.
> * The on/off button to enable a DAG does not appear.
> * Next to each DAG an info icon appears with the hover text "This DAG seems
> to be existing only locally. The master scheduler doesn't seem to be aware of
> its existence."
> * DAGs cannot be run from the command line.
> I am able via the command line to list the dags and list the tasks for a dag,
> but trigger_dag does not actually cause any tasks to run (but no error
> messages either).
> {code}
> airflow@41e0e6920223:~$ airflow trigger_dag example_bash_operator
> [2016-12-01 19:24:21,928] {__init__.py:50} INFO - Using executor
> CeleryExecutor
> [2016-12-01 19:24:22,218] {models.py:162} INFO - Filling up the DagBag from
> /usr/local/airflow/dags
> [2016-12-01 19:24:22,341] {cli.py:180} INFO - Created <DagRun
> example_bash_operator @ 2016-12-01 19:24:22:
> manual__2016-12-01T19:24:22.295709, externally triggered: True>
> {code}
> This commit was #1636 "[AIRFLOW-160] Parse DAG files through child
> processes". I haven't identified what part of this commit is to blame. Or,
> what part of my configuration/setup is incompatible with this change.
> My setup:
> * Docker and docker-compose with...
> * Ubuntu 16.04
> * Python 3.5.2 (via Miniconda distribution)
> * Separate container instances for scheduler, webserver, flower, and worker.
> * Docker container running mysql 5.7
> * Docker container running redis 3.2
> * Using CeleryExecutor with redis broker
> * Building and installing from master using `pip install -e git://...`
> I have a github project that uses Docker and docker-compose to build and test
> these components.
> [https://github.com/dennisobrien/airflow-docker-compose-example] .
> * pip install line in
> [Dockerfile|https://github.com/dennisobrien/airflow-docker-compose-example/blob/master/Dockerfile#L58]
> * [airflow.cfg
> template|https://github.com/dennisobrien/airflow-docker-compose-example/blob/master/config/airflow.cfg.tpl]
> *
> [docker-entrypoint.sh|https://github.com/dennisobrien/airflow-docker-compose-example/blob/master/bin/docker-entrypoint.sh]
> defines how each service is run.
> *
> [docker-compose.yml|https://github.com/dennisobrien/airflow-docker-compose-example/blob/master/docker-compose.yml]
> orchestrates the components.
> Let me know if there are additional tests I can do to narrow this down. I
> suspect since others have not complained about this that it is probably
> something related to my setup.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
