[
https://issues.apache.org/jira/browse/AIRFLOW-2615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kevin Yang updated AIRFLOW-2615:
--------------------------------
Description: From what I can tell, the app cached
[here|https://github.com/apache/incubator-airflow/blob/master/airflow/bin/cli.py#L790]
attempt to cache the app for later use-likely to be for the expensive DagBag()
creation. Before I dive into the webserver parsing everything in one process
problem, I was hoping this cached app would save me sometime. However it seems
to me that every subprocess spun up by gunicorn is trying to create the
DagBag() right after they've been created--make sense to me since we didn't
share the cached app to the subprocess( doubt we can). If what I observed is
true, why do we cache the app at all in the parent process? (was: From what I
can tell, the app cached
[here|https://github.com/apache/incubator-airflow/blob/master/airflow/bin/cli.py#L790]
attempt to cache the app for later use-likely to be for the expensive DagBag()
creation. Before I dive into the webserver parsing everything in one process
problem, I was hoping this cached app would save me sometime. However it seems
to me that every subprocess spun up by gunicorn is trying to create the
DagBag() right after they've been created--make sense to me since we didn't
share the cached app to the subprocess( doubt we can). If what I observed is
true, why do we cache the app at all?)
> Webserver parent not using cached app
> -------------------------------------
>
> Key: AIRFLOW-2615
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2615
> Project: Apache Airflow
> Issue Type: Bug
> Reporter: Kevin Yang
> Priority: Major
>
> From what I can tell, the app cached
> [here|https://github.com/apache/incubator-airflow/blob/master/airflow/bin/cli.py#L790]
> attempt to cache the app for later use-likely to be for the expensive
> DagBag() creation. Before I dive into the webserver parsing everything in one
> process problem, I was hoping this cached app would save me sometime. However
> it seems to me that every subprocess spun up by gunicorn is trying to create
> the DagBag() right after they've been created--make sense to me since we
> didn't share the cached app to the subprocess( doubt we can). If what I
> observed is true, why do we cache the app at all in the parent process?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
