[
https://issues.apache.org/jira/browse/AIRFLOW-2615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kevin Yang updated AIRFLOW-2615:
--------------------------------
Description: From what I can tell, the app cached
[here|https://github.com/apache/incubator-airflow/blob/master/airflow/bin/cli.py#L790]
attempt to cache the app for later use-likely to be for the expensive DagBag()
creation. Before I dive into the webserver parsing everything in one process
problem, I was hoping this cached app would save me sometime. However it seems
to me that every subprocess spun up by gunicorn is trying to create the
DagBag() right after they've been created--make sense to me since we didn't
share the cached app to the subprocess( doubt we can). If what I observed is
true, why do we cache the app at all? (was: From what I can tell, the app
cached here attempt to cache the app for later use-likely to be for the
expensive DagBag() creation. Before I dive into the webserver parsing
everything in one process problem, I was hoping this cached app would save me
sometime. However it seems to me that every subprocess spun up by gunicorn is
trying to create the DagBag() right after they've been created--make sense to
me since we didn't share the cached app to the subprocess( doubt we can). If
what I observed is true, why do we cache the app at all?)
> Webserver not using cached app
> ------------------------------
>
> Key: AIRFLOW-2615
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2615
> Project: Apache Airflow
> Issue Type: Bug
> Reporter: Kevin Yang
> Priority: Major
>
> From what I can tell, the app cached
> [here|https://github.com/apache/incubator-airflow/blob/master/airflow/bin/cli.py#L790]
> attempt to cache the app for later use-likely to be for the expensive
> DagBag() creation. Before I dive into the webserver parsing everything in one
> process problem, I was hoping this cached app would save me sometime. However
> it seems to me that every subprocess spun up by gunicorn is trying to create
> the DagBag() right after they've been created--make sense to me since we
> didn't share the cached app to the subprocess( doubt we can). If what I
> observed is true, why do we cache the app at all?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
