jedcunningham commented on code in PR #32102:
URL: https://github.com/apache/airflow/pull/32102#discussion_r1280896747
##########
airflow/cli/commands/webserver_command.py:
##########
@@ -422,11 +422,18 @@ def webserver(args):
run_args += ["airflow.www.app:cached_app()"]
- # To prevent different workers creating the web app and
- # all writing to the database at the same time, we use the --preload
option.
- # With the preload option, the app is loaded before the workers are
forked, and each worker will
- # then have a copy of the app
- run_args += ["--preload"]
+ reload_on_plugin_change = conf.getboolean("webserver",
"reload_on_plugin_change", fallback=False)
+ if not reload_on_plugin_change:
+ # To prevent different workers creating the web app and
+ # all writing to the database at the same time, we use the
--preload option.
+ run_args += ["--preload"]
+ if reload_on_plugin_change:
+ log.warning(
+ "reload_on_plugin_change prevent running gunicorn with
--preload option. "
+ "With the preload option, the app is loaded before the workers
are forked, and each worker "
+ "will then have a copy of the app. It may cause IntegrityError
during webserver startup, so "
+ "avoid using reload_on_plugin_change in production."
Review Comment:
```suggestion
"will then have a copy of the app. reload_on_plugin_change
may cause IntegrityErrors during "
"webserver startup, so avoid using it in production."
```
##########
airflow/cli/commands/webserver_command.py:
##########
@@ -422,11 +422,18 @@ def webserver(args):
run_args += ["airflow.www.app:cached_app()"]
- # To prevent different workers creating the web app and
- # all writing to the database at the same time, we use the --preload
option.
- # With the preload option, the app is loaded before the workers are
forked, and each worker will
- # then have a copy of the app
- run_args += ["--preload"]
+ reload_on_plugin_change = conf.getboolean("webserver",
"reload_on_plugin_change", fallback=False)
+ if not reload_on_plugin_change:
+ # To prevent different workers creating the web app and
+ # all writing to the database at the same time, we use the
--preload option.
+ run_args += ["--preload"]
+ if reload_on_plugin_change:
+ log.warning(
+ "reload_on_plugin_change prevent running gunicorn with
--preload option. "
Review Comment:
```suggestion
"reload_on_plugin_change prevents running gunicorn with the
--preload option. "
```
nit
##########
airflow/config_templates/config.yml:
##########
@@ -1445,7 +1445,8 @@ webserver:
reload_on_plugin_change:
description: |
If set to True, Airflow will track files in plugins_folder directory.
When it detects changes,
- then reload the gunicorn.
+ then reload the gunicorn. If set to True, gunicorn starts without
``--preload`` setting and uses
+ more memory.
Review Comment:
```suggestion
then reload the gunicorn. If set to True, gunicorn starts without
``--preload``, which
is slower and uses more memory.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
