kcphila opened a new issue, #24420:
URL: https://github.com/apache/airflow/issues/24420
### Apache Airflow version
2.3.2 (latest released)
### What happened
This seems odd and there may be another answer for it, but I can't find
anything. We have been running 2.1.4 on Ubuntu Xenial for about a year without
much trouble. So, we are switching over to the new Ubuntu 22.04 LTS environment
with all up to date libraries.
When attempting to run Airflow 2.3.2, I am getting strange config-related
errors when processing our custom config variable (full stack trace at the
bottom).
` log_processor_filename_template = {{{{ filename }}}}.log` - This is the
value included in the default template, but it fails to process with the error
`jinja2.exceptions.TemplateSyntaxError: expected token ':', got '}'`
The default `log_filename_template` value also fails to process with a
similar error
Commenting these out allows the config file to process, however it is not
substituting templates correctly. The multiple `{AIRFLOW_HOME}` references are
being interpreted literally instead of being substituted for the actual value.
If I change these to reference an environment variable (e.g.,
`${AIRFLOW_HOME}`), it works fine.
Here's the stack trace for the log filename template. I put in a custom
breakpoint to determine that it indeed is just for the log_filename_template
and log_processor_filename_template variables:
```
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: Unable to load the config,
contains a configuration error.
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: Traceback (most recent call
last):
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File
"/usr/lib/python3.10/logging/config.py", line 565, in configure
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: handler =
self.configure_handler(handlers[name])
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File
"/usr/lib/python3.10/logging/config.py", line 746, in configure_handler
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: result =
factory(**kwargs)
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File
"/usr/local/lib/python3.10/dist-packages/airflow/utils/log/file_processor_handler.py",
line 44, in _init_
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: self.filename_template,
self.filename_jinja_template = parse_template_string(filename_template)
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File
"/usr/local/lib/python3.10/dist-packages/airflow/utils/helpers.py", line 176,
in parse_template_string
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: return None,
jinja2.Template(template_string)
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File
"/usr/lib/python3/dist-packages/jinja2/environment.py", line 1195, in _new_
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: return
env.from_string(source, template_class=cls)
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File
"/usr/lib/python3/dist-packages/jinja2/environment.py", line 1092, in
from_string
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: return
cls.from_code(self, self.compile(source), gs, None)
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File
"/usr/lib/python3/dist-packages/jinja2/environment.py", line 757, in compile
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]:
self.handle_exception(source=source_hint)
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File
"/usr/lib/python3/dist-packages/jinja2/environment.py", line 925, in
handle_exception
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: raise
rewrite_traceback_stack(source=source)
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "<unknown>", line 1,
in template
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]:
jinja2.exceptions.TemplateSyntaxError: expected token ':', got '}'
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: The above exception was the
direct cause of the following exception:
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: Traceback (most recent call
last):
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File
"/usr/local/bin/airflow", line 5, in <module>
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: from airflow._main_
import main
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File
"/usr/local/lib/python3.10/dist-packages/airflow/_init_.py", line 47, in
<module>
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: settings.initialize()
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File
"/usr/local/lib/python3.10/dist-packages/airflow/settings.py", line 561, in
initialize
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: LOGGING_CLASS_PATH =
configure_logging()
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File
"/usr/local/lib/python3.10/dist-packages/airflow/logging_config.py", line 73,
in configure_logging
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: raise e
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File
"/usr/local/lib/python3.10/dist-packages/airflow/logging_config.py", line 68,
in configure_logging
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]:
dictConfig(logging_config)
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File
"/usr/lib/python3.10/logging/config.py", line 810, in dictConfig
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]:
dictConfigClass(config).configure()
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File
"/usr/lib/python3.10/logging/config.py", line 572, in configure
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: raise
ValueError('Unable to configure handler '
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: ValueError: Unable to
configure handler 'processor'
```
### What you think should happen instead
Whatever is in the [official default
template](https://github.com/apache/airflow/blob/2.3.2/airflow/config_templates/default_airflow.cfg)
should work. I don't mind using the bash syntax in the config file, but this
seems very strange.
### How to reproduce
_No response_
### Operating System
Ubuntu 22.04 AMD64 running on AWS
### Versions of Apache Airflow Providers
Airflow 2.3.2
### Deployment
Other
### Deployment details
pip-based install. I've tried this with jinja2 versions 3.1 and 3.0.3
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
