ecodina opened a new issue, #65945:
URL: https://github.com/apache/airflow/issues/65945
### Under which category would you file this issue?
Airflow Core
### Apache Airflow version
3.2.1
### What happened and how to reproduce it?
I upgraded from 3.1.8 to 3.2.1 and the Triggerer didn't start properly for
my prod instance.
My staging instance worked well (and has worked well for over a week). It
uses the same exact Docker image as prod, and the same airflow.cfg (except the
value of parallelism).
The triggerer was sending the heartbeat, but the logs only showed:
`[info] Starting the triggerer
[airflow.jobs.triggerer_job_runner.TriggererJobRunner]
loc=triggerer_job_runner.py:201`
I changed the log level to debug and this error appeared. All other logs
looked as if it was working correctly:
```
Stack (most recent call last):
File "/usr/python/lib/python3.12/threading.py", line 1032, in _bootstrap
self._bootstrap_inner()
self.run()
File "/usr/python/lib/python3.12/threading.py", line 1075, in
_bootstrap_inner
File "/usr/python/lib/python3.12/threading.py", line 1012, in run
self._target(*self._args, **self._kwargs)
File "/usr/python/lib/python3.12/asyncio/base_events.py", line 645, in
run_forever
self._run_once()
File "/usr/python/lib/python3.12/asyncio/events.py", line 88, in _run
handle._run()
File "/usr/python/lib/python3.12/asyncio/base_events.py", line 1999, in
_run_once
self._context.run(self._callback, *self._args)
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/api_fastapi/execution_api/app.py",
line 353, in start_lifespan
await cm.enter_async_context(app.router.lifespan_context(app))
File "/usr/python/lib/python3.12/contextlib.py", line 659, in
enter_async_context
result = await _enter(cm)
File "/usr/python/lib/python3.12/contextlib.py", line 210, in __aenter__
return await anext(self.gen)
File
"/home/airflow/.local/lib/python3.12/site-packages/fastapi/routing.py", line
216, in merged_lifespan
async with original_context(app) as maybe_original_state:
File "/home/airflow/.local/lib/python3.12/site-packages/svcs/fastapi.py",
line 66, in __call__
async with self.registry, cm(app, self.registry) as state:
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/api_fastapi/execution_api/app.py",
line 96, in lifespan
registry.register_factory(JWTGenerator, _jwt_generator)
File "/home/airflow/.local/lib/python3.12/site-packages/svcs/_core.py",
line 294, in register_factory
File "/usr/python/lib/python3.12/logging/__init__.py", line 1527, in debug
log.debug(
self._log(DEBUG, msg, args, **kwargs)
File "/usr/python/lib/python3.12/logging/__init__.py", line 1684, in _log
File "/usr/python/lib/python3.12/logging/__init__.py", line 999, in format
File "/usr/python/lib/python3.12/logging/__init__.py", line 1160, in emit
msg = self.format(record)
self.emit(record)
File "/usr/python/lib/python3.12/logging/__init__.py", line 1028, in handle
File "/usr/python/lib/python3.12/logging/__init__.py", line 1762, in
callHandlers
self.callHandlers(record)
File "/usr/python/lib/python3.12/logging/__init__.py", line 1700, in handle
self.handle(record)
hdlr.handle(record)
return fmt.format(record)
```
Airflow sent the tasks to the triggerer, but the trigger code wasn't
executed.
I had to downgrade to 3.1.8, which left the task instances and dag runs
corrupted, so I had to recover the backup.
### Environment
Deployed on docker swarm using the official image (with some requirements
installed)
We use LocalExecutor
### What you think should happen instead?
_No response_
### Operating System
_No response_
### Deployment
Other Docker-based deployment
### Apache Airflow Provider(s)
_No response_
### Versions of Apache Airflow Providers
All latest versions existing the 27/april/2026
### Official Helm Chart version
Not Applicable
### Kubernetes Version
_No response_
### Helm Chart configuration
_No response_
### Docker Image customizations
The Dockerfile installs the following requirements:
```
apache-airflow-client
apache-airflow-core[gunicorn]>=3.2.0
apache-airflow[async,graphviz,otel,standard]
apache-airflow[postgres,ftp,http,imap,sftp,smtp,ssh,redis,keycloak,git]
```
It also installs our provider, that contains the deferrable operator code
and custom logging:
log_config.py:
```
LOGGING_CONFIG["handlers"]["task"] = {
"class":
"example.airflow.provider.logging.example_logs.SlurmSharedFileTaskHandler",
"formatter": "airflow",
"base_log_folder": BASE_LOG_FOLDER,
"filters": ["mask_secrets_core"],
}
```
The `SlurmSharedFileTaskHandler` inherits from `FileTaskHandler` and
`LoggingMixin` and only overwrites the `_read()` method (which I believe it is
only used from the api server).
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
