Pebble32 opened a new issue, #76542:
URL: https://github.com/apache/airflow/issues/76542
### Under which category would you file this issue?
Airflow Core
### Apache Airflow version
3.4.0rc3 (also on 3.4.0rc2)
### What happened and how to reproduce it?
When a custom timetable calls `Variable.get` while it is being deserialized,
the call closes the database session of the scheduler or Dag processor that
triggered the deserialization. This happens whether the Variable exists or not.
In server processes, the SDK `Variable.get` goes to
`MetastoreBackend.get_variable`, which is decorated with `@provide_session` and
called without a session. `create_session()` then takes `settings.Session()`,
which is the same scoped session the caller is using, and commits and closes it
on exit. Trace captured on rc3 by spying on `Session.close`:
```
airflow/serialization/serialized_objects.py 2427 timetable
airflow/serialization/decoders.py 456 decode_timetable
airflow/sdk/definitions/variable.py 54 get
airflow/sdk/execution_time/context.py 383 _get_variable
airflow/utils/session.py 99 wrapper
airflow/utils/session.py 48 create_session <- closes the caller's session
```
Effects:
- Dag processor: the next access to the `DagModel` rows raises
`DetachedInstanceError`, logged as `Failed to persist parsing result for
rc_timetable_dags.py ...`, and DagModel updates for that file are dropped. The
Dags stay stale with no import error shown, even after the Variable is set.
- Scheduler: `DetachedInstanceError: Instance <DagModel at ...> is not bound
to a Session` in the scheduling loop, which stops the scheduler.
Steps to reproduce (SQLite, LocalExecutor, clean virtualenv with 3.4.0rc3):
1. Plugin in `$AIRFLOW_HOME/plugins/rc_timetable.py`:
```python
from airflow.plugins_manager import AirflowPlugin
from airflow.sdk import Variable
from airflow.timetables.trigger import CronTriggerTimetable
class VariableCronTimetable(CronTriggerTimetable):
@classmethod
def deserialize(cls, data):
Variable.get("rc_test_var")
return super().deserialize(data)
class RcTimetablePlugin(AirflowPlugin):
name = "rc_timetable_plugin"
timetables = [VariableCronTimetable]
```
2. Dag file `rc_timetable_dags.py`:
```python
import pendulum
from rc_timetable import VariableCronTimetable
from airflow.providers.standard.operators.empty import EmptyOperator
from airflow.sdk import DAG
with DAG(
"rc_bad_timetable",
schedule=VariableCronTimetable("0 0 1 1 *", timezone="UTC"),
start_date=pendulum.datetime(2030, 1, 1, tz="UTC"),
catchup=False,
):
EmptyOperator(task_id="noop")
with DAG(
"rc_good_same_file",
schedule="* * * * *",
start_date=pendulum.datetime(2026, 1, 1, tz="UTC"),
catchup=False,
):
EmptyOperator(task_id="noop")
```
3. `airflow db reset -y`, then `airflow variables set rc_test_var ok`.
4. `airflow dag-processor -n 3`: the log shows `Failed to persist parsing
result` with `DetachedInstanceError ... lazy load operation of attribute
'schedule_asset_references'`, and `rc_bad_timetable` is left stale with no
`fileloc`.
5. Running the scheduler with a run of `rc_bad_timetable` present hits
`DetachedInstanceError` on a `DagModel` and exits (seen on rc2; steps 1 to 4
were rerun on rc3).
### What you think should happen instead?
`Variable.get` called from code running inside the scheduler or Dag
processor should not commit or close the caller's session. It should either
reuse it or use a separate, non-scoped session.
#71968 ("Allow Variable.get to reuse the caller's database session") is
included in rc3 but does not cover this path.
### Operating System
macOS 26 (Darwin 25.6.0)
### Deployment
Virtualenv installation
### Anything else?
Found while testing 3.4.0rc2 for #74499 and #74709 (see the comment on
#75865). It is not caused by those changes: it also happens with them reverted.
Timetables that read Variables in `deserialize` are the trigger I hit, but any
server side code that calls the SDK `Variable.get` (or `Connection.get`) while
the scheduler or Dag processor holds ORM objects in the scoped session is
likely affected the same way, though I have only tested timetables.
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's Code of Conduct
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
