The GitHub Actions job "Tests AMD" on airflow.git/change-to-copy has succeeded.
Run started by GitHub user kaxil (triggered by kaxil).

Head commit for run:
ba4f01944c5e3666153171d3ab972ceedbb612ab / Kaxil Naik <kaxiln...@gmail.com>
Fix Jinja2 Template deep copy error with ``dag.test``

The `dag.test` method currently (in 3.0.2) clears dag runs before running ( 
[code](https://github.com/apache/airflow/blob/3.0.2/task-sdk/src/airflow/sdk/definitions/dag.py#L1092-L1097)
 ) the dag. For that it uses `SchedulerDAG.clear_dags`, which create 
`SchedulerDAG` from Task SDK DAG -- to do that it uses 
[`deepcopy.copy`](https://github.com/apache/airflow/blob/3.0.2/airflow-core/src/airflow/models/dag.py#L1797).

Now that works for most cases, but not where `jinja2.Template` class is passed 
as a Task/Operator argument because of [this 
issue](https://github.com/pallets/jinja/issues/758).

```python
In [1]: from copy import deepcopy
   ...:
   ...: from jinja2 import Template
   ...:
   ...: deepcopy(Template(''))
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[1], line 5
      1 from copy import deepcopy
      3 from jinja2 import Template
----> 5 deepcopy(Template(''))

File 
~/.local/share/uv/python/cpython-3.12.8-macos-aarch64-none/lib/python3.12/copy.py:162,
 in deepcopy(x, memo, _nil)
    160                 y = x
    161             else:
--> 162                 y = _reconstruct(x, memo, *rv)
    164 # If is its own copy, don't memoize.
    165 if y is not x:

File 
~/.local/share/uv/python/cpython-3.12.8-macos-aarch64-none/lib/python3.12/copy.py:253,
 in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy)
    251 if deep and args:
    252     args = (deepcopy(arg, memo) for arg in args)
--> 253 y = func(*args)
    254 if deep:
    255     memo[id(x)] = y

File 
~/.local/share/uv/python/cpython-3.12.8-macos-aarch64-none/lib/python3.12/copyreg.py:99,
 in __newobj__(cls, *args)
     98 def __newobj__(cls, *args):
---> 99     return cls.__new__(cls, *args)

TypeError: Template.__new__() missing 1 required positional argument: 'source'
```

This is a general issue that can affect any DAG using custom arguments that 
store or cache Jinja2 Template objects or similar.
Clearly, the longer term solution for dag.test is to not use SchedulerDAG like 
that and instead call some task Execution endpoint to clear dagruns -- but this 
works for 3.0.x.

Report URL: https://github.com/apache/airflow/actions/runs/15619554146

With regards,
GitHub Actions via GitBox


---------------------------------------------------------------------
To unsubscribe, e-mail: jobs-unsubscr...@airflow.apache.org
For additional commands, e-mail: jobs-h...@airflow.apache.org

Reply via email to