GitHub user ecodina created a discussion: Dag Versioning keeps increasing
Hello,
I am having trouble finding out why some Dags keep increasing their version. I
am using Airflow 3.1.7.
We have an internal webpage that generates Python Dags based on a web form. An
example of a generated Dag is the following:
```python
from airflow.sdk import DAG
from airflow.timetables.interval import CronDataIntervalTimetable
from datetime import datetime, timedelta
from myprovider.airflow.provider.operators.slurm import SlurmOperator
default_args = {
'owner': 'myuser',
'start_date': datetime(2024, 10, 31),
'end_date': datetime(2026, 10, 31),
'weight_rule': 'downstream',
}
with DAG(
dag_id='example.redacted',
default_args=default_args,
description='redacted',
schedule=CronDataIntervalTimetable(cron='*/2 * * * *', timezone='UTC'),
catchup=False,
max_active_runs=1,
tags=['OBS', 'MANAGED'],
) as dag:
get_obs = SlurmOperator(
task_id='get_obs',
command='redacted_it_is_a_string',
retries=1,
retry_delay=timedelta(seconds=300),
retry_exponential_backoff=False,
depends_on_past=False,
wait_for_downstream=False,
execution_timeout=timedelta(seconds=660),
max_active_tis_per_dag=1,
do_xcom_push=False,
priority_weight=1,
tdelta_between_checks=5,
env={'SBATCH_MEM_PER_NODE': '1G', 'SBATCH_PARTITION': 'high',
'SBATCH_TIMELIMIT': '00:10:00'},
slurm_options={'CHDIR': '.', 'NODES': 1, 'NTASKS': 1},
cluster='XXX',
notification_emails=['[email protected]'],
)
```
The above Dag has 2365 versions:
<img width="280" height="455" alt="image"
src="https://github.com/user-attachments/assets/63502ac6-ed59-410b-940a-0d2209ba1852";
/>
However, it has not been modified since February 11.
The generated dags live in a folder inside $AIRFLOW_HOME/dags. This folder is a
docker volume, mounted on a NFS system and shared with the internal web page.
The volume options are: `nfsvers=4.2,addr=172.XX.XX.XX,actimeo=0,hard,rw`
The Dag has everything hard-coded, so I am not sure what could be changing. My
only idea is that this has something to do with the NFS share?
It is also strange that it increases the version only in some days (as you can
see in the screenshot, the last day it increased versions was 2 weeks ago). Not
all Dags have the same problem, and not all the Dags that increase their
version randomly, increase it at the same day/hour.
Any help will be welcome!
GitHub link: https://github.com/apache/airflow/discussions/66103
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
