ashmeet13 commented on issue #11144:
URL: https://github.com/apache/airflow/issues/11144#issuecomment-701439517
Some progress that I have made. Thanks @ashb for guiding towards
`TaskInstance` for getting the complete context.
Also towards `BaseOperator.render_template`
With this I was able to improve the logic to extract undefined variables in
a template to this -
```
def check_rendered(rendered):
if isinstance(rendered, six.string_types):
return set(re.findall(r"{{(.*?)}}", rendered))
elif isinstance(rendered, (tuple,list,set)):
parsed_templates = set()
for element in rendered:
parsed_templates.union(check_rendered(element))
return parsed_templates
elif isinstance(rendered, dict):
parsed_templates = set()
for key, value in rendered.items():
parsed_templates.union(check_rendered(value))
return parsed_templates
def check(dag):
dag.template_undefined = jinja2.DebugUndefined
for task in dag.tasks:
jinja_env = task.get_template_env()
task_instance = TaskInstance(task=task,
execution_date=timezone.utcnow())
template_context = task_instance.get_template_context()
for attr_name in task.template_fields:
content = getattr(task, attr_name)
if content:
rendered = task.render_template(content, template_context)
undefined = check_rendered(rendered)
for element in undefined:
print(element)
```
Take into the consideration the following DAG -
```
dag = DAG(
'tutorial',
default_args=default_args,
description='A simple tutorial DAG',
schedule_interval=timedelta(days=1),
)
templated_command_string = """
{% for i in range(5) %}
echo "{{ params.defined }}"
echo "{{ execution_date.month }}"
echo "{{ params.undefined }}"
echo "{{ foo }}"
{% endfor %}
"""
t1 = BashOperator(
task_id='templated_string',
depends_on_past=False,
bash_command=templated_command_string,
params={'defined': 'test_val'},
dag=dag,
)
```
The above check would print/return this -
```
foo
no such element: dict object['undefined']
```
One blocker still that I have is how exactly will I be able to get the dags
when the user runs `airflow upgrade-check`
Thanks again for the help!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
