Max, this is awesome! Here is the working example if someone needs it - you
almost typed it right :)
def foo(templates_dict, *args, **kwargs):
return templates_dict['ds']
# All of the values in the `templates_dict` should get templated by the
Airflow engine,
# meaning that in this context, `foo` should print the value of the `ds`.
t4 = PythonOperator(
task_id='t4',
python_callable=foo,
provide_context=True,
templates_dict={
'ds': '{{ ds }}'
},
dag=dag)
On Thu, Jan 19, 2017 at 11:58 AM, Maxime Beauchemin <
maximebeauche...@gmail.com> wrote:
> I didn't test the code bellow, but hopefully you get the idea:
> ----------------------
>
> def foo(templates_dict, *args, **kwargs):
> print(templates_dict['ds'])
>
> op = PythonOperator('a_task_id', python_callable=foo, templates_dict={'ds':
> '{{ ds }}'})
> # All of the values in the `templates_dict` should get templated by the
> Airflow engine, meaning that in this context, `foo` should print the value
> of the `ds`.
>
> Max
>
> On Thu, Jan 19, 2017 at 6:22 AM, Boris Tyukin <bo...@boristyukin.com>
> wrote:
>
> > Maxime, I have a related question. Can you explain how template_dict work
> > with PythonOperator? Documentation is very vague about it. Example would
> be
> > nice. I could not find a way to use jinja template with Python operator
> > i.e. passing templated parameters to a pythonoperator callable. Someone
> > told me the only way to do it is to create a wrapper function and use
> xcom
> > or macro from there
> >
> > On Wed, Jan 18, 2017 at 10:09 PM, Maxime Beauchemin <
> > maximebeauche...@gmail.com> wrote:
> >
> > > Here's the list of variables and macros exposed in the jinja context:
> > > https://airflow.apache.org/code.html#macros
> > >
> > > Those are all exposed by the framework, meaning they are usable for any
> > > templated field without doing anything special.
> > >
> > > There are ways to pass your own variables and methods using the
> `params`
> > > attribute of any operator as shown in the tutorial here
> > > <https://airflow.apache.org/tutorial.html#example-pipeline-definition
> >,
> > > and
> > > there's also a `user_defined_macros` parameter as a dict you can pass
> > while
> > > creating the DAG object. Every key in that dictionary is made available
> > in
> > > the global jinja namespace.
> > >
> > > https://airflow.apache.org/code.html#airflow.models.DAG
> > >
> > > Max
> > >
> > > On Wed, Jan 18, 2017 at 11:31 AM, Boris Tyukin <bo...@boristyukin.com>
> > > wrote:
> > >
> > > > Hi Guilherme,
> > > >
> > > > I guess it depends what exactly you want to do as not everything
> works
> > > with
> > > > jinja.
> > > >
> > > > From documentation:
> > > >
> > > > https://pythonhosted.org/airflow/concepts.html#jinja-templating
> > > >
> > > > You can use Jinja templating with every parameter that is marked as
> > > > “templated” in the documentation.
> > > >
> > > > You can open source code for operators and see what parameters are
> > > actually
> > > > templated.
> > > >
> > > > For example, if you open source code for BashOperator
> > > > https://pythonhosted.org/airflow/_modules/bash_
> > > operator.html#BashOperator
> > > > you will see this line of code:
> > > > template_fields = ('bash_command', 'env')
> > > >
> > > > It means only bash_command and env will work with jinja templates.
> > > >
> > > > On Wed, Jan 18, 2017 at 12:45 PM, Guilherme Marthe <
> > > > guilherme.mar...@enjoei.com.br> wrote:
> > > >
> > > > > Hey folks!
> > > > >
> > > > > I am trying to write a dag that works well with jinja templating,
> > since
> > > > > through my study of the documentation, is the best way to ensure
> > > > > compatibility with the back-fill function.
> > > > >
> > > > > Are you guys aware of any examples online with this functionality
> > > > working?
> > > > > The, documentation is still a bit dry on insightful examples, and I
> > am
> > > a
> > > > > noobie developer, so I am trying to make the best use of the tool
> in
> > > the
> > > > > "correct way".
> > > > >
> > > > > Thank you in advance for any resources you guys can share!
> > > > >
> > > > > Gui
> > > > >
> > > > > Ps: I ve been told I can email you guys with questions like these
> :)
> > > Hope
> > > > > this is not a hassle ^_^
> > > > >
> > > >
> > >
> >
>
