I didn't test the code bellow, but hopefully you get the idea:
----------------------
def foo(templates_dict, *args, **kwargs):
print(templates_dict['ds'])
op = PythonOperator('a_task_id', python_callable=foo, templates_dict={'ds':
'{{ ds }}'})
# All of the values in the `templates_dict` should get templated by the
Airflow engine, meaning that in this context, `foo` should print the value
of the `ds`.
Max
On Thu, Jan 19, 2017 at 6:22 AM, Boris Tyukin <bo...@boristyukin.com> wrote:
> Maxime, I have a related question. Can you explain how template_dict work
> with PythonOperator? Documentation is very vague about it. Example would be
> nice. I could not find a way to use jinja template with Python operator
> i.e. passing templated parameters to a pythonoperator callable. Someone
> told me the only way to do it is to create a wrapper function and use xcom
> or macro from there
>
> On Wed, Jan 18, 2017 at 10:09 PM, Maxime Beauchemin <
> maximebeauche...@gmail.com> wrote:
>
> > Here's the list of variables and macros exposed in the jinja context:
> > https://airflow.apache.org/code.html#macros
> >
> > Those are all exposed by the framework, meaning they are usable for any
> > templated field without doing anything special.
> >
> > There are ways to pass your own variables and methods using the `params`
> > attribute of any operator as shown in the tutorial here
> > <https://airflow.apache.org/tutorial.html#example-pipeline-definition>,
> > and
> > there's also a `user_defined_macros` parameter as a dict you can pass
> while
> > creating the DAG object. Every key in that dictionary is made available
> in
> > the global jinja namespace.
> >
> > https://airflow.apache.org/code.html#airflow.models.DAG
> >
> > Max
> >
> > On Wed, Jan 18, 2017 at 11:31 AM, Boris Tyukin <bo...@boristyukin.com>
> > wrote:
> >
> > > Hi Guilherme,
> > >
> > > I guess it depends what exactly you want to do as not everything works
> > with
> > > jinja.
> > >
> > > From documentation:
> > >
> > > https://pythonhosted.org/airflow/concepts.html#jinja-templating
> > >
> > > You can use Jinja templating with every parameter that is marked as
> > > “templated” in the documentation.
> > >
> > > You can open source code for operators and see what parameters are
> > actually
> > > templated.
> > >
> > > For example, if you open source code for BashOperator
> > > https://pythonhosted.org/airflow/_modules/bash_
> > operator.html#BashOperator
> > > you will see this line of code:
> > > template_fields = ('bash_command', 'env')
> > >
> > > It means only bash_command and env will work with jinja templates.
> > >
> > > On Wed, Jan 18, 2017 at 12:45 PM, Guilherme Marthe <
> > > guilherme.mar...@enjoei.com.br> wrote:
> > >
> > > > Hey folks!
> > > >
> > > > I am trying to write a dag that works well with jinja templating,
> since
> > > > through my study of the documentation, is the best way to ensure
> > > > compatibility with the back-fill function.
> > > >
> > > > Are you guys aware of any examples online with this functionality
> > > working?
> > > > The, documentation is still a bit dry on insightful examples, and I
> am
> > a
> > > > noobie developer, so I am trying to make the best use of the tool in
> > the
> > > > "correct way".
> > > >
> > > > Thank you in advance for any resources you guys can share!
> > > >
> > > > Gui
> > > >
> > > > Ps: I ve been told I can email you guys with questions like these :)
> > Hope
> > > > this is not a hassle ^_^
> > > >
> > >
> >
>
