Yea I like that plan. Curious to hear what other users think?
On Wed, Apr 29, 2020 at 12:33 PM Ash Berlin-Taylor <a...@apache.org> wrote:
> We could do this globally for 1.10, but for 2.0 I think it might be nice
> to change this as follows:
>
> Rename template_ext to file_exts, and it looks for files with those
> extensions, but doesn't render them.
>
> And then make it only render files with, for example, .sh.jinja2.
>
> Right now it's impossible to have scripts/files external to the DAG that
> aren't rendered, so you have to be careful to escape {{ in them, which
> can be annoying.
>
> WDYT?
>
> -a
>
> On Apr 29 2020, at 12:17 pm, Ori Popowski <ori....@gmail.com> wrote:
>
> > No. Any extension mentioned in `template_ext` will still be templated.
> But
> > in addition, also those ending with `.jinja2`, as long it's preceded
> > by one
> > of those extensions (meaning that `file.jinja2` won't be templated)
> >
> > On Wed, Apr 29, 2020 at 1:41 PM Ash Berlin-Taylor <a...@apache.org>
> wrote:
> >
> >> Are you also proposing that, for example .sh would not be templated, but
> >> that .sh.jinja2 would be templated?
> >>
> >> Ash
> >>
> >> On 29 April 2020 11:39:13 BST, Ori Popowski <ori....@gmail.com> wrote:
> >> >At the moment, various operators support reading templates from given
> >> >filenames (for example, `bash_command` of `BashOperator`). However, all
> >> >of
> >> >these operators specify extensions such as .sh, .json etc., so it means
> >> >users need to write Jinja2 templates inside .sh, .json files. This is
> >> >problematic for 3 reasons:
> >> >
> >> >1. No semantic support when writing templates (major IDEs support this
> >> >extension, including Github itself)
> >> >2. It's not aligned to widespread convention of naming template files
> >> >with
> >> >.jinja2 extension
> >> >3. It's harder to distinguish between template files and non-template
> >> >files
> >> >
> >> >A better way to do it would be supporting the .jinja2 extension.
> >> >
> >> >The proposition at the moment is to globally accept .jinja2 extension
> >> >in
> >> >`BaseOperator` whenever `template_ext` is defined. It means that all
> >> >inheriting operators will now automatically support their already
> >> >defined
> >> >extensions but also with .jinja2 appended to them (i.e. if you have
> >> >`template_ext = ('.json', '.yml')` then also .json.jinja2 and
> >> >.yml.jinja2
> >> >will be added)).
> >> >
> >> >Since this affects a vast part of the codebase, this suggestion is open
> >> >to
> >> >discussion. WDYT?
> >> >
> >> >For more info, see https://github.com/apache/airflow/issues/8603
> >>
> >
>
