We could do this globally for 1.10, but for 2.0 I think it might be nice
to change this as follows:
Rename template_ext to file_exts, and it looks for files with those
extensions, but doesn't render them.
And then make it only render files with, for example, .sh.jinja2.
Right now it's impossible to have scripts/files external to the DAG that
aren't rendered, so you have to be careful to escape {{ in them, which
can be annoying.
WDYT?
-a
On Apr 29 2020, at 12:17 pm, Ori Popowski <ori....@gmail.com> wrote:
> No. Any extension mentioned in `template_ext` will still be templated. But
> in addition, also those ending with `.jinja2`, as long it's preceded
> by one
> of those extensions (meaning that `file.jinja2` won't be templated)
>
> On Wed, Apr 29, 2020 at 1:41 PM Ash Berlin-Taylor <a...@apache.org> wrote:
>
>> Are you also proposing that, for example .sh would not be templated, but
>> that .sh.jinja2 would be templated?
>>
>> Ash
>>
>> On 29 April 2020 11:39:13 BST, Ori Popowski <ori....@gmail.com> wrote:
>> >At the moment, various operators support reading templates from given
>> >filenames (for example, `bash_command` of `BashOperator`). However, all
>> >of
>> >these operators specify extensions such as .sh, .json etc., so it means
>> >users need to write Jinja2 templates inside .sh, .json files. This is
>> >problematic for 3 reasons:
>> >
>> >1. No semantic support when writing templates (major IDEs support this
>> >extension, including Github itself)
>> >2. It's not aligned to widespread convention of naming template files
>> >with
>> >.jinja2 extension
>> >3. It's harder to distinguish between template files and non-template
>> >files
>> >
>> >A better way to do it would be supporting the .jinja2 extension.
>> >
>> >The proposition at the moment is to globally accept .jinja2 extension
>> >in
>> >`BaseOperator` whenever `template_ext` is defined. It means that all
>> >inheriting operators will now automatically support their already
>> >defined
>> >extensions but also with .jinja2 appended to them (i.e. if you have
>> >`template_ext = ('.json', '.yml')` then also .json.jinja2 and
>> >.yml.jinja2
>> >will be added)).
>> >
>> >Since this affects a vast part of the codebase, this suggestion is open
>> >to
>> >discussion. WDYT?
>> >
>> >For more info, see https://github.com/apache/airflow/issues/8603
>>
>
