That skip func had a typo (conf where it should have been context)...
this is more likely to work:
def skip_if_specified(context):
if not context:
return
dr = context.get('dag_run')
ti = context.get('task_instance')
if not (dr and ti):
return
conf = dr.conf
Ok and here's an example you could try:
The callable:
def skip_if_specified(context):
dr = context.get('dag_run')
if not dr:
return
conf = dr.conf
if not conf:
return
skip_list = conf.get('skip_list')
if not skip_list:
return
ti = conf.get('task
Sorry in this case I would be NOT in favor of pep8. I misread it.
-feluelle
Sent from ProtonMail for iOS
On Fri, Feb 4, 2022 at 07:22, Felix Uellendall wrote:
> I am in favor of pep8 guidelines. I think it makes sense to clearly see
> separation between words/tokens. For me uppercase is harde
I am in favor of pep8 guidelines. I think it makes sense to clearly see
separation between words/tokens. For me uppercase is harder to read and I don’t
like getting screamed at. As long as there is documentation, this can be
written there correctly as you can split it like normal words (by space
And if not with cluster policy, then you could pass such a "conditionally
skip" callable to all tasks in a dag with default_args
On Thu, Feb 3, 2022, 10:16 PM Daniel Standish
wrote:
> It wouldn't necessarily need to involve the scheduler or the executor.
> You could add logic to pre_execute to r
It wouldn't necessarily need to involve the scheduler or the executor. You
could add logic to pre_execute to read dag run conf and skip self is
specified in the conf. In fact you could probably implement this currently
with this approach using cluster policy, since we can supply pre_execute
calla
Hello everyone,
Just want to clarify a few points regarding "specify tasks to skip when
triggering DAG".
First of all, the use case is not necessarily only backfill. We may want to
have this for manual triggers so users can do it easily in UI, rather than
using backfill CLI. This is similar to ho
Hi
I would like to call a vote on the following proposal:
Helper function `days_ago` is to be deprecated (warning of pending removal
with each call) with removal targeted for 3.0. The deprecation warning
should guide user and perhaps point to some documentation concerning how to
maintain the equ
How should we name, for example, an operator such as `BranchSQLOperator`?
This operator used to be called `BranchSqlOperator` but at some point in
the past was renamed.
Meanwhile we have SparkSqlOperator which uses the other convention.
And we have `EmrBaseSensor`, and `EMRContainerOperator` coe
Thanks Jarek, I'm happy I could help
On Thu, Feb 3, 2022 at 5:07 PM Jarek Potiuk wrote:
> Actually - maybe even bring it to the state in the 3rd decade of
> the century ;)
>
> On Thu, Feb 3, 2022 at 5:05 PM Jarek Potiuk wrote:
>
>> Hello everyone,
>>
>> Just to give some information on the prog
Actually - maybe even bring it to the state in the 3rd decade of
the century ;)
On Thu, Feb 3, 2022 at 5:05 PM Jarek Potiuk wrote:
> Hello everyone,
>
> Just to give some information on the progress and plans in the
> Open-Telemetry area.
>
> I just had a talk with Howard, and we are going to wo
Hello everyone,
Just to give some information on the progress and plans in the
Open-Telemetry area.
I just had a talk with Howard, and we are going to work together on the AIP
proposal on "whys", "hows", and "whats" of the Open-Telemetry for Airflow.
We have enough information already from the P
12 matches
Mail list logo
