tirkarthi commented on PR #38674:
URL: https://github.com/apache/airflow/pull/38674#issuecomment-2040960545
Related :
https://github.com/apache/airflow/pull/31808
https://github.com/apache/airflow/issues/31718
--
This is an automated message from the Apache Git Service.
To respo
uranusjr commented on PR #38674:
URL: https://github.com/apache/airflow/pull/38674#issuecomment-2042181174
I wonder if we could somehow infer `starts_execution_from_triggerer` from
the combination of other arguments, instead of setting the flag explicitly.
--
This is an automated message
Lee-W commented on PR #38674:
URL: https://github.com/apache/airflow/pull/38674#issuecomment-2042328464
> I wonder if we could somehow infer `starts_execution_from_triggerer` from
the combination of other arguments, instead of setting the flag explicitly.
For this specific case, I wou
uranusjr commented on PR #38674:
URL: https://github.com/apache/airflow/pull/38674#issuecomment-2044338249
If we also infer `starts_execution_from_triggerer` from it, `start_trigger`
is not so bad to me, and actually better than `trigger` because it more clearly
describes the attributes con
Lee-W commented on PR #38674:
URL: https://github.com/apache/airflow/pull/38674#issuecomment-2047292509
I have encountered a [has-type] mypy issue. Has anyone faced something
similar before and have any suggestions? Thank you.
https://github.com/apache/airflow/assets/5144808/bf5a22d1-
Lee-W commented on PR #38674:
URL: https://github.com/apache/airflow/pull/38674#issuecomment-2047289208
In the current design, I introduced `start_trigger` and `next_method` to the
`BaseOperator`. When `start_trigger` is not `None`, the scheduler will call the
`defer_task` method of the tas
uranusjr commented on code in PR #38674:
URL: https://github.com/apache/airflow/pull/38674#discussion_r1571829890
##
airflow/decorators/base.py:
##
@@ -509,6 +509,9 @@ def _expand(self, expand_input: ExpandInput, *, strict:
bool) -> XComArg:
# task's expand() contr
uranusjr commented on code in PR #38674:
URL: https://github.com/apache/airflow/pull/38674#discussion_r1571831966
##
airflow/models/baseoperator.py:
##
@@ -1691,6 +1693,16 @@ def inherits_from_empty_operator(self):
# of its subclasses (which don't inherit from anything
uranusjr commented on code in PR #38674:
URL: https://github.com/apache/airflow/pull/38674#discussion_r1571831512
##
airflow/models/baseoperator.py:
##
@@ -1072,6 +1072,8 @@ def __init__(
if SetupTeardownContext.active:
SetupTeardownContext.update_context_m
uranusjr commented on code in PR #38674:
URL: https://github.com/apache/airflow/pull/38674#discussion_r1571832173
##
airflow/models/baseoperator.py:
##
@@ -1691,6 +1693,16 @@ def inherits_from_empty_operator(self):
# of its subclasses (which don't inherit from anything
Lee-W commented on code in PR #38674:
URL: https://github.com/apache/airflow/pull/38674#discussion_r1571987070
##
airflow/models/baseoperator.py:
##
@@ -1691,6 +1693,16 @@ def inherits_from_empty_operator(self):
# of its subclasses (which don't inherit from anything but
Lee-W commented on code in PR #38674:
URL: https://github.com/apache/airflow/pull/38674#discussion_r1571987975
##
airflow/models/baseoperator.py:
##
@@ -1691,6 +1693,16 @@ def inherits_from_empty_operator(self):
# of its subclasses (which don't inherit from anything but
Lee-W commented on code in PR #38674:
URL: https://github.com/apache/airflow/pull/38674#discussion_r1571990864
##
airflow/models/baseoperator.py:
##
@@ -1072,6 +1072,8 @@ def __init__(
if SetupTeardownContext.active:
SetupTeardownContext.update_context_map(
Lee-W commented on code in PR #38674:
URL: https://github.com/apache/airflow/pull/38674#discussion_r1572048134
##
airflow/decorators/base.py:
##
@@ -509,6 +509,9 @@ def _expand(self, expand_input: ExpandInput, *, strict:
bool) -> XComArg:
# task's expand() contribu
eladkal commented on PR #38674:
URL: https://github.com/apache/airflow/pull/38674#issuecomment-2066199167
> For some operators such as S3KeySensor with deferrable set to True,
running execute method in workers might not be necessary.
How would that work in case user uses `soft_fail`?
uranusjr commented on code in PR #38674:
URL: https://github.com/apache/airflow/pull/38674#discussion_r1572203632
##
airflow/models/baseoperator.py:
##
@@ -1072,6 +1072,8 @@ def __init__(
if SetupTeardownContext.active:
SetupTeardownContext.update_context_m
uranusjr commented on code in PR #38674:
URL: https://github.com/apache/airflow/pull/38674#discussion_r1572203632
##
airflow/models/baseoperator.py:
##
@@ -1072,6 +1072,8 @@ def __init__(
if SetupTeardownContext.active:
SetupTeardownContext.update_context_m
Lee-W commented on code in PR #38674:
URL: https://github.com/apache/airflow/pull/38674#discussion_r1572255021
##
airflow/models/baseoperator.py:
##
@@ -1072,6 +1072,8 @@ def __init__(
if SetupTeardownContext.active:
SetupTeardownContext.update_context_map(
Lee-W commented on PR #38674:
URL: https://github.com/apache/airflow/pull/38674#issuecomment-2066635766
> > For some operators such as S3KeySensor with deferrable set to True,
running execute method in workers might not be necessary.
>
> How would that work in case user uses `soft_fai
Lee-W commented on code in PR #38674:
URL: https://github.com/apache/airflow/pull/38674#discussion_r1572436018
##
airflow/decorators/base.py:
##
@@ -509,6 +509,9 @@ def _expand(self, expand_input: ExpandInput, *, strict:
bool) -> XComArg:
# task's expand() contribu
Lee-W commented on code in PR #38674:
URL: https://github.com/apache/airflow/pull/38674#discussion_r1574405076
##
airflow/decorators/base.py:
##
@@ -509,6 +509,9 @@ def _expand(self, expand_input: ExpandInput, *, strict:
bool) -> XComArg:
# task's expand() contribu
uranusjr commented on code in PR #38674:
URL: https://github.com/apache/airflow/pull/38674#discussion_r1575701802
##
airflow/models/baseoperator.py:
##
@@ -1693,6 +1698,16 @@ def inherits_from_empty_operator(self):
# of its subclasses (which don't inherit from anything
Lee-W commented on code in PR #38674:
URL: https://github.com/apache/airflow/pull/38674#discussion_r1575716908
##
airflow/models/baseoperator.py:
##
@@ -1693,6 +1698,16 @@ def inherits_from_empty_operator(self):
# of its subclasses (which don't inherit from anything but
Lee-W commented on code in PR #38674:
URL: https://github.com/apache/airflow/pull/38674#discussion_r1575716908
##
airflow/models/baseoperator.py:
##
@@ -1693,6 +1698,16 @@ def inherits_from_empty_operator(self):
# of its subclasses (which don't inherit from anything but
uranusjr commented on code in PR #38674:
URL: https://github.com/apache/airflow/pull/38674#discussion_r1575761706
##
airflow/models/baseoperator.py:
##
@@ -1693,6 +1698,16 @@ def inherits_from_empty_operator(self):
# of its subclasses (which don't inherit from anything
Lee-W commented on code in PR #38674:
URL: https://github.com/apache/airflow/pull/38674#discussion_r1575823438
##
airflow/models/baseoperator.py:
##
@@ -1693,6 +1698,16 @@ def inherits_from_empty_operator(self):
# of its subclasses (which don't inherit from anything but
Lee-W commented on code in PR #38674:
URL: https://github.com/apache/airflow/pull/38674#discussion_r1576293260
##
airflow/models/baseoperator.py:
##
@@ -1693,6 +1698,16 @@ def inherits_from_empty_operator(self):
# of its subclasses (which don't inherit from anything but
uranusjr commented on code in PR #38674:
URL: https://github.com/apache/airflow/pull/38674#discussion_r1577423825
##
docs/apache-airflow/authoring-and-scheduling/deferring.rst:
##
@@ -141,6 +141,60 @@ The ``self.defer`` call raises the ``TaskDeferred``
exception, so it can work
Lee-W commented on PR #38674:
URL: https://github.com/apache/airflow/pull/38674#issuecomment-2081989095
I'm planning on merging this PR later today. Please let me know if anyone
has concerns. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, p
ephraimbuddy commented on code in PR #38674:
URL: https://github.com/apache/airflow/pull/38674#discussion_r1582607222
##
airflow/serialization/serialized_objects.py:
##
@@ -1006,6 +1007,12 @@ def _serialize_node(cls, op: BaseOperator |
MappedOperator, include_deps: bool)
ephraimbuddy commented on code in PR #38674:
URL: https://github.com/apache/airflow/pull/38674#discussion_r1582607222
##
airflow/serialization/serialized_objects.py:
##
@@ -1006,6 +1007,12 @@ def _serialize_node(cls, op: BaseOperator |
MappedOperator, include_deps: bool)
Lee-W merged PR #38674:
URL: https://github.com/apache/airflow/pull/38674
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.ap
eladkal commented on code in PR #38674:
URL: https://github.com/apache/airflow/pull/38674#discussion_r1604504585
##
docs/apache-airflow/authoring-and-scheduling/deferring.rst:
##
@@ -141,6 +141,60 @@ The ``self.defer`` call raises the ``TaskDeferred``
exception, so it can work
Lee-W commented on code in PR #38674:
URL: https://github.com/apache/airflow/pull/38674#discussion_r1604969145
##
docs/apache-airflow/authoring-and-scheduling/deferring.rst:
##
@@ -141,6 +141,60 @@ The ``self.defer`` call raises the ``TaskDeferred``
exception, so it can work an
34 matches
Mail list logo
