jscheffl closed pull request #38432: Use Task Output as Extra for Dataset
Trigger and DAG Run Conf
URL: https://github.com/apache/airflow/pull/38432
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to t
jscheffl commented on PR #38432:
URL: https://github.com/apache/airflow/pull/38432#issuecomment-2024093337
> I gave this a longer thought and I think I’m not particularly fond of the
approach mainly because it mixes `Dataset.extra` and `DatasetEvent.extra`. As
described in #37810, the curre
jscheffl commented on code in PR #38432:
URL: https://github.com/apache/airflow/pull/38432#discussion_r1542109168
##
airflow/models/taskinstance.py:
##
@@ -2455,7 +2456,7 @@ def _run_raw_task(
try:
if not mark_success:
-self._e
jscheffl commented on code in PR #38432:
URL: https://github.com/apache/airflow/pull/38432#discussion_r1542107183
##
airflow/models/taskinstance.py:
##
@@ -2553,21 +2554,30 @@ def _run_raw_task(
return None
-def _register_dataset_changes(self, *, session: Se
jscheffl commented on code in PR #38432:
URL: https://github.com/apache/airflow/pull/38432#discussion_r1542106481
##
airflow/models/taskinstance.py:
##
@@ -2553,21 +2554,30 @@ def _run_raw_task(
return None
-def _register_dataset_changes(self, *, session: Se
uranusjr commented on PR #38432:
URL: https://github.com/apache/airflow/pull/38432#issuecomment-2022100403
I gave this a longer thought and I think I’m not particularly fond of the
approach mainly because it mixes `Dataset.extra` and `DatasetEvent.extra`. As
described in #37810, the current
shrukul commented on code in PR #38432:
URL: https://github.com/apache/airflow/pull/38432#discussion_r1539796806
##
airflow/models/taskinstance.py:
##
@@ -2455,7 +2456,7 @@ def _run_raw_task(
try:
if not mark_success:
-self._ex
uranusjr commented on PR #38432:
URL: https://github.com/apache/airflow/pull/38432#issuecomment-2017487671
I also don’t quite feel comfortable the return value is still pushed to XCom
if it is used as event extra. IMO it should go either one or the other, not
both.
--
This is an automate
uranusjr commented on code in PR #38432:
URL: https://github.com/apache/airflow/pull/38432#discussion_r1537220837
##
airflow/jobs/scheduler_job_runner.py:
##
@@ -1282,9 +1282,15 @@ def _create_dag_runs_dataset_triggered(
events=dataset_events,
uranusjr commented on code in PR #38432:
URL: https://github.com/apache/airflow/pull/38432#discussion_r1537218134
##
airflow/models/taskinstance.py:
##
@@ -2553,21 +2554,30 @@ def _run_raw_task(
return None
-def _register_dataset_changes(self, *, session: Se
uranusjr commented on code in PR #38432:
URL: https://github.com/apache/airflow/pull/38432#discussion_r1537214822
##
airflow/models/taskinstance.py:
##
@@ -2553,21 +2554,30 @@ def _run_raw_task(
return None
-def _register_dataset_changes(self, *, session: Se
dirrao commented on code in PR #38432:
URL: https://github.com/apache/airflow/pull/38432#discussion_r1537095769
##
airflow/models/taskinstance.py:
##
@@ -2553,21 +2554,30 @@ def _run_raw_task(
return None
-def _register_dataset_changes(self, *, session: Sess
jscheffl opened a new pull request, #38432:
URL: https://github.com/apache/airflow/pull/38432
This PR is a proposal to fix/resolve the request for feature as described in
#37810 as a thin approach.
Idea for this solution:
- Allow using task result as dynamic `extra`
- If d
13 matches
Mail list logo
