Re: [PR] Support multiple XCom output in the BaseOperator [airflow]

2024-07-26 Thread via GitHub
jannisko commented on PR #37297: URL: https://github.com/apache/airflow/pull/37297#issuecomment-2252886312 @joycechan-KSO yes this feature was available for the [TaskFlow](https://airflow.apache.org/docs/apache-airflow/stable/tutorial/taskflow.html) style operators already, but missing for

Re: [PR] Support multiple XCom output in the BaseOperator [airflow]

2024-07-25 Thread via GitHub
joycechan-KSO commented on PR #37297: URL: https://github.com/apache/airflow/pull/37297#issuecomment-2250333988 Hi, I think this feature been added by airflow 2.0 + version ![image](https://github.com/user-attachments/assets/3873f6cf-f7be-4eaa-92ff-7e19aa39c790)

Re: [PR] Support multiple XCom output in the BaseOperator [airflow]

2024-02-12 Thread via GitHub
hussein-awala merged PR #37297: URL: https://github.com/apache/airflow/pull/37297 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Support multiple XCom output in the BaseOperator [airflow]

2024-02-11 Thread via GitHub
hussein-awala commented on code in PR #37297: URL: https://github.com/apache/airflow/pull/37297#discussion_r1485695577 ## airflow/providers/sftp/decorators/sensors/sftp.py: ## @@ -52,7 +52,6 @@ def __init__( task_id: str, **kwargs, ) -> None: -

Re: [PR] Support multiple XCom output in the BaseOperator [airflow]

2024-02-11 Thread via GitHub
Taragolis commented on code in PR #37297: URL: https://github.com/apache/airflow/pull/37297#discussion_r1485684905 ## airflow/providers/sftp/decorators/sensors/sftp.py: ## @@ -52,7 +52,6 @@ def __init__( task_id: str, **kwargs, ) -> None: -

Re: [PR] Support multiple XCom output in the BaseOperator [airflow]

2024-02-10 Thread via GitHub
potiuk commented on PR #37297: URL: https://github.com/apache/airflow/pull/37297#issuecomment-1937064243 > I didn't actually notice that there was type-hint based inferral until now  > I feel like this feature is too niche to warrant introducing a breaking change to all operators

Re: [PR] Support multiple XCom output in the BaseOperator [airflow]

2024-02-10 Thread via GitHub
jannisko commented on PR #37297: URL: https://github.com/apache/airflow/pull/37297#issuecomment-1937048094 > Yes, we can do that. I wonder if we need to duplicate all the behaviors, for example, inferring `multiple_outputs` (#20800) when the execute method has a mapping as the returned

Re: [PR] Support multiple XCom output in the BaseOperator [airflow]

2024-02-10 Thread via GitHub
hussein-awala commented on PR #37297: URL: https://github.com/apache/airflow/pull/37297#issuecomment-1937021429 > I'm not sure if it makes sense to recreate this behavior. Yes, we can do that. I wonder if we need to duplicate all the behaviors, for example, inferring

Re: [PR] Support multiple XCom output in the BaseOperator [airflow]

2024-02-10 Thread via GitHub
jannisko commented on PR #37297: URL: https://github.com/apache/airflow/pull/37297#issuecomment-1936972290 The implementation for the taskflow decorator seems to return the xcoms separately (with different keys) **and** in its original form with the default key:

Re: [PR] Support multiple XCom output in the BaseOperator [airflow]

2024-02-10 Thread via GitHub
rawwar commented on code in PR #37297: URL: https://github.com/apache/airflow/pull/37297#discussion_r1485049460 ## airflow/models/taskinstance.py: ## @@ -450,7 +450,16 @@ def _execute_callable(context, **execute_callable_kwargs): else: xcom_value = None

Re: [PR] Support multiple XCom output in the BaseOperator [airflow]

2024-02-10 Thread via GitHub
potiuk commented on code in PR #37297: URL: https://github.com/apache/airflow/pull/37297#discussion_r1485043421 ## airflow/models/taskinstance.py: ## @@ -450,7 +450,16 @@ def _execute_callable(context, **execute_callable_kwargs): else: xcom_value = None

Re: [PR] Support multiple XCom output in the BaseOperator [airflow]

2024-02-10 Thread via GitHub
jannisko commented on PR #37297: URL: https://github.com/apache/airflow/pull/37297#issuecomment-1936935927 Thanks for jumping on this request! @hussein-awala I was kind of hoping to contribute myself but I guess I was too slow  -- This is an automated message from the Apache Git

Re: [PR] Support multiple XCom output in the BaseOperator [airflow]

2024-02-10 Thread via GitHub
jannisko commented on code in PR #37297: URL: https://github.com/apache/airflow/pull/37297#discussion_r1485018215 ## airflow/models/taskinstance.py: ## @@ -450,7 +450,16 @@ def _execute_callable(context, **execute_callable_kwargs): else: xcom_value = None

Re: [PR] Support multiple XCom output in the BaseOperator [airflow]

2024-02-10 Thread via GitHub
jannisko commented on code in PR #37297: URL: https://github.com/apache/airflow/pull/37297#discussion_r1485018215 ## airflow/models/taskinstance.py: ## @@ -450,7 +450,16 @@ def _execute_callable(context, **execute_callable_kwargs): else: xcom_value = None

Re: [PR] Support multiple XCom output in the BaseOperator [airflow]

2024-02-09 Thread via GitHub
rawwar commented on code in PR #37297: URL: https://github.com/apache/airflow/pull/37297#discussion_r1484897715 ## airflow/models/taskinstance.py: ## @@ -450,7 +450,16 @@ def _execute_callable(context, **execute_callable_kwargs): else: xcom_value = None

Re: [PR] Support multiple XCom output in the BaseOperator [airflow]

2024-02-09 Thread via GitHub
rawwar commented on code in PR #37297: URL: https://github.com/apache/airflow/pull/37297#discussion_r1484897715 ## airflow/models/taskinstance.py: ## @@ -450,7 +450,16 @@ def _execute_callable(context, **execute_callable_kwargs): else: xcom_value = None

Re: [PR] Support multiple XCom output in the BaseOperator [airflow]

2024-02-09 Thread via GitHub
rawwar commented on code in PR #37297: URL: https://github.com/apache/airflow/pull/37297#discussion_r1484897249 ## airflow/models/taskinstance.py: ## @@ -450,7 +450,16 @@ def _execute_callable(context, **execute_callable_kwargs): else: xcom_value = None

[PR] Support multiple XCom output in the BaseOperator [airflow]

2024-02-09 Thread via GitHub
hussein-awala opened a new pull request, #37297: URL: https://github.com/apache/airflow/pull/37297 closes: #37284 This PR adds a new argument `multiple_outputs ` to BaseOperator to push multiple XCom when it set to True instead of pushing the result to `return_value` XCom. --