jscheffl commented on PR #64068: URL: https://github.com/apache/airflow/pull/64068#issuecomment-4113207750
> > > @jscheffl Why not push XCom’s directly on task_instance of trigger? Why? Because now the XCom’s are passed via the EventTrigger which is fully persisted in DB. I would push them directly via task_instance on trigger, it won’t be persisted with the event in the DB, which can grow easily with XCom’s. > > > > > > Does not work, the `self.task_instance` in the triggerer execution is from type `airflow.executors.workloads.TaskInstance` and this is a minimal stup with no `xcom_push()` implementation. > > Also a manual XCom push is not working as in the triggerer code there is no DB session. > > So, yes this proposal here has the trade-off that XCom is serialized once in the event and then another time from the event back to XCom. > > I could if this [PR](https://github.com/apache/airflow/pull/55068) would get merged, we would just need to always convert the `airflow.executors.workloads.TaskInstance` to the `RuntimeTaskInstance` instead of only when start_from_trigger is enabled. > > Here is the code that handles it: https://github.com/apache/airflow/pull/55068/changes#diff-e4cc497f1c786d142ce4c930f43e33b0bb4b53d375d274278fa82f4d5567608aR1005 Oh, ah, yeah! That would make it easier. Whereas also respecting the comments from @uranusjr the integration of a custom XCom backend also is limited in Triggerer? Because usually no XCom backend implements IO in async... so I am not sure besides the other PR proposed this is also adding a hairball of complexity to "just push XCom" from Triggerer :-( -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
