potiuk commented on code in PR #28900:
URL: https://github.com/apache/airflow/pull/28900#discussion_r1308430456
##########
airflow/models/dag.py:
##########
@@ -1373,6 +1374,40 @@ def normalized_schedule_interval(self) ->
ScheduleInterval:
_schedule_interval = self.schedule_interval
return _schedule_interval
+ @staticmethod
+ @internal_api_call
+ @provide_session
+ def fetch_callback(
+ dag: DAG,
+ dagrun: DagRun | DagRunPydantic,
Review Comment:
I think this is the place where we have a little trouble, but the solution
is easy. We are not supposed to send `DagRun | DagRunPydantic` classes from
client to server. What we are supposed to do is to replace it with "dag_run_id:
str" and then locally get the dagrun as usual from SQLalchemy session.
This is very efficient when it comes to "db" mode. If the object is already
present in the session, it should just be returned without any DB operations
involved (so we will have the same DagRun object that the caller has). In case
of the DB-mode, it will be, on-the-other hand retrieved from the DB as expected.
The only thing we have to make sure about (and this is something when we
verify when we run tests in db-less mode) is to make sure that this DagRun
object in the caller has not been modified since it was retrieved - if it was
then such a modification (in DagFileProcessor or Triggerer) should be replaced
with an internal_api method call to do such a modification. I think those cases
will be extremely rare though (if at all). Most of such modifications happens
in Scheduler, not DagFileProcessor and Triggerer. Only LocalTaskRunner does
effectively modify DagRun and those are things that we will have to carefully
wrap in an internal_api_call method.
I think this is something that we should realise (and maybe even enforce) -
while we can return DagRun | DagRunPydantic (and friends) from `internal_api`
decorated method - we should never pass then as an argument. We should only
pass "primitive" values like strings and regular objects (like DAG) - but not
potentially ORM models.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]