potiuk commented on code in PR #66820:
URL: https://github.com/apache/airflow/pull/66820#discussion_r3257829420
##########
airflow-core/src/airflow/jobs/scheduler_job_runner.py:
##########
@@ -1776,6 +1776,15 @@ def _do_scheduling(self, session: Session) -> int:
self._start_queued_dagruns(session)
guard.commit()
+ # Clear DagRun objects loaded by phase 1 from the identity map so
+ # phase 2 reloads them fresh. Otherwise stale rows can be
re-dirtied
+ # by flush/merge in _schedule_all_dag_runs and committed in a
row-lock
+ # order that differs from what other scheduler replicas are taking
+ # for their own work, producing A-B / B-A deadlocks on dag_run and
+ # task_instance under HA scheduler deployments. See
+ # https://github.com/apache/airflow/issues/66817.
+ session.expunge_all()
Review Comment:
> Hope that helps ease the concern. Thanks for raising it directly — easier
to address that way than not.
Hello @1fanwang
Let me also re-raise the concern of @ephraimbuddy and state it more clearly
(and I already wrote it elsewhere). We **absolutely** do not need issues
created in order to accept PRs. If you see any issues in Airflow on your
deployment and have fixes to propose - Just open the PR. Just adjust to our
rules.
While I understand in some corporate environments you create issues to do
some kind of accounting, for us this is absolutely unnecessary noise. We do not
need issues if we know PR with the fix is coming.
Also - I am quite sure you are using Agents to do it (which is not a bad
thing - I am using agents for nearly everything now). If you have a good agent,
that's cool. Unfortumate creating issues invites other people with
not-such-good experience as you, and not-such-good agents you seem to have -
and it might result in a flood of PRs that will be implementing issues you
opened, even if all you wanted to do is to plan your work.
So: if you have a fix - just open PRs explaining the issue with the fix. And
you can also tell your agent to do it - if for some reason it did not take
seriously what we wrote in the contributing guide:
See here:
https://github.com/apache/airflow/blob/main/contributing-docs/04_how_to_contribute.rst#issue-reporting-and-resolution-process
So we are absolutely not against big number of good PRs fixing real issues.
Just ... don't create issues for those please :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
