uranusjr commented on PR #71072: URL: https://github.com/apache/airflow/pull/71072#issuecomment-6762182206
My remaining concern is where the check runs. Right now it’s inside the scheduler loop, and while the backlog stays above the cap (which can be a steady state, as you said), every loop runs an extra GROUP BY count over all pending APDRs. Would it work to move the count and the audit write into a periodic callback instead? The callback gets its own session, so we also wouldn’t need the separate audit session or the empty/short-fetch handling. Rows would show up a bit later, but that seems fine for this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
