hkc-8010 commented on PR #66854: URL: https://github.com/apache/airflow/pull/66854#issuecomment-5044732007
@uranusjr following up on the queue design we landed on in Slack. After @jason810496's review I simplified this to be queue-only: the execution API commits a durable `asset_event_queue` marker in the same transaction as the task-success state, and the scheduler drain is now the single writer that runs `register_asset_changes_in_db`. That drops the inline registration fast-path I had added on the request path, so there is no asset registration inside the state-update request at all now, which is the cleanest form of the scheduler-drained queue we discussed. The tradeoff is that a task's asset events (and the downstream asset-triggered runs) become visible after a drain interval instead of immediately, and `dag.test` does not register them at all since it has no scheduler. Both are documented in the newsfragment and the asset-scheduling note. Wanted to confirm you are good dropping the inline path before I treat the direction as settled. @jason810496 thanks for pushing on this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
