kremogen commented on PR #65839: URL: https://github.com/apache/airflow/pull/65839#issuecomment-4334786316
> I don't think we should add another endpoint for this. > > It's using a token on purpose. That token is retrieved from the dag API and is a 'file token'. Based on the bundle and file loc. So basically retrieve (GET) on the dag api, will return the token that you can then use to trigger a new reparse of that file containing the dag. > > Dag parser doesn't parse a 'dag' but a file. So basically triggering a reparse for 'dag x' will trigger a parse for the file associated, which can have multiple dags in there. (multiple dag per file). > > A 'per dag' vision isn't correct IMO. Hey, thanks for the review! A few thoughts on the motivation behind this: DAG reparsing is a fairly expensive operation, and running it every 30 seconds generates significant load when you have 1000+ DAGs. On the other hand, increasing the reparse interval introduces a noticeable delay between deployment and the DAG actually showing up in Airflow, which directly increases TTM. Allowing teams to trigger reparsing on demand means they can wire it into their CI/CD pipelines and shorten the feedback loop between development iterations considerably. As a follow-up, it would be great to expose this through the CLI as well. Happy to make any changes if needed - let me know! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
