justinpakzad commented on PR #63665: URL: https://github.com/apache/airflow/pull/63665#issuecomment-4148200474
I've update the PR. Just wanted to leave a couple of notes on the approach. The first call uses the users (or default) `offset` and `limit` to fetch the data for the response and the total entries. Then we loop through all entries and get the relevant dag ids that need to be updated. I used the limit as the batch size here (for fetching) as it made the most sense (rather than just specifying a fixed number (e.g, 500)) but happy to change that if there is a preferred or optimal batch size. The reason we iterate over all the dags in the loop instead of using the results from the first call is that the caller's offset may not be 0, so we can't guarantee the first page covers the start of the dataset. I ran some tests, pausing/unpausing 100+ dags and it's working as expected. I added to the docstring to explain the wildcard pattern and created additional tests cases to test matching any tags, multiple tags, and multiple dags patched. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
