Re: [PR] iterate through blobs before checking prefixes [airflow]

2023-12-15 Thread via GitHub
shahar1 commented on PR #36202: URL: https://github.com/apache/airflow/pull/36202#issuecomment-1857712133 Looks great! Thanks :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [PR] iterate through blobs before checking prefixes [airflow]

2023-12-14 Thread via GitHub
potiuk commented on PR #36202: URL: https://github.com/apache/airflow/pull/36202#issuecomment-1855973268 Just in time for RC3 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] iterate through blobs before checking prefixes [airflow]

2023-12-14 Thread via GitHub
potiuk merged PR #36202: URL: https://github.com/apache/airflow/pull/36202 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] iterate through blobs before checking prefixes [airflow]

2023-12-13 Thread via GitHub
Lee-W commented on PR #36202: URL: https://github.com/apache/airflow/pull/36202#issuecomment-1853827512 This test https://github.com/apache/airflow/pull/36202/files#diff-76fef248bc8b2046dae5e6e1fe60dcb916e4114d85190c563454d2d123937b43R829 mainly checks the prefixes is actually used if exist

Re: [PR] iterate through blobs before checking prefixes [airflow]

2023-12-13 Thread via GitHub
phanikumv commented on PR #36202: URL: https://github.com/apache/airflow/pull/36202#issuecomment-1853815188 This PR will potentially fix the error on the google provider rc. Could you kindly include it in the next rc for google provider. cc @potiuk -- This is an automated

Re: [PR] iterate through blobs before checking prefixes [airflow]

2023-12-13 Thread via GitHub
atrbgithub commented on code in PR #36202: URL: https://github.com/apache/airflow/pull/36202#discussion_r1425254462 ## airflow/providers/google/cloud/hooks/gcs.py: ## @@ -933,16 +934,17 @@ def list_by_timespan( delimiter=delimiter, versi

[PR] iterate through blobs before checking prefixes [airflow]

2023-12-13 Thread via GitHub
Lee-W opened a new pull request, #36202: URL: https://github.com/apache/airflow/pull/36202 According to https://github.com/googleapis/python-storage/blob/v2.14.0/google/cloud/storage/client.py#L1213-L1217, the prefixes are not returned until the blobs are consumed