shahar1 commented on issue #29115: URL: https://github.com/apache/airflow/issues/29115#issuecomment-1528809010
> Due to name of argument as `delimiter`, description and code of Operator and [google-cloud-storage](https://cloud.google.com/python/docs/reference/storage/latest/buckets#listblobsmaxresultsnone-pagetokennone-prefixnone-delimiternone-startoffsetnone-endoffsetnone-includetrailingdelimiternone-versionsnone-projectionnoacl-fieldsnone-clientnone-timeout60-retrygoogleapicoreretryretry-object) I thought it work correctly. However have no idea who actually need this functionality. > > I thought better implement new argument with proper implementation and with name `suffix` Based on the following resources: [1] https://cloud.google.com/storage/docs/json_api/v1/objects/list?apix_params=%7B%22bucket%22%3A%22reproducing_issue_11874%22%2C%22delimiter%22%3A%22.PNG_2%22%7D#try-it [2] https://stackoverflow.com/questions/44702835/google-cloud-storage-ignoring-delimiter It seems that treating `delimiter` as `suffix` is a misuse, and filtering according to suffixes shouldn't be done as part of the API call to GCS but as a separate logic within the hook by simple list filtering. Therefore, I suggest deprecating the `delimiter` parameter (as it should almost always be `"/") and adding a `suffix` parameter instead for use as a filter, as follows: If `delimiter` is set and `suffix` is not already provided, warn that this parameter is deprecated and assign `self. Suffix` its value (), while `self.delimiter` will be set to `"/"`. This usage of the `delimiter` parameter is incorrect, but it won't break if already used in this context. 2. If `delimiter` is set, warn that this parameter is deprecated and completely ignore its value. This may break existing usages of `delimiter` in this context. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
