shahar1 commented on issue #29115:
URL: https://github.com/apache/airflow/issues/29115#issuecomment-1528808991

   > Due to name of argument as `delimiter`, description and code of Operator 
and 
[google-cloud-storage](https://cloud.google.com/python/docs/reference/storage/latest/buckets#listblobsmaxresultsnone-pagetokennone-prefixnone-delimiternone-startoffsetnone-endoffsetnone-includetrailingdelimiternone-versionsnone-projectionnoacl-fieldsnone-clientnone-timeout60-retrygoogleapicoreretryretry-object)
 I thought it work correctly. However have no idea who actually need this 
functionality.
   > 
   > I thought better implement new argument with proper implementation and 
with name `suffix`
   
   Based on the following resources:
   [1] 
https://cloud.google.com/storage/docs/json_api/v1/objects/list?apix_params=%7B%22bucket%22%3A%22reproducing_issue_11874%22%2C%22delimiter%22%3A%22.PNG_2%22%7D#try-it
   [2] 
https://stackoverflow.com/questions/44702835/google-cloud-storage-ignoring-delimiter
   
   
   It seems that treating `delimiter` as `suffix` is a misuse, and filtering 
according to suffixes shouldn't be done as part of the API call to GCS but as a 
separate logic within the hook by simple list filtering.
   Therefore, I suggest deprecating the `delimiter` parameter (as it should 
almost always be `"/") and adding a `suffix` parameter instead for use as a 
filter, as follows:
   If `delimiter` is set and `suffix` is not already provided, warn that 
`delimiter` is deprecated and assign `self.suffix` its value, while 
`self.delimiter` will be set to a constant `"/"`. If `suffix` is provided, it 
will behave as expected regardless of the value of `delimiter`.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to