shunping commented on code in PR #33384:
URL: https://github.com/apache/beam/pull/33384#discussion_r1907724615
##########
sdks/python/apache_beam/io/gcp/gcsio.py:
##########
@@ -589,8 +589,29 @@ def __init__(
blob,
chunk_size=DEFAULT_READ_BUFFER_SIZE,
enable_read_bucket_metric=False,
- retry=DEFAULT_RETRY):
- super().__init__(blob, chunk_size=chunk_size, retry=retry)
+ retry=DEFAULT_RETRY,
+ raw_download=True):
+ # By default, we always request to retrieve raw data from GCS even if the
+ # object meets the criteria of decompressive transcoding
+ # (https://cloud.google.com/storage/docs/transcoding).
+ super().__init__(
+ blob, chunk_size=chunk_size, retry=retry, raw_download=raw_download)
Review Comment:
Good question and that is a valid concern.
The `raw_download` is included as one of the valid download parameters
within the kwargs:
-
https://github.com/googleapis/python-storage/blob/ca998db2db7c7b028f6fc145f1cc6b8b2c2a967b/google/cloud/storage/fileio.py#L108
-
https://github.com/googleapis/python-storage/blob/ca998db2db7c7b028f6fc145f1cc6b8b2c2a967b/google/cloud/storage/fileio.py#L38
Let me ask the devs of the GCS lib about what is their plan for that API.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
