nkemnitz commented on code in PR #782:
URL:
https://github.com/apache/arrow-rs-object-store/pull/782#discussion_r3641517697
##########
src/client/header.rs:
##########
@@ -139,8 +147,12 @@ pub(crate) fn header_meta(
Err(e) => return Err(e),
};
+ // Prefer `Content-Length`, falling back to a store-provided size header:
GCS omits
Review Comment:
The description for what `x-goog-stored-content-length` contains:
https://docs.cloud.google.com/storage/docs/xml-api/reference-headers#xgoogstoredcontentlength
In the Google Cloud Rust repo, a code maintainer reports that this header is
indeed always present:
https://github.com/googleapis/google-cloud-rust/discussions/3713#discussioncomment-14883469
The part that confirms that `Content-Length` and `Content-Encoding` get
dropped for decompressive transcoding:
https://docs.cloud.google.com/storage/docs/transcoding#decompressive_transcoding
(Although I don't really care about decompressive transcoding - it creates
large transfers and as described under the same link in the paragraph below, it
prevents integrity checks)
I could not find any official documentation regarding the chunked
Transfer-Encoding switch... only that it does... Could weaken that claim to
"Google _may_ omit the `Content-Length`".
Other finds:
*
[google-cloud-cpp](https://github.com/googleapis/google-cloud-cpp/blob/95511adfa1a6e733ca20dd17aaefd898611fe2f5/google/cloud/storage/internal/rest/object_read_source.cc#L47-L68)
prefers `x-goog-stored-content-length` > `content-range` > `content-length`
*
[google-cloud-rust](https://github.com/googleapis/google-cloud-rust/blob/18cd4258a67b424a814d29c4b63fad501952c43d/src/storage/src/storage/read_object/parse_http_response.rs#L42)
only uses x-goog-stored-content-length and nothing else?
* Their [python
code](https://github.com/googleapis/python-storage/blob/ab4997ce0f7b85947e84b226bd0edf6d714a946a/google/cloud/storage/_media/requests/download.py#L163)
seems to use it to distinguish truncated downloads from transcoded ones
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
