URL:
<https://savannah.gnu.org/bugs/?60106>
Summary: wget mistakenly interprets Content-Range as bytes,
even with Range-Unit: items, and truncates content
Project: GNU Wget
Submitted by: afranke
Submitted on: Tue 23 Feb 2021 07:10:59 PM UTC
Category: Program Logic
Severity: 3 - Normal
Priority: 5 - Normal
Status: None
Privacy: Public
Assigned to: None
Originator Name:
Originator Email:
Open/Closed: Open
Release: 1.19.4
Discussion Lock: Any
Operating System: GNU/Linux
Reproducibility: Every Time
Fixed Release: None
Planned Release: None
Regression: No
Work Required: None
Patch Included: None
_______________________________________________________
Details:
When a server sends the following response headers:
Accept-Ranges: items
Range-Unit: items
Content-Range: 0-110/111
but *no* 'Content-Length' response header, then wget says
Length: 111 [...]
and truncates the response content after 111 bytes (or, when using https,
after a few KB), even though the full response content is much larger (e.g.
720 KB).
What's worse, --ignore-length doesn't help with this at all.
While a range unit of "bytes" is the most common case, IMO wget should not
assume a range unit of "bytes" when the 'Range-Unit' header says something
else.
To aggravate the problem even further, wget then retries with a range request
for the remaining content using a bytes range, even though the server just
stated it does NOT support bytes ranges, but only items ranges. (This latest
problem can be avoided by disallowing retries with the -t1 option.)
This issue can occur e.g. with a REST api providing lists of items as
application/json content.
_______________________________________________________
Reply to this item at:
<https://savannah.gnu.org/bugs/?60106>
_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/
