Hi, Thanks for looking into this, it seems though that wget fails to use the correct URL anyway, as when using "--content-on-error=on", a generic 404 error page is returned by the server instead of the blog post.
Here is an example using a better URL without the encoding. This produces curl.htm and wget.htm: > curl -L > 'https://www.bbc.co.uk/blogs/adamcurtis/entries/a2094c9d-9864-348e-a241-7aa93adf0c09' > > curl.htm > > wget --content-on-error=on > 'https://www.bbc.co.uk/blogs/adamcurtis/entries/a2094c9d-9864-348e-a241-7aa93adf0c09' > -O wget.htm curl.htm is the desired content, whereas wget.htm contains a generic 404 error page "Page cannot be found", suggesting wget is not redirecting to the correct URL. Thanks On Sun, Aug 23, 2026, at 11:45 AM, Tim Rühsen wrote: > Hi, > > > Wget fails to download pages from Adam Curtis' archived blog, such as > this: > > This is expected since the server answers with 404 instead of 200 and > wget dicards any content in this case (by default). > > To do store the content on error, you have to add `--content-on-error` > on the command line. > > Cheers, Tim > > On 8/22/26 13:24, [email protected] wrote: > > Wget fails to download pages from Adam Curtis' archived blog, such as this: > > > > https://www.bbc.co.uk/webarchive/https%253A%252F%252Fwww.bbc.co.uk%252Fblogs%252Fadamcurtis%252Fentries%252Fa2094c9d-9864-348e-a241-7aa9 > > > > But Curl succeeds. This may be because it is re-encoding the % symbols. > > > > output of wget -d: > > > >> 22.122345.635 Local URI encoding = 'UTF-8' > >> 22.122345.635 Input URI encoding = 'UTF-8' > >> 22.122345.635 Fetched HSTS data from > >> '/home/miles/.local/share/wget/.wget-hsts' > >> 22.122345.635 Fetched HPKP data from > >> '/home/miles/.local/share/wget/.wget-hpkp' > >> 22.122345.635 set_exit_status(0) > >> 22.122345.635 path > >> webarchive/https%3A%2F%2Fwww.bbc.co.uk%2Fblogs%2Fadamcurtis%2Fentries%2Fa2094c9d-9864-348e-a241-7aa9 > >> -> > >> 22.122345.635 > >> webarchive/https%3A%2F%2Fwww.bbc.co.uk%2Fblogs%2Fadamcurtis%2Fentries%2Fa2094c9d-9864-348e-a241-7aa9 > >> 22.122345.635 local filename = > >> 'https%3A%2F%2Fwww.bbc.co.uk%2Fblogs%2Fadamcurtis%2Fentries%2Fa2094c9d-9864-348e-a241-7aa9' > >> 22.122345.635 host_add_job: job fname > >> https%3A%2F%2Fwww.bbc.co.uk%2Fblogs%2Fadamcurtis%2Fentries%2Fa2094c9d-9864-348e-a241-7aa9 > >> 22.122345.635 host_add_job: 0x5622fb64ec70 > >> https://www.bbc.co.uk/webarchive/https%253A%252F%252Fwww.bbc.co.uk%252Fblogs%252Fadamcurtis%252Fentries%252Fa2094c9d-9864-348e-a241-7aa9 > >> 22.122345.635 host_add_job: qsize 1 host-qsize=1 > >> 22.122345.635 queue_size: qsize=1 > >> 22.122345.635 queue_size: qsize=1 > >> 22.122345.635 queue_size: qsize=1 > >> 22.122345.635 [0] action=1 pending=0 host=0x0 goaway=n/a > >> 22.122345.635 dequeue job > >> https://www.bbc.co.uk/webarchive/https%253A%252F%252Fwww.bbc.co.uk%252Fblogs%252Fadamcurtis%252Fentries%252Fa2094c9d-9864-348e-a241-7aa9 > >> 22.122345.635 resolving www.bbc.co.uk:443... > >> 22.122345.643 has 212.58.237.1:443 > >> 22.122345.643 has 212.58.236.129:443 > >> 22.122345.643 has 212.58.237.129:443 > >> 22.122345.643 has 212.58.235.1:443 > >> 22.122345.643 has 212.58.235.129:443 > >> 22.122345.643 has 212.58.236.1:443 > >> 22.122345.643 has 212.58.237.129:443 > >> 22.122345.643 has 212.58.236.1:443 > >> 22.122345.643 has 212.58.235.1:443 > >> 22.122345.643 has 212.58.237.1:443 > >> 22.122345.643 has 212.58.235.129:443 > >> 22.122345.643 has 212.58.236.129:443 > >> 22.122345.643 trying 212.58.237.1:443... > >> 22.122345.659 GnuTLS init > >> 22.122345.835 GnuTLS system certificate store is empty > >> 22.122345.835 Certificates loaded: 388 > >> 22.122345.835 GnuTLS init done > >> 22.122345.836 TLS False Start requested > >> 22.122345.836 SNI www.bbc.co.uk > >> 22.122345.836 OCSP stapling requested for www.bbc.co.uk > >> 22.122345.836 ALPN offering h2 > >> 22.122345.836 ALPN offering http/1.1 > >> 22.122345.894 OCSP stapling is not supported by 'www.bbc.co.uk' > >> 22.122345.894 host has no pubkey pinnings stored in hpkp db > >> 22.122345.894 host has no pubkey pinnings stored in hpkp db > >> 22.122345.894 host has no pubkey pinnings stored in hpkp db > >> 22.122345.894 host has no pubkey pinnings stored in hpkp db > >> 22.122345.894 host has no pubkey pinnings stored in hpkp db > >> 22.122345.894 host has no pubkey pinnings stored in hpkp db > >> 22.122345.894 host has no pubkey pinnings stored in hpkp db > >> 22.122345.894 host has no pubkey pinnings stored in hpkp db > >> 22.122345.894 TLS False Start: off > >> 22.122345.894 ALPN: Server accepted protocol 'h2' > >> 22.122345.895 Handshake completed > >> 22.122345.895 established connection www.bbc.co.uk > >> 22.122345.895 cookie_create_request_header for host=www.bbc.co.uk > >> path=webarchive/https%3A%2F%2Fwww.bbc.co.uk%2Fblogs%2Fadamcurtis%2Fentries%2Fa2094c9d-9864-348e-a241-7aa9 > >> 22.122345.895 HTTP2 stream id 1 > >> 22.122345.895 [0] action=1 pending=1 host=0x5622fb64e6b0 goaway=false > >> 22.122345.895 [0] action=2 pending=1 host=0x5622fb64e6b0 goaway=false > >> 22.122345.895 ## pending_requests = 1 > >> 22.122345.895 [FRAME 0] > SETTINGS > >> 22.122345.895 [FRAME 0] > WINDOW_UPDATE > >> 22.122345.895 [FRAME 1] > HEADERS > >> 22.122345.895 [FRAME 1] > :method: GET > >> 22.122345.895 [FRAME 1] > :path: > >> /webarchive/https%253A%252F%252Fwww.bbc.co.uk%252Fblogs%252Fadamcurtis%252Fentries%252Fa2094c9d-9864-348e-a241-7aa9 > >> 22.122345.895 [FRAME 1] > :scheme: https > >> 22.122345.895 [FRAME 1] > :authority: www.bbc.co.uk > >> 22.122345.895 [FRAME 1] > accept-encoding: gzip, deflate, br, zstd > >> 22.122345.895 [FRAME 1] > accept: */* > >> 22.122345.895 [FRAME 1] > user-agent: Wget/2.2.1 > >> 22.122345.909 [FRAME 0] < SETTINGS > >> 22.122345.909 [FRAME 0] < WINDOW_UPDATE > >> 22.122345.909 [FRAME 0] > SETTINGS > >> 22.122345.910 [FRAME 0] < SETTINGS > >> 22.122346.113 :status: 404 > >> 22.122346.113 date: Sat, 22 Aug 2026 11:23:46 GMT > >> 22.122346.113 content-type: text/html;charset=utf-8 > >> 22.122346.113 content-length: 9687 > >> 22.122346.113 belfrage-cache-status: MISS > >> 22.122346.113 bid: bruce > >> 22.122346.113 brequestid: 268b641cf47a4cd78df3312be105da1c > >> 22.122346.113 bsig: b24bbece8fc068ce8b9dfd6803e8e57e > >> 22.122346.113 cache-control: public, stale-if-error=90, > >> stale-while-revalidate=30, max-age=5 > >> 22.122346.113 content-encoding: gzip > >> 22.122346.113 req-svc-chain: GTM,BELFRAGE,MOZART > >> 22.122346.113 vary: accept-encoding > >> 22.122346.114 via: 2.0 BBC-GTM, 1.1 Belfrage > >> 22.122346.114 x-content-type-options: nosniff > >> 22.122346.114 x-mrid: n1 > >> 22.122346.114 x-xss-protection: 1; mode=block > >> 22.122346.114 origin-agent-cluster: ?0 > >> 22.122346.114 x-lb-nocache: true > >> 22.122346.114 nel: > >> {"report_to":"default","max_age":2592000,"include_subdomains":true,"failure_fraction":0.25} > >> 22.122346.114 x-bbc-edge-cache-status: MISS > >> 22.122346.114 x-bbc-origin-response-status: 404 > >> 22.122346.114 report-to: > >> {"group":"default","max_age":2592000,"endpoints":[{"url":"https://default.bbc-reporting-api.app/report-endpoint","priority":1}],"include_subdomains":true > >> > >> <https://default.bbc-reporting-api.app/report-endpoint%22,%22priority%22:1%7D],%22include_subdomains%22:true>} > >> 22.122346.114 server: BBC-GTM > >> 22.122346.114 strict-transport-security: max-age=31536000; preload > >> 22.122346.114 permissions-policy: browsing-topics=(), > >> join-ad-interest-group=(), run-ad-auction=() > >> 22.122346.114 x-robots-tag: bingbot: noarchive > >> 22.122346.114 [FRAME 1] < HEADERS > >> 22.122346.114 closing stream 1 > >> 22.122346.114 ## response status 404 > >> 22.122346.114 set_exit_status(8) > >> 22.122346.114 keep_alive=1 > >> 22.122346.114 update HSTS www.bbc.co.uk:443 (maxage=31536000, > >> includeSubDomains=0) > >> 22.122346.114 _host_remove_job: 0x5622fb64ec70 > >> 22.122346.114 host_remove_job: qsize=0 host->qsize=0 > >> 22.122346.114 [0] action=1 pending=0 host=0x5622fb64e6b0 goaway=false > >> 22.122346.114 closing connection > >> 22.122346.115 [0] action=1 pending=0 host=0x0 goaway=n/a > >> 22.122346.115 main: wake up > >> 22.122346.115 main: done > >> 22.122346.115 Successfully updated > >> '/home/miles/.local/share/wget/.wget-hsts'. > >> 22.122346.115 Saved 1 HSTS entry into > >> '/home/miles/.local/share/wget/.wget-hsts' > >> 22.122346.115 blacklist > >> https://www.bbc.co.uk/webarchive/https%253A%252F%252Fwww.bbc.co.uk%252Fblogs%252Fadamcurtis%252Fentries%252Fa2094c9d-9864-348e-a241-7aa9 > >> ---- > >> Certificate info [0]: > >> Valid since: Mon 13 Jul 2026 14:21:36 BST > >> Expires: Thu 28 Jan 2027 13:21:36 GMT > >> Fingerprint: e3756b8bc1f345cc3018a2b146b88ca5 > >> Serial number: 518996aacb43bd1f78166375 > >> Public key: EC/ECDSA, High (256 bits) > >> Version: #3 > >> DN: C=GB,ST=London,L=London,O=BRITISH BROADCASTING > >> CORPORATION,CN=www.bbc.co.uk > >> Issuer's DN: C=BE,O=GlobalSign nv-sa,CN=GlobalSign ECC OV SSL CA 2018 > >> Issuer's OID: 2.5.4.6 > >> Issuer's UID: 2.5.4.6 > >> Certificate info [1]: > >> Valid since: Wed 21 Nov 2018 00:00:00 GMT > >> Expires: Tue 21 Nov 2028 00:00:00 GMT > >> Fingerprint: 9635de986bc7f9b19f09aec50e3a3909 > >> Public key: EC/ECDSA, Ultra (384 bits) > >> Version: #3 > >> DN: C=BE,O=GlobalSign nv-sa,CN=GlobalSign ECC OV SSL CA 2018 > >> Issuer's DN: OU=GlobalSign ECC Root CA - R5,O=GlobalSign,CN=GlobalSign > >> Issuer's OID: 2.5.4.11 > >> Issuer's UID: 2.5.4.11 > >> Certificate info [2]: > >> Valid since: Wed 21 Nov 2018 00:00:00 GMT > >> Expires: Sun 18 Mar 2029 10:00:00 GMT > >> Fingerprint: 8f665697f9d7f2c6c6b96a99259b7ffc > >> Serial number: 01ee5f223e3d355b992bc82ab4 > >> Public key: EC/ECDSA, Ultra (384 bits) > >> Version: #3 > >> DN: OU=GlobalSign ECC Root CA - R5,O=GlobalSign,CN=GlobalSign > >> Issuer's DN: OU=GlobalSign Root CA - R3,O=GlobalSign,CN=GlobalSign > >> Issuer's OID: 2.5.4.11 > >> Issuer's UID: 2.5.4.11 > >> Certificate info [3]: > >> Valid since: Wed 19 Sep 2018 01:00:00 BST > >> Expires: Fri 28 Jan 2028 12:00:00 GMT > >> Fingerprint: 58bfb0eacf17e2d0a65a4d061042c3bf > >> Serial number: 01ee5f169dff97352b6465d66a > >> Public key: RSA, Medium (2048 bits) > >> Version: #3 > >> DN: OU=GlobalSign Root CA - R3,O=GlobalSign,CN=GlobalSign > >> Issuer's DN: C=BE,O=GlobalSign nv-sa,OU=Root CA,CN=GlobalSign Root CA > >> Issuer's OID: 2.5.4.6 > >> Issuer's UID: 2.5.4.6 > >> ---- > >> Ephemeral ECDH using curve (null) > >> Key Exchange: ECDHE-RSA > >> Protocol: TLS1.3 > >> Certificate Type: X.509 > >> Cipher: AES-256-GCM > >> MAC: AEAD > >> ---- > >> [0] Downloading > >> 'https://www.bbc.co.uk/webarchive/https%253A%252F%252Fwww.bbc.co.uk%252Fblogs%252Fadamcurtis%252Fentries%252Fa2094c9d-9864-348e-a241-7aa9' > >> ... > >> HTTP ERROR response 404 > >> [https://www.bbc.co.uk/webarchive/https%253A%252F%252Fwww.bbc.co.uk%252Fblogs%252Fadamcurtis%252Fentries%252Fa2094c9d-9864-348e-a241-7aa9] > >> > > > > *Attachments:* > • OpenPGP_signature.asc
