The -D flag seems to be ignored when a request is answered by a HTTP 302 Object moved. I am currently using GNU Wget 1.10.2 . This appear to be an old problem, searching the email archives, I found that Remco Kapinga sent an email on Tue, 09 Mar 2004 08:25:23, subjects ' wget 1.9.1, -D, 301 redirects' and just to refresh everyones memory, here is his email:
Hello, I use Wget (1.9.1) to stress-test my custom build ISAPI application, by using -r to recursivly follow all links. In addition I use -Dmydomain.org to prevent content from other domains to be requested. However, when my ISAPI application sends a redirect (HTTP Errorcode 301), wget simply followes the suggested link, even when outside the domain specified. What am I doing wrong? my command-line: wget --convert-links -r -p --save-headers --tries 2 -T 10 --html-extension -Dmydomain.org --save-headers Please send CC's to me, since I'm not ubscribed. Remco Kapinga I also include is a part of wget --debug output when i run into the problem: ---request begin--- GET /control.asp?CommCode=LM108&TargetApp=Car&isFromPortalPage=True&BrandID=blah&Promotion=DIRECT_TYPE HTTP/1.0 Referer: https://my-domain/ User-Agent: Wget/1.10.2 Accept: */* Host: my-domain Connection: Keep-Alive Cookie: ASPSESSIONIDSCBABADB=HIPNELBBIDCPOFJLAHNNLDLJ; GI80=IMLXZJSgiSn1001CKMMO ---request end--- ---response begin--- HTTP/1.1 302 Object moved Cache-Control: private Connection: keep-alive Date: Thu, 09 Feb 2006 11:26:49 GMT Content-Length: 280 Content-Type: text/html Location: https://other-domain/ProductsClassic/Car/?CommCode=LM108&TargetApp=Car&isFromPortalPage=True&BrandID=blah&Promotion=DIRECT_TYPE P3P: CP="NOI ADM DEV PSAi COM NAV OUR OTR STP IND DEM" Set-Cookie: blah=CARPROMOTION=DIRECT%5FTYPE&CARCCODE=LM108; expires=Sun, 26-Mar-2006 00:00:00 GMT; path=/ ---response end--- ---response end--- Stored cookie my-domain -1 (ANY) / <permanent> <insecure> [expiry 2006-03-26 00:00:00] blah CARPROMOTION=DIRECT%5FTYPE&CARCCODE=LM108 Skipping 280 bytes of body: [<head><title>Object moved</title></head> <body><h1>Object Moved</h1>This object may be found <a HREF="https://other-domain/ProductsClassic/Car/?CommCode=LM108&TargetApp=Car&isFromPortalPage=True&BrandID=blah&Promotion=DIRECT_TYPE";>here</a>.</body> ] done. Caching other-domain => other-domains-IP-address Created socket 4. Releasing 0x00402a00 (new refcount 1). Initiating SSL handshake. Handshake successful; connected socket 4 to SSL handle 0x00415d00 certificate: subject: xxx issuer: xxx WARNING: Certificate verification error for other-domain: unable to get local issuer certificate ---request begin--- GET /ProductsClassic/Car/?CommCode=LM108&TargetApp=Car&isFromPortalPage=True&BrandID=blah&Promotion=DIRECT_TYPE HTTP/1.0 Referer: https://my-domain/ User-Agent: Wget/1.10.2 Accept: */* Host: other-domain Connection: Keep-Alive ---request end--- Hope this is enough information to solve the problem. Kind regards Christian
