Hi, I'm a varnish noob. I've only just started rolling out a cache in front of a VBulletin site running Apache that is currently using pound for load balancing.
I'm running 2.1.5 on a debian lenny box. Testing is going well, apart from one problem. The site runs VBSEO to generate sitemap files. Without excpetion, every time Googlebot tries to request these files Varnish returns a 503: 66.249.66.246 - - [05/Mar/2011:09:33:53 +0000] "GET http://www.sitename.net/sitemap_151.xml.gz HTTP/1.1" 503 419 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" I can request these files via wget direct from the backend as well as direct from varnish without a problem: --2011-03-05 09:23:39-- http://www.sitename.net/sitemap_362.xml.gz HTTP request sent, awaiting response... HTTP/1.1 200 OK Server: Apache Content-Type: application/x-gzip Content-Length: 130283 Date: Sat, 05 Mar 2011 09:23:38 GMT X-Varnish: 1282440127 Age: 0 Via: 1.1 varnish Connection: keep-alive Length: 130283 (127K) [application/x-gzip] Saving to: `/dev/null' 2011-03-05 09:23:39 (417 KB/s) - `/dev/null' saved [130283/130283] I've reverted back to default.vcl, the only changes being to define my own backends. Varnishlog output is below. Having googled a bit the only thing I've found is bug #813, but that was apparently fixed prior to 2.1.5. Am I missing something obvious? -Ronan Varnishlog output 18 ReqStart c 66.249.66.246 63009 1282436348 18 RxRequest c GET 18 RxURL c /sitemap_362.xml.gz 18 RxProtocol c HTTP/1.1 18 RxHeader c Host: www.sitename.net 18 RxHeader c Connection: Keep-alive 18 RxHeader c Accept: */* 18 RxHeader c From: googlebot(at)googlebot.com 18 RxHeader c User-Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) 18 RxHeader c Accept-Encoding: gzip,deflate 18 RxHeader c If-Modified-Since: Sat, 05 Mar 2011 08:40:46 GMT 18 VCL_call c recv 18 VCL_return c lookup 18 VCL_call c hash 18 VCL_return c hash 18 VCL_call c miss 18 VCL_return c fetch 18 Backend c 40 sitename sitename1 40 TxRequest b GET 40 TxURL b /sitemap_362.xml.gz 40 TxProtocol b HTTP/1.1 40 TxHeader b Host: www.sitename.net 40 TxHeader b Accept: */* 40 TxHeader b From: googlebot(at)googlebot.com 40 TxHeader b User-Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) 40 TxHeader b Accept-Encoding: gzip,deflate 40 TxHeader b X-Forwarded-For: 66.249.66.246 40 TxHeader b X-Varnish: 1282436348 40 RxProtocol b HTTP/1.1 40 RxStatus b 200 40 RxResponse b OK 40 RxHeader b Date: Sat, 05 Mar 2011 09:17:37 GMT 40 RxHeader b Server: Apache 40 RxHeader b Content-Length: 130327 40 RxHeader b Content-Encoding: gzip 40 RxHeader b Vary: Accept-Encoding 40 RxHeader b Content-Type: application/x-gzip 18 TTL c 1282436348 RFC 10 1299316657 0 0 0 0 18 VCL_call c fetch 18 VCL_return c deliver 18 ObjProtocol c HTTP/1.1 18 ObjStatus c 200 18 ObjResponse c OK 18 ObjHeader c Date: Sat, 05 Mar 2011 09:17:37 GMT 18 ObjHeader c Server: Apache 18 ObjHeader c Content-Encoding: gzip 18 ObjHeader c Vary: Accept-Encoding 18 ObjHeader c Content-Type: application/x-gzip 18 FetchError c straight read_error: 0 40 Fetch_Body b 4 4294967295 1 40 BackendClose b sitename1 18 VCL_call c error 18 VCL_return c deliver 18 VCL_call c deliver 18 VCL_return c deliver 18 TxProtocol c HTTP/1.1 18 TxStatus c 503 18 TxResponse c Service Unavailable 18 TxHeader c Server: Varnish 18 TxHeader c Retry-After: 0 18 TxHeader c Content-Type: text/html; charset=utf-8 18 TxHeader c Content-Length: 419 18 TxHeader c Date: Sat, 05 Mar 2011 09:17:38 GMT 18 TxHeader c X-Varnish: 1282436348 18 TxHeader c Age: 1 18 TxHeader c Via: 1.1 varnish 18 TxHeader c Connection: close 18 Length c 419 18 ReqEnd c 1282436348 1299316657.660784483 1299316658.684726000 0.478523970 1.023897409 0.000044107 18 SessionClose c error 18 StatSess c 66.249.66.246 63009 6 1 5 0 0 4 2984 32012 _______________________________________________ varnish-misc mailing list [email protected] http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
