I'll fire up tcpdump. You may be on to something; there is some authentication but I'm not sure if ATS is configured to ignore the headers. Thanks!
On Sat, Mar 30, 2019 at 12:38 AM Leif Hedstrom <[email protected]> wrote: > It looks like the requests you did with duel ended up caching and serve > from cache, but the ones via the VIP are not? > > Is anything injecting something in the other requests that makes it > uncacheable? Pragma: no-cache, cc: max-age=0, adding some authentication > headers etc etc? > > I think you have to capture some requests / response headers from those > transactions. > > — Leif > > On Mar 30, 2019, at 00:21, John Garvin <[email protected]> wrote: > > Hi - > encouraging the correct behavior is the idea. How do I get from here to > there with ATS is the problem. > > Here are two sequential requests to the origin: > curl -v > http://192.168.5.80:80/hs1/comprehensive.4/incremental/cksum/16824/16824090.txt; > sleep 3; curl -v > http://192.168.5.80:80/hs1/comprehensive.4/incremental/cksum/16824/16824090.txt > * Trying 192.168.5.80... > * Connected to 192.168.5.80 (192.168.5.80) port 80 (#0) > > GET /hs1/comprehensive.4/incremental/cksum/16824/16824090.txt HTTP/1.1 > > Host: 192.168.5.80 > > User-Agent: curl/7.43.0 > > Accept: */* > > > < HTTP/1.1 200 OK > < Date: Fri, 29 Mar 2019 23:02:09 GMT > < Server: Apache > < Last-Modified: Fri, 29 Mar 2019 18:55:49 GMT > < ETag: "218e7e-6a-5854038173740" > < Accept-Ranges: bytes > < Content-Length: 106 > < Cache-Control: max-age=604800, stale-if-error=86400 > < Content-Type: text/plain; charset=ISO-8859-1 > < > 9826accf5f02a84ce3898777405466b5dd5069cc > 2f4a9e3528e70a63c19f67963991537aac1c87b36e98fdd37496316e7a386c64 > * Connection #0 to host 192.168.5.80 left intact > * Trying 192.168.5.80... > * Connected to 192.168.5.80 (192.168.5.80) port 80 (#0) > > GET /hs1/comprehensive.4/incremental/cksum/16824/16824090.txt HTTP/1.1 > > Host: 192.168.5.80 > > User-Agent: curl/7.43.0 > > Accept: */* > > > < HTTP/1.1 200 OK > < Date: Fri, 29 Mar 2019 23:02:12 GMT > < Server: Apache > < Last-Modified: Fri, 29 Mar 2019 18:55:49 GMT > < ETag: "218e7e-6a-5854038173740" > < Accept-Ranges: bytes > < Content-Length: 106 > < Cache-Control: max-age=604800, stale-if-error=86400 > < Content-Type: text/plain; charset=ISO-8859-1 > < > 9826accf5f02a84ce3898777405466b5dd5069cc > 2f4a9e3528e70a63c19f67963991537aac1c87b36e98fdd37496316e7a386c64 > * Connection #0 to host 192.168.5.80 left intact > > and to ATS, whose origin server is 5.80: > jgarvin@opsshell901(~)$ curl -v > http://muproxy009:3128/hs1/comprehensive.4/incremental/cksum/16824/16824090.txt > * About to connect() to muproxy009 port 3128 (#0) > * Trying 192.168.5.199... connected > * Connected to muproxy009 (192.168.5.199) port 3128 (#0) > > GET /hs1/comprehensive.4/incremental/cksum/16824/16824090.txt HTTP/1.1 > > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/ > 3.12.9.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2 > > Host: muproxy009:3128 > > Accept: */* > > > < HTTP/1.1 200 OK > < Date: Fri, 29 Mar 2019 23:09:50 GMT > < Server: ATS/7.1.4 > < Last-Modified: Fri, 29 Mar 2019 18:55:49 GMT > < ETag: "218e7e-6a-5854038173740" > < Accept-Ranges: bytes > < Content-Length: 106 > < Cache-Control: max-age=604800, stale-if-error=86400 > < Content-Type: text/plain; charset=ISO-8859-1 > < Age: 102 > < Connection: keep-alive > < > 9826accf5f02a84ce3898777405466b5dd5069cc > 2f4a9e3528e70a63c19f67963991537aac1c87b36e98fdd37496316e7a386c64 > * Connection #0 to host muproxy009 left intact > * Closing connection #0 > jgarvin@opsshell901(~)$ curl -v > http://muproxy009:3128/hs1/comprehensive.4/incremental/cksum/16824/16824090.txt > * About to connect() to muproxy009 port 3128 (#0) > * Trying 192.168.5.199... connected > * Connected to muproxy009 (192.168.5.199) port 3128 (#0) > > GET /hs1/comprehensive.4/incremental/cksum/16824/16824090.txt HTTP/1.1 > > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/ > 3.12.9.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2 > > Host: muproxy009:3128 > > Accept: */* > > > > > < HTTP/1.1 200 OK > < Date: Fri, 29 Mar 2019 23:11:38 GMT > < Server: ATS/7.1.4 > < Last-Modified: Fri, 29 Mar 2019 18:55:49 GMT > < ETag: "218e7e-6a-5854038173740" > < Accept-Ranges: bytes > < Content-Length: 106 > < Cache-Control: max-age=604800, stale-if-error=86400 > < Content-Type: text/plain; charset=ISO-8859-1 > < Age: 8 > < Connection: keep-alive > < > 9826accf5f02a84ce3898777405466b5dd5069cc > 2f4a9e3528e70a63c19f67963991537aac1c87b36e98fdd37496316e7a386c64 > * Connection #0 to host muproxy009 left intact > * Closing connection #0 > > And the squid ascii log: > 1553901072.923 1 192.168.5.1 TCP_HIT/200 435 GET > http://192.168.5.80:80/hs1/comprehensive.4/incremental/cksum/16824/16824090.txt > - NONE/- text/plain > 1553901074.984 1 192.168.5.200 TCP_MISS/200 434 GET > http://192.168.5.80:80/hs1/comprehensive_sender_rep.4/incremental/cksum/16824/16824090.txt > - DIRECT/192.168.5.80 text/plain > 1553901079.354 0 192.168.5.1 TCP_MEM_HIT/200 435 GET > http://192.168.5.80:80/hs1/comprehensive.4/incremental/cksum/16824/16824090.txt > - NONE/- text/plain > 1553901092.670 0 192.168.5.1 TCP_MEM_HIT/200 436 GET > http://192.168.5.80:80/hs1/comprehensive.4/incremental/cksum/16824/16824090.txt > - NONE/- text/plain > 1553901094.088 3 192.168.5.200 TCP_MISS/200 434 GET > http://192.168.5.80:80/hs1/sp_sender_rep.4/incremental/cksum/16824/16824090.txt > - DIRECT/192.168.5.80 text/plain > 1553901098.777 0 192.168.5.200 TCP_MISS/200 434 GET > http://192.168.5.80:80/hs1/comprehensive.4/incremental/cksum/16824/16824090.txt > - DIRECT/192.168.5.80 text/plain > 1553901098.801 1 192.168.5.200 TCP_MISS/200 434 GET > http://192.168.5.80:80/hs1/sp_sender_rep.4/incremental/cksum/16824/16824090.txt > - DIRECT/192.168.5.80 text/plain > 1553901106.405 1 192.168.5.1 TCP_HIT/200 434 GET > http://192.168.5.80:80/hs1/comprehensive.4/incremental/cksum/16824/16824090.txt > - NONE/- text/plain > 1553901127.592 1 192.168.5.200 TCP_MISS/200 434 GET > http://192.168.5.80:80/hs1/comprehensive_sender_rep.4/incremental/cksum/16824/16824090.txt > - DIRECT/192.168.5.80 text/plain > 1553901167.274 1 192.168.5.200 TCP_MISS/200 434 GET > http://192.168.5.80:80/hs1/comprehensive.4/incremental/cksum/16824/16824090.txt > - DIRECT/192.168.5.80 text/plain > 1553901182.731 1 192.168.5.200 TCP_MISS/200 434 GET > http://192.168.5.80:80/hs1/sp_sender_rep.4/incremental/cksum/16824/16824090.txt > - DIRECT/192.168.5.80 text/plain > 1553901238.084 1 192.168.5.200 TCP_MISS/200 434 GET > http://192.168.5.80:80/hs1/comprehensive_sender_rep.4/incremental/cksum/16824/16824090.txt > - DIRECT/192.168.5.80 text/plain > 1553901267.457 11 192.168.5.200 TCP_MISS/200 434 GET > http://192.168.5.80:80/hs1/comprehensive.4/incremental/cksum/16824/16824090.txt > - DIRECT/192.168.5.80 text/plain > 1553901272.687 1 192.168.5.200 TCP_MISS/200 434 GET > http://192.168.5.80:80/hs1/sp_sender_rep.4/incremental/cksum/16824/16824090.txt > - DIRECT/192.168.5.80 text/plain > 1553901287.571 0 192.168.5.200 TCP_MISS/200 434 GET > http://192.168.5.80:80/hs1/comprehensive_sender_rep.4/incremental/cksum/16824/16824090.txt > - DIRECT/192.168.5.80 text/plain > 1553901299.911 0 192.168.5.200 TCP_MISS/200 434 GET > http://192.168.5.80:80/hs1/sp_sender_rep.4/incremental/cksum/16824/16824090.txt > - DIRECT/192.168.5.80 text/plain > 1553901313.480 0 192.168.5.200 TCP_MISS/200 434 GET > http://192.168.5.80:80/hs1/sp_sender_rep.4/incremental/cksum/16824/16824090.txt > - DIRECT/192.168.5.80 text/plain > 1553901317.052 67 192.168.5.200 TCP_MISS/200 434 GET > http://192.168.5.80:80/hs1/comprehensive_sender_rep.4/incremental/cksum/16824/16824090.txt > - DIRECT/192.168.5.80 text/plain > 1553901324.131 1 192.168.5.200 TCP_MISS/200 434 GET > http://192.168.5.80:80/hs1/comprehensive.4/incremental/cksum/16824/16824090.txt > - DIRECT/192.168.5.80 text/plain > 1553901330.033 1 192.168.5.200 TCP_MISS/200 434 GET > http://192.168.5.80:80/hs1/comprehensive.4/incremental/cksum/16824/16824090.txt > - DIRECT/192.168.5.80 text/plain > 1553901376.407 1 192.168.5.200 TCP_MISS/200 434 GET > http://192.168.5.80:80/hs1/sp_sender_rep.4/incremental/cksum/16824/16824090.txt > - DIRECT/192.168.5.80 text/plain > > where 192.168.5.1 must be the client IP which made the request. > (192.168.5.200 is the VIP fronting the cache tier). > > Lots of TCP_MISSes, for a file that should have a max-age of one week. > This is what's puzzling me. > > > On Fri, Mar 29, 2019 at 3:55 PM Miles Libbey <[email protected]> wrote: > >> Could you show 2 requests in a row to the same file? The logs you >> posted were requests to different files (...241.txt, ...326.tt, >> ...327.txt). >> >> Since your origin seems like its doing the right thing, I'd stay away >> from the url_regex, and continue encouraging the correct behavior :) >> >> miles >> >> On Fri, Mar 29, 2019 at 1:28 PM John Garvin <[email protected]> wrote: >> > >> > Hi - >> > I'm seeing consistent TCP_MISSes for URIs that happen to follow a >> pattern. >> > >> > We distribute two kinds of .txt files. One is a serial number, say >> serial.txt, with a sequential number incremented every 2-5 seconds. The >> origin server, Apache http 2.2.x, has set the Cache-Control header like so: >> > < Cache-Control: max-age=5, stale-if-error=86400 >> > >> > The second type of txt file is a checksum. There is a checksum file for >> every increment of the serial number. The Cache-Control header is >> > < Cache-Control: max-age=604800, stale-if-error=86400 >> > >> > I see consistent TCP_MISS messages in squid ascii logs for the cksum >> files. >> > >> > 1553890048.810 1 192.168.5.200 TCP_IMS_HIT/304 189 GET >> http://192.168.5.80:80/ebl/srl.txt - NONE/- - >> > 1553890048.817 1 192.168.5.200 TCP_MISS/200 434 GET >> http://192.168.5.80:80/hs1/sp_sender_rep.4/incremental/cksum/16824/16824241.txt >> - DIRECT/192.168.5.80 text/plain >> > 1553890048.818 0 192.168.5.200 TCP_MISS/200 433 GET >> http://192.168.5.80:80/hs1/sp_sender_rep.4/incremental/cksum/16824/16824326.txt >> - DIRECT/192.168.5.80 text/plain >> > 1553890048.838 1 192.168.5.200 TCP_MISS/200 434 GET >> http://192.168.5.80:80/hs1/comprehensive.4/incremental/cksum/16824/16824302.txt >> - DIRECT/192.168.5.80 text/plain >> > 1553890048.839 1 192.168.5.200 TCP_MISS/200 433 GET >> http://192.168.5.80:80/hs1/sp_sender_rep.4/incremental/cksum/16824/16824327.txt >> - DIRECT/192.168.5.80 text/plain >> > >> > First, would anyone have any insight into why ATS 7.1.4 is not caching >> the cksum files (\d+\.txt) but IS caching the serial placeholders? >> > The only difference I see is that the max-age value in the >> Cache-Control header. >> > Any pointers to look towards would be appreciated. >> > >> > Second, would anyone like to take a stab at a url_regex to match the >> cksum paths? I have >> > >> > url_regex==cksum/\d+/\d+\.txt ttl-in-cache=1d >> > >> > but I don't see any change in the logging. >> > >> > Thanks for any pointers. >> > >> > -- >> > John Garvin >> > >> > > > -- > John Garvin > 415 816 4295 > > -- John Garvin 415 816 4295
