Thanks for this Michael.
This is so surprising. If someone decides to Dos and crawls the website with a rogue header, this will essentially bypass the cache and put a strain on the website. In fact, I was hit by a dos attack that’s when I started looking at logs and realized the large number of MISSes. Can someone please help? - Cherian On Sat, May 12, 2018 at 12:01 PM, Friscia, Michael <michael.fris...@yale.edu > wrote: > I'm not sure if this will help, but I ignore/hide a lot, this is in my > config > > > proxy_ignore_headers X-Accel-Expires Expires Cache-Control Set-Cookie; > proxy_hide_header X-Accel-Expires; > proxy_hide_header Pragma; > proxy_hide_header Server; > proxy_hide_header Request-Context; > proxy_hide_header X-Powered-By; > proxy_hide_header X-AspNet-Version; > proxy_hide_header X-AspNetMvc-Version; > > > I have not experienced the problem you mention, I just thought I would > offer my config. > > > ___________________________________________ > > Michael Friscia > > Office of Communications > > Yale School of Medicine > > (203) 737-7932 – office > > (203) 931-5381 – mobile > > http://web.yale.edu > <https://mailtrack.io/trace/link/8357a0bdd8c40c2ff5b7d91c7797cbc7a8535ffb?url=http%3A%2F%2Fweb.yale.edu%2F&userId=74734&signature=d652edf1f4f21323> > > > ------------------------------ > *From:* nginx <nginx-boun...@nginx.org> on behalf of Quintin Par < > quintin...@gmail.com> > *Sent:* Saturday, May 12, 2018 1:32 PM > *To:* nginx@nginx.org > *Subject:* Re: Debugging Nginx Cache Misses: Hitting high number of MISS > despite high proxy valid > > > That’s the tricky part. These MISSes are intermittent. Whenever I run curl > I get HITs but I end up seeing a lot of MISS in the logs. > > > > How do I log these MiSSes with the reason? I want to know what headers > ended up bypassing the cache. > > > > Here’s my caching config > > > > proxy_pass http://127.0.0.1:8000 > <https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1-3A8000&d=DwMFaQ&c=cjytLXgP8ixuoHflwc-poQ&r=wvXEDjvtDPcv7AlldT5UvDx32KXBEM6um_lS023SJrs&m=F-qGMOyS74uE8JM-dOLmNH92bQ1xQ-7Rj1d6k-_WST4&s=NHvlb1WColNwTWBF36P1whJdu5iWHK9_50IDHugaEdQ&e=> > ; > > proxy_set_header X-Real-IP $remote_addr; > > proxy_set_header X-Forwarded-For > $proxy_add_x_forwarded_for; > > proxy_set_header X-Forwarded-Proto https; > > proxy_set_header X-Forwarded-Port 443; > > > > # If logged in, don't cache. > > if ($http_cookie ~* > "comment_author_|wordpress_(?!test_cookie)|wp-postpass_" > ) { > > set $do_not_cache 1; > > } > > proxy_cache_key "$scheme://$host$request_uri$ > do_not_cache"; > > proxy_cache staticfilecache; > > add_header Cache-Control public; > > proxy_cache_valid 200 120d; > > proxy_hide_header "Set-Cookie"; > > proxy_ignore_headers "Set-Cookie"; > > proxy_ignore_headers "Cache-Control"; > > proxy_hide_header "Cache-Control"; > > proxy_pass_header X-Accel-Expires; > > > > proxy_set_header Accept-Encoding ""; > > proxy_ignore_headers Expires; > > add_header X-Cache-Status $upstream_cache_status; > > proxy_cache_use_stale timeout; > > proxy_cache_bypass $arg_nocache $do_not_cache; > - Quintin > > > On Sat, May 12, 2018 at 10:29 AM Lucas Rolff <lu...@lucasrolff.com> wrote: > > It can be as simple as doing a curl to your “origin” url (the one you > proxy_pass to) for the files you see that gets a lot of MISS’s – if there’s > odd headers such as cookies etc, then you’ll most likely experience a bad > cache if your nginx is configured to not ignore those headers. > > > > *From: *nginx <nginx-boun...@nginx.org> on behalf of Quintin Par < > quintin...@gmail.com> > *Reply-To: *"nginx@nginx.org" <nginx@nginx.org> > *Date: *Saturday, 12 May 2018 at 18.26 > *To: *"nginx@nginx.org" <nginx@nginx.org> > *Subject: *Debugging Nginx Cache Misses: Hitting high number of MISS > despite high proxy valid > > > > [image: > https://mailtrack.io/trace/mail/86a613eb1ce46a4e7fa6f9eb96989cddae639800.png?u=74734] > > My proxy cache path is set to a very high size > > > > proxy_cache_path /var/lib/nginx/cache levels=1:2 > keys_zone=staticfilecache:180m max_size=700m; > > and the size used is only > > > > sudo du -sh * > > 14M cache > > 4.0K proxy > > Proxy cache valid is set to > > > > proxy_cache_valid 200 120d; > > I track HIT and MISS via > > > > add_header X-Cache-Status $upstream_cache_status; > > Despite these settings I am seeing a lot of MISSes. And this is for pages > I intentionally ran a cache warmer an hour ago. > > > > How do I debug why these MISSes are happening? How do I find out if the > miss was due to eviction, expiration, some rogue header etc? Does Nginx > provide commands for this? > > > > - Quintin > _______________________________________________ > nginx mailing list > nginx@nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx > <https://mailtrack.io/trace/link/122c3dbd333c388f47f5c2776af9ebc3fc75ae10?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttp-3A__mailman.nginx.org_mailman_listinfo_nginx%26d%3DDwMFaQ%26c%3DcjytLXgP8ixuoHflwc-poQ%26r%3DwvXEDjvtDPcv7AlldT5UvDx32KXBEM6um_lS023SJrs%26m%3DF-qGMOyS74uE8JM-dOLmNH92bQ1xQ-7Rj1d6k-_WST4%26s%3DD3LnZhfobOtlEStCvCDrcwmHydEHaGRFC4gnWvRT5Uk%26e%3D&userId=74734&signature=56c7a7ad18b2c057> > > > _______________________________________________ > nginx mailing list > nginx@nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx > <https://mailtrack.io/trace/link/92c2700d67bd6891ca1606e2df4e0f11c6d82260?url=http%3A%2F%2Fmailman.nginx.org%2Fmailman%2Flistinfo%2Fnginx&userId=74734&signature=3763121afa828bb7> >
_______________________________________________ nginx mailing list nginx@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx