Re: Varnish logging and data merging

2010-01-10 Thread Michael S. Fischer
Varnish does keep a log if you ask it to. On Jan 10, 2010, at 10:37 PM, pub crawler wrote: > Alright, up and running with Varnish successfully. Quite happy with > Varnish. Our app servers no longer are failing / overwhelmed. > > Here's our new problem... > > We have a lot of logging going on

Varnish logging and data merging

2010-01-10 Thread pub crawler
Alright, up and running with Varnish successfully. Quite happy with Varnish. Our app servers no longer are failing / overwhelmed. Here's our new problem... We have a lot of logging going on in our applications. Logs pages, IP info, time date, URL parameters, etc. Since many pages are being serv

Re: AW: Varnish poisoned cache avoidance

2010-01-10 Thread Michael S. Fischer
It has been my experience that anti-DoS is usually easiest to implement at the origin server level, where the request handlers are typically more flexible and easiest to program. Even forking servers like Apache can issue 4xx responses lightning fast, without many resources being consumed. --M

Re: Caching pages with URL parameters

2010-01-10 Thread pub crawler
Well, again, I believe I've solved my own problem with my VCL through trial and error :) Managed to get Varnish stripping out the cookies on pages we want to cache and don't care about cookies on. Pretty impressed with Varnish. Is there a place to submit bug tracking items and feature requests f

Re: varnish suddenly restarting / flushing itself after several hours?

2010-01-10 Thread Frank van Lingen
Thanks David, I do monitor the memory and cpu load but there seem to be no apparent increase when this happens. I should point out that I run this in on a hosted VPS but I am well below their memory and cpu requirements. Varnish does restart itself gracefully when this happens. I guess if prefet

Re: Caching pages with URL parameters

2010-01-10 Thread pub crawler
I managed to perfect the caching of our pages spitting out cookies through trial and error. So this page for instance now caches in Varnish even though the cookies come along with the page: http://www.pubcrawler.com/Template/dsp_restaurant_zoom.cfm/flat/ID=415517/muma%20restaurant Accomplished th

Re: Caching pages with URL parameters

2010-01-10 Thread pub crawler
I wanted to describe this in better degree to see if anyone can recommend a way to get Varnish caching these sorts of pages. This page: http://www.pubcrawler.com/Template/dsp_restaurant_zoom.cfm/flat/ID=460401 If I look at underlying data via curl: HTTP/1.1 200 OK Set-Cookie: CFID=12184891;expire

Re: AW: Varnish poisoned cache avoidance

2010-01-10 Thread pub crawler
The antiDoS features would be a good enhancement to Varnish. I realize it's a very complex and resource intensive thing to approach. There are likely many other ways some of these functions could be used in other ways for other solutions. In our instance we are not experiencing a true denial of s

Re: Varnish poisoned cache avoidance

2010-01-10 Thread pub crawler
Thanks Ken, you input is very valuable to me. There's so much to learn from others using Varnish. Very impressed with Varnish and community. I took the approach of blocking IPs at firewall level to protect our app servers. Lots of these banned IPs are malicious, hack attempts - others are repeti

Re: AW: Varnish poisoned cache avoidance

2010-01-10 Thread Poul-Henning Kamp
In message <01cf01ca91db$8c29b790$a47d26...@de>, "Mike Schiessl" writes: >How can varnishd help me prevent DDOS / DOS attacks ? Firstly, by being damn fast. Originally we had some plans for specific antiDoS measures, something like: sub vcl_recv { if (client.bandwidth >

AW: Varnish poisoned cache avoidance

2010-01-10 Thread Mike Schiessl
When I read this messages, theres something I miss, too. Some function in varnishd, similar to the lighty/apache mod_evasive. It enables you the possibility to create dynamic hit/connection rate depending blocking of ips. How can varnishd help me prevent DDOS / DOS attacks ? Regards Mike Schies