Hello, first of all, thank you for your recommendations.
On Monday 26 March 2012 16:34:21 Marcus Kool wrote: > Youtube may be hogging your pipe but it is better to know than to guess. Of course, before we decided for the proxy setup we investigated bandwidth usage. HTTP Traffic was about 60-70% of our traffic, and a good chunk of that was youtube. That's why we decided, that we would try a squid setup with youtube caching. However it took a while before we found a solution for caching youtube, as 3.1 hasn't implemented the necessary features yet. > The access.log shows content sizes so with a simple awk script it should > be easy to find out. > > I have also seen many sites where advertisements and trackers consume 15% > bandwidth. This may vary. So blocking ads and trackers is a thing to > consider. Thanks for this insight! This would of course be a welcome saving of bandwidth in my personal opinion. I'm just not sure if we're allowed to do this, as the patron of the proxy is a public-law institution and as such bound to anti- censorship laws. Need to check with a Legalese translator. > > Do not expect too much from web caching. More and more websites > intentionally make their sites not cacheable. Look at the percentage of > TCP_MISS in access.log or use a second awk script to find out more about > cacheability. Every bit counts. Before we apply for an increase of (expensive) uplink bandwidth we want to play every trick we have up our sleeve. At the moment our cache is still cold, because for getting the proxy running again I had to completely wipe the cache. At the moment we have a hit:miss ratio of about 1:5. For youtube caching we have a saved bandwidth around 100 GB for the 27th of march (one video in particular had a size of 768 MB and was watched 19 times). Online lectures are currently en vogue. > > I recommend going for a newer Squid: 3.1.19 is stable and fixes issues that > 3.1.10 has. Will do so. > On Linux, aufs has a better performance than diskd Thanks again for this tip! > > > Additional memory for storing objects is 2048 MB: > > > > cache_mem 2048 MB > > Seems right. But you also need virtual memory for Squid being able to > fork processes without issues. Do have have 8 GB swap ? Yes. 10 GB actually. > > But read the FAQ about memory usage and a large disk cache: > http://wiki.squid-cache.org/SquidFaq/SquidMemory > Squid uses an additional 512*14 MB = 7.1 GB for the index of the disk > cache. I suggest to downsize to 1 GB in-memory index which implies to > use only 73 GB disk cache. Ah okay, here's one of my initial mistakes. I used only 10 MB for my calculation but of course we use a 64bit squid. Out of curiosity and because I want to learn: the reasoning for shrinking the disk cache from 512 GB to 73 GB is because a big cache as we have it at the moment only leads to lots of stale objects in cache which additionally burden the CPU and RAM because of in- memory metadata? Best regards, - Christian Loth
