Re: [squid-users] Config suggestion
On Tue, 17 Mar 2009 09:54:00 +0100, Matus UHLAR - fantomas uh...@fantomas.sk wrote: is that one quad-core with hyperthreading, two quad-cores without HT or two dual-cores with HT? We apparently should count HT CPU's as one, not two. On 17.03.09 12:35, Herbert Faleiros wrote: 2 Xeon Quad-cores (4 cores per/processor, 8 total), no HT... Good, you can safely use a few squid processes there, total used free sharedbuffers cached Mem: 32148 2238 29910 0244 823 -/+ buffers/cache: 1169 30978 Swap:15264 0 15264 swap is quite useless here I'd say... Uptime was 1/2 min. Look at it now: I meant that configuring swap space on system with that much of ram should be useless. Of course, some OSes can progressively store data onto swap even when not needed. But as long as swapping will be needed, the system needs tuning or more RAM. I'm not to up on the L1/L2 efficiencies, but 64 256 or higher L1 seems to be better for larger dir sizes. OK, I will try... even 256 256 on 300G disks. [cut] Note that for 300GiB HDD you will be using max 250, more probably 200 and some ppl would advise 150GiB of cache. Leave some space for metadata and some for reserve - filesystems may benefit of it. I always configure (to use) only 80% HDD... 80% of what? announced disk space? of the filesystems' free space after creating? Or that much as will 80% after filling up? (they can and apparently will differ). The latter, the better imho. Btw may I know those numbers (just for curiosity). -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. The box said 'Requires Windows 95 or better', so I bought a Macintosh.
Re: [squid-users] Config suggestion
# cat /proc/cpuinfo | egrep -i xeon | uniq model name : Intel(R) Xeon(R) CPU E5405 @ 2.00GHz # cat /proc/cpuinfo | egrep -i xeon | wc -l 8 is that one quad-core with hyperthreading, two quad-cores without HT or two dual-cores with HT? We apparently should count HT CPU's as one, not two. total used free sharedbuffers cached Mem: 32148 2238 29910 0244823 -/+ buffers/cache: 1169 30978 Swap:15264 0 15264 swap is quite useless here I'd say... # fdisk -l | grep GB Disk /dev/sda: 73.5 GB, 73557090304 bytes Disk /dev/sdb: 300.0 GB, 3000 bytes Disk /dev/sdc: 146.8 GB, 146815737856 bytes Disk /dev/sdd: 300.0 GB, 3000 bytes Disk /dev/sde: 300.0 GB, 3000 bytes # uname -srm Linux 2.6.27.7 x86_64 # cat /etc/squid/squid.conf | grep -E cache_'mem|dir'\ you apparently really wandet cache_'(mem|dir)' btw... cache_mem 8192 MB cache_dir aufs /var/cache/proxy/cache1 102400 16 256 cache_dir aufs /var/cache/proxy/cache2 102400 16 256 cache_dir aufs /var/cache/proxy/cache3 102400 16 256 cache_dir aufs /var/cache/proxy/cache4 102400 16 256 cache_dir aufs /var/cache/proxy/cache5 102400 16 256 cache_dir aufs /var/cache/proxy/cache6 102400 16 256 cache_dir aufs /var/cache/proxy/cache7 102400 16 256 cache_dir aufs /var/cache/proxy/cache8 102400 16 256 # cat /etc/fstab | grep proxy /dev/vg00/cache /var/cache/proxy ext3defaults 1 2 Yes, I know, LVM, ext3 and aufs are bad ideas... I'm particularly interested in a better cache_dir configuration (maximizing disk's usage) and the correct cache_mem parameter to this hardware. (and others possible/useful tips) lvm is surely bad idea for proxy, not sure that about ext3 (very stable) and aufs is _good_ idea. On 17.03.09 12:58, Amos Jeffries wrote: You have 5 physical disks by the looks of it. Best usage of those is to split the cache_dir one per disk (sharing a disk leads to seek clashes). I'd say that the 73.5 Gb disk should be used only for OS, logs etc. I'm not to up on the L1/L2 efficiencies, but 64 256 or higher L1 seems to be better for larger dir sizes. L1 should be imho increased with 1 for every 65536 objects (256 L2 dirs * 256 files in each of them), with average size of 13KiB (default) it roughly means one for each GB of cache_dir size. Depending on you maximum_object_size the averags size may be higher, but that doesn't change much. Note that for 300GiB HDD you will be using max 250, more probably 200 and some ppl would advise 150GiB of cache. Leave some space for metadata and some for reserve - filesystems may benefit of it. For a quad or higher CPU machine, you may do well to have multiple Squid running (one per 2 CPUs or so). One squid doing the caching on the 300GB drives and one on the smaller ~100 GB drives (to get around a small bug where mismatched AUFS dirs cause starvation in small dir), peered together with no-proxy option to share info without duplicating cache. Maybe even one master squid with big memory_cache, accessed by clients, having those with cache_dir's (zero cache_mem) as parents and never_direct set to on, and off only for files you surely don't cache e.g. the default query acl, if you didn't comment that out. I'm currently not sure if we can ask the master squid to fetch directly everything it surely won't cache... -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. M$ Win's are shit, do not use it !
Re: [squid-users] Config suggestion
On Tue, 17 Mar 2009 09:54:00 +0100, Matus UHLAR - fantomas uh...@fantomas.sk wrote: [cut] is that one quad-core with hyperthreading, two quad-cores without HT or two dual-cores with HT? We apparently should count HT CPU's as one, not two. 2 Xeon Quad-cores (4 cores per/processor, 8 total), no HT... [cut] total used free sharedbuffers cached Mem: 32148 2238 29910 0244 823 -/+ buffers/cache: 1169 30978 Swap:15264 0 15264 swap is quite useless here I'd say... Uptime was 1/2 min. Look at it now: $ free -m total used free sharedbuffers cached Mem: 32151 31996155 0 1891 24108 -/+ buffers/cache: 5996 26155 Swap:15264 6 15258 [cut] I'd say that the 73.5 Gb disk should be used only for OS, logs etc. I did it. [cut] I'm not to up on the L1/L2 efficiencies, but 64 256 or higher L1 seems to be better for larger dir sizes. OK, I will try... [cut] Note that for 300GiB HDD you will be using max 250, more probably 200 and some ppl would advise 150GiB of cache. Leave some space for metadata and some for reserve - filesystems may benefit of it. I always configure (to use) only 80% HDD... [cut] For a quad or higher CPU machine, you may do well to have multiple Squid running (one per 2 CPUs or so). One squid doing the caching on the 300GB drives and one on the smaller ~100 GB drives (to get around a small bug where mismatched AUFS dirs cause starvation in small dir), peered together with no-proxy option to share info without duplicating cache. Cool! Thanks... -- Herbert
Re: [squid-users] Config suggestion
On Tue, 17 Mar 2009 12:58:08 +1200 (NZST), Amos Jeffries squ...@treenet.co.nz wrote: [cut] You have 5 physical disks by the looks of it. Best usage of those is to split the cache_dir one per disk (sharing a disk leads to seek clashes). OK, I will disable LVM and try it. I'm not to up on the L1/L2 efficiencies, but 64 256 or higher L1 seems to be better for larger dir sizes. OK... For a quad or higher CPU machine, you may do well to have multiple Squid running (one per 2 CPUs or so). One squid doing the caching on the 300GB drives and one on the smaller ~100 GB drives (to get around a small bug where mismatched AUFS dirs cause starvation in small dir), peered together with no-proxy option to share info without duplicating cache. 4 Squid's, 1 disk per/Squid proc. and a cache-peer config... Sounds good. [cut] Absolutely minimal swapping of memory. Decreased to 2GiB, the rule in faq/wiki about x% cache_dir (disk) should be y% cache_mem seems confused to me. -- Herbert
[squid-users] Config suggestion
Hardware (only running Squid): # cat /proc/cpuinfo | egrep -i xeon | uniq model name : Intel(R) Xeon(R) CPU E5405 @ 2.00GHz # cat /proc/cpuinfo | egrep -i xeon | wc -l 8 # free -m total used free sharedbuffers cached Mem: 32148 2238 29910 0244823 -/+ buffers/cache: 1169 30978 Swap:15264 0 15264 # lsscsi [0:0:0:0]diskMAXTOR ATLAS15K2_73WLS JNZH /dev/sda [0:0:1:0]diskSEAGATE ST3300655LW 0003 /dev/sdb [0:0:4:0]diskSEAGATE ST3146807LC 0007 /dev/sdc [3:0:0:0]diskSEAGATE ST3300655SS 0004 /dev/sdd [3:0:1:0]diskSEAGATE ST3300655SS 0004 /dev/sde # fdisk -l | grep GB Disk /dev/sda: 73.5 GB, 73557090304 bytes Disk /dev/sdb: 300.0 GB, 3000 bytes Disk /dev/sdc: 146.8 GB, 146815737856 bytes Disk /dev/sdd: 300.0 GB, 3000 bytes Disk /dev/sde: 300.0 GB, 3000 bytes # lspci | grep -Ei 'sas|scsi' 04:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1064ET PCI-Express Fusion-MPT SAS (rev 04) 06:02.0 SCSI storage controller: Adaptec ASC-29320LP U320 (rev 03) # uname -srm Linux 2.6.27.7 x86_64 Squid: # squid -v Squid Cache: Version 3.0.STABLE13 configure options: '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--libexecdir=/usr/libexec' '--datadir=/usr/share/squid' '--sysconfdir=/etc/squid' '--libdir=/usr/lib' '--includedir=/usr/include' '--mandir=/usr/man' '--localstatedir=/var' '--enable-async-io' '--with-pthreads' '--enable-xmalloc-statistics' '--enable-storeio=aufs' '--enable-removal-policies' '--enable-err-languages=English Portuguese' '--enable-linux-netfilter' '--disable-wccp' '--disable-wccpv2' '--disable-ident-lookups' '--enable-snmp' '--enable-kill-parent-hack' '--enable-delay-pools' '--enable-follow-x-forwarded-for' '--with-large-files' '--with-filedescriptors=65536' 'CFLAGS= -march=native' 'CXXFLAGS= -march=native' # cat /etc/squid/squid.conf | grep -E cache_'mem|dir'\ cache_mem 8192 MB cache_dir aufs /var/cache/proxy/cache1 102400 16 256 cache_dir aufs /var/cache/proxy/cache2 102400 16 256 cache_dir aufs /var/cache/proxy/cache3 102400 16 256 cache_dir aufs /var/cache/proxy/cache4 102400 16 256 cache_dir aufs /var/cache/proxy/cache5 102400 16 256 cache_dir aufs /var/cache/proxy/cache6 102400 16 256 cache_dir aufs /var/cache/proxy/cache7 102400 16 256 cache_dir aufs /var/cache/proxy/cache8 102400 16 256 # cat /etc/fstab | grep proxy /dev/vg00/cache /var/cache/proxy ext3defaults 1 2 Yes, I know, LVM, ext3 and aufs are bad ideas... I'm particularly interested in a better cache_dir configuration (maximizing disk's usage) and the correct cache_mem parameter to this hardware. (and others possible/useful tips) Thanks, -- Herbert
Re: [squid-users] Config suggestion
Hardware (only running Squid): # cat /proc/cpuinfo | egrep -i xeon | uniq model name : Intel(R) Xeon(R) CPU E5405 @ 2.00GHz # cat /proc/cpuinfo | egrep -i xeon | wc -l 8 # free -m total used free sharedbuffers cached Mem: 32148 2238 29910 0244823 -/+ buffers/cache: 1169 30978 Swap:15264 0 15264 # lsscsi [0:0:0:0]diskMAXTOR ATLAS15K2_73WLS JNZH /dev/sda [0:0:1:0]diskSEAGATE ST3300655LW 0003 /dev/sdb [0:0:4:0]diskSEAGATE ST3146807LC 0007 /dev/sdc [3:0:0:0]diskSEAGATE ST3300655SS 0004 /dev/sdd [3:0:1:0]diskSEAGATE ST3300655SS 0004 /dev/sde # fdisk -l | grep GB Disk /dev/sda: 73.5 GB, 73557090304 bytes Disk /dev/sdb: 300.0 GB, 3000 bytes Disk /dev/sdc: 146.8 GB, 146815737856 bytes Disk /dev/sdd: 300.0 GB, 3000 bytes Disk /dev/sde: 300.0 GB, 3000 bytes # lspci | grep -Ei 'sas|scsi' 04:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1064ET PCI-Express Fusion-MPT SAS (rev 04) 06:02.0 SCSI storage controller: Adaptec ASC-29320LP U320 (rev 03) # uname -srm Linux 2.6.27.7 x86_64 Squid: # squid -v Squid Cache: Version 3.0.STABLE13 configure options: '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--libexecdir=/usr/libexec' '--datadir=/usr/share/squid' '--sysconfdir=/etc/squid' '--libdir=/usr/lib' '--includedir=/usr/include' '--mandir=/usr/man' '--localstatedir=/var' '--enable-async-io' '--with-pthreads' '--enable-xmalloc-statistics' '--enable-storeio=aufs' '--enable-removal-policies' '--enable-err-languages=English Portuguese' '--enable-linux-netfilter' '--disable-wccp' '--disable-wccpv2' '--disable-ident-lookups' '--enable-snmp' '--enable-kill-parent-hack' '--enable-delay-pools' '--enable-follow-x-forwarded-for' '--with-large-files' '--with-filedescriptors=65536' 'CFLAGS= -march=native' 'CXXFLAGS= -march=native' # cat /etc/squid/squid.conf | grep -E cache_'mem|dir'\ cache_mem 8192 MB cache_dir aufs /var/cache/proxy/cache1 102400 16 256 cache_dir aufs /var/cache/proxy/cache2 102400 16 256 cache_dir aufs /var/cache/proxy/cache3 102400 16 256 cache_dir aufs /var/cache/proxy/cache4 102400 16 256 cache_dir aufs /var/cache/proxy/cache5 102400 16 256 cache_dir aufs /var/cache/proxy/cache6 102400 16 256 cache_dir aufs /var/cache/proxy/cache7 102400 16 256 cache_dir aufs /var/cache/proxy/cache8 102400 16 256 # cat /etc/fstab | grep proxy /dev/vg00/cache /var/cache/proxy ext3defaults 1 2 Yes, I know, LVM, ext3 and aufs are bad ideas... I'm particularly interested in a better cache_dir configuration (maximizing disk's usage) and the correct cache_mem parameter to this hardware. (and others possible/useful tips) You have 5 physical disks by the looks of it. Best usage of those is to split the cache_dir one per disk (sharing a disk leads to seek clashes). I'm not to up on the L1/L2 efficiencies, but 64 256 or higher L1 seems to be better for larger dir sizes. For a quad or higher CPU machine, you may do well to have multiple Squid running (one per 2 CPUs or so). One squid doing the caching on the 300GB drives and one on the smaller ~100 GB drives (to get around a small bug where mismatched AUFS dirs cause starvation in small dir), peered together with no-proxy option to share info without duplicating cache. Absolutely minimal swapping of memory. Amos
