RPM-packages of varnish-2.0.5 for RHEL and Fedora available
I have submitted varnish-2.0.5 for Fedora and Fedora EPEL, and updates to the stable releases will be requested, so they will trickle down to the stable repos in a few weeks. For RHEL, both el4 and el5 packages are now in the EPEL testing repo. For those who are too impatient to wait for stable, or want to participate in testing, you can download the package with yum: rhel5# yum --enablerepo=epel-testing update varnish ... or download the package from RedHat: http://download.fedora.redhat.com/pub/epel/testing/ Fedora packages are still pending for testing, but will be visible in a few days, I guess. If you need packages for Fedora now, try http://kojipkgs.fedoraproject.org/packages/varnish/ Bugs in the package can be reported in Red Hat's Bugzilla: http://bugzilla.redhat.com/ or to varnish-d...@projects.linpro.no. Ingvar ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Varnish stuck on stresstest/approved by real traffic
I have tried setting session_linger =50 on 2.0.4 and it seems that is solves the problem ( I wasnt able to reproduce after that) Kristian Lyngstol napsal(a): (Excessive trimming ahead. Whoohoo) On Tue, Nov 03, 2009 at 11:51:22AM +0100, Václav Bílek wrote: When testing varnish throughput and scalability I have found strange varnish behavior. What's the cpu load at that point? Also: use sess_linger. No session_linger == kaboom when things get too loaded. It's 50ms default in 2.0.5/trunk, but set to 0ms in 2.0.4 and previous. The behaviour in trunk is slightly different/better, but it's still worth using it in 2.0.4. - Kristian ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: varnish 2.0.4 questions - no IMS, no persistence cache - please help
Thanks. I checked /proc/cpuinfo and it shows intel processor. So even with Intel, we see this limitation of 340 GB. This is a serious limitation to me, since in Squid, we were using 1.5 TB of storage and i thought i could mmap and use all the space for Varnish. Any workarounds or working kernel version in linux, please let me know. mylinux version: RH4 2.6.9-89.ELsmp #1 SMP Mon Apr 20 10:33:05 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux ulimit -a: core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited file size (blocks, -f) unlimited pending signals (-i) 1024 max locked memory (kbytes, -l) 32 max memory size (kbytes, -m) unlimited open files (-n) 65535 pipe size(512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 stack size (kbytes, -s) 10240 cpu time (seconds, -t) unlimited max user processes (-u) 278528 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited cat /proc/cpufinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 23 model name : Intel(R) Xeon(R) CPU L5240 @ 3.00GHz stepping: 6 cpu MHz : 2992.505 cache size : 6144 KB physical id : 0 siblings: 2 core id : 0 cpu cores : 2 fpu : yes fpu_exception : yes cpuid level : 10 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl est tm2 cx16 xtpr lahf_lm bogomips: 5989.00 clflush size: 64 cache_alignment : 64 address sizes : 38 bits physical, 48 bits virtual power management: processor : 1 vendor_id : GenuineIntel cpu family : 6 model : 23 model name : Intel(R) Xeon(R) CPU L5240 @ 3.00GHz stepping: 6 cpu MHz : 2992.505 cache size : 6144 KB physical id : 3 siblings: 2 core id : 6 cpu cores : 2 fpu : yes fpu_exception : yes cpuid level : 10 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl est tm2 cx16 xtpr lahf_lm bogomips: 5985.03 clflush size: 64 cache_alignment : 64 address sizes : 38 bits physical, 48 bits virtual power management: processor : 2 vendor_id : GenuineIntel cpu family : 6 model : 23 model name : Intel(R) Xeon(R) CPU L5240 @ 3.00GHz stepping: 6 cpu MHz : 2992.505 cache size : 6144 KB physical id : 0 siblings: 2 core id : 1 cpu cores : 2 fpu : yes fpu_exception : yes cpuid level : 10 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl est tm2 cx16 xtpr lahf_lm bogomips: 5984.96 clflush size: 64 cache_alignment : 64 address sizes : 38 bits physical, 48 bits virtual power management: processor : 3 vendor_id : GenuineIntel cpu family : 6 model : 23 model name : Intel(R) Xeon(R) CPU L5240 @ 3.00GHz stepping: 6 cpu MHz : 2992.505 cache size : 6144 KB physical id : 3 siblings: 2 core id : 7 cpu cores : 2 fpu : yes fpu_exception : yes cpuid level : 10 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl est tm2 cx16 xtpr lahf_lm bogomips: 5985.04 clflush size: 64 cache_alignment : 64 address sizes : 38 bits physical, 48 bits virtual power management: Ganesh On Tue, Nov 10, 2009 at 1:48 AM, cripy ccr...@gmail.com wrote: GaneshKumar Natarajan writes: Tue, 20 Oct 2009 12:35:00 -0700 3. mmap storage : max i can configure is 340 GB. I was able to use only 340 GB of cache. any size after this, i would get error. child (25790) Started Pushing vcls failed: dlopen(./vcl.1P9zoqAU.so): ./vcl.1P9zoqAU.so: failed to map segment from shared object: Cannot allocate memory -- I was having this issue too. After some googling it appears this is a AMD64 Linux 2.6 issue. According to http://lists.humbug.org.au/pipermail/general/2004-July/024139.html It maybe important to note that as of the latest 2.6 kernels, Linux on the AMD64 platform can only memory map a 340GB per process. This is due mainly to a VM paging system ported from the ia32 platform that should have been left on the hillside at birth to die. I have not tested *BSD because we have not done enough research to confirm if the Linux emulation works on AMD64
Re: varnish 2.0.4 questions - no IMS, no persistence cache - please help
amd64 refers to the architecture (AKA x86_64), not the particular CPU vendor. (As a matter of fact, I was unaware of this limitation; AFAIK it does not exist in FreeBSD.) In any event, mmap()ing 340GB even on a 64GB box is a recipe for disaster; you will probably suffer death by paging if your working set is larger than RAM. If it's smaller than RAM, then, well, there's no harm in making it just under the total RAM size. --Michael On Nov 11, 2009, at 1:04 AM, GaneshKumar Natarajan wrote: Thanks. I checked /proc/cpuinfo and it shows intel processor. So even with Intel, we see this limitation of 340 GB. This is a serious limitation to me, since in Squid, we were using 1.5 TB of storage and i thought i could mmap and use all the space for Varnish. Any workarounds or working kernel version in linux, please let me know. mylinux version: RH4 2.6.9-89.ELsmp #1 SMP Mon Apr 20 10:33:05 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux ulimit -a: core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited file size (blocks, -f) unlimited pending signals (-i) 1024 max locked memory (kbytes, -l) 32 max memory size (kbytes, -m) unlimited open files (-n) 65535 pipe size(512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 stack size (kbytes, -s) 10240 cpu time (seconds, -t) unlimited max user processes (-u) 278528 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited cat /proc/cpufinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 23 model name : Intel(R) Xeon(R) CPU L5240 @ 3.00GHz stepping: 6 cpu MHz : 2992.505 cache size : 6144 KB physical id : 0 siblings: 2 core id : 0 cpu cores : 2 fpu : yes fpu_exception : yes cpuid level : 10 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl est tm2 cx16 xtpr lahf_lm bogomips: 5989.00 clflush size: 64 cache_alignment : 64 address sizes : 38 bits physical, 48 bits virtual power management: processor : 1 vendor_id : GenuineIntel cpu family : 6 model : 23 model name : Intel(R) Xeon(R) CPU L5240 @ 3.00GHz stepping: 6 cpu MHz : 2992.505 cache size : 6144 KB physical id : 3 siblings: 2 core id : 6 cpu cores : 2 fpu : yes fpu_exception : yes cpuid level : 10 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl est tm2 cx16 xtpr lahf_lm bogomips: 5985.03 clflush size: 64 cache_alignment : 64 address sizes : 38 bits physical, 48 bits virtual power management: processor : 2 vendor_id : GenuineIntel cpu family : 6 model : 23 model name : Intel(R) Xeon(R) CPU L5240 @ 3.00GHz stepping: 6 cpu MHz : 2992.505 cache size : 6144 KB physical id : 0 siblings: 2 core id : 1 cpu cores : 2 fpu : yes fpu_exception : yes cpuid level : 10 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl est tm2 cx16 xtpr lahf_lm bogomips: 5984.96 clflush size: 64 cache_alignment : 64 address sizes : 38 bits physical, 48 bits virtual power management: processor : 3 vendor_id : GenuineIntel cpu family : 6 model : 23 model name : Intel(R) Xeon(R) CPU L5240 @ 3.00GHz stepping: 6 cpu MHz : 2992.505 cache size : 6144 KB physical id : 3 siblings: 2 core id : 7 cpu cores : 2 fpu : yes fpu_exception : yes cpuid level : 10 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl est tm2 cx16 xtpr lahf_lm bogomips: 5985.04 clflush size: 64 cache_alignment : 64 address sizes : 38 bits physical, 48 bits virtual power management: Ganesh On Tue, Nov 10, 2009 at 1:48 AM, cripy ccr...@gmail.com wrote: GaneshKumar Natarajan writes: Tue, 20 Oct 2009 12:35:00 -0700 3. mmap storage : max i can configure is 340 GB. I was able to use only 340 GB of cache. any size after this, i would get error. child (25790) Started Pushing vcls failed: dlopen(./vcl.1P9zoqAU.so):
Re: varnish 2.0.4 questions - no IMS, no persistence cache - please help
Note that the linked article is from 2004. The kernels that RedHat uses are a bag of hurt, not to mention ancient. If you can upgrade to RHELl5 that may be the easiest fix (I can only assume that the mmap limitation has been removed). Perhaps RedHat has newer RHELl4 kernels in a bleeding edge repository, or perhaps Fedora/CentOS have packages that could upgrade your kernel. That being said, there may be other pratfalls in the libc on RHELl4; bad 64-bit judgment calls persist in libc to this day (e.g., MAP_32BIT). Ken On Nov 10, 2009, at 11:47 AM, Michael S. Fischer wrote: amd64 refers to the architecture (AKA x86_64), not the particular CPU vendor. (As a matter of fact, I was unaware of this limitation; AFAIK it does not exist in FreeBSD.) In any event, mmap()ing 340GB even on a 64GB box is a recipe for disaster; you will probably suffer death by paging if your working set is larger than RAM. If it's smaller than RAM, then, well, there's no harm in making it just under the total RAM size. --Michael On Nov 11, 2009, at 1:04 AM, GaneshKumar Natarajan wrote: Thanks. I checked /proc/cpuinfo and it shows intel processor. So even with Intel, we see this limitation of 340 GB. This is a serious limitation to me, since in Squid, we were using 1.5 TB of storage and i thought i could mmap and use all the space for Varnish. Any workarounds or working kernel version in linux, please let me know. mylinux version: RH4 2.6.9-89.ELsmp #1 SMP Mon Apr 20 10:33:05 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux ulimit -a: core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited file size (blocks, -f) unlimited pending signals (-i) 1024 max locked memory (kbytes, -l) 32 max memory size (kbytes, -m) unlimited open files (-n) 65535 pipe size(512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 stack size (kbytes, -s) 10240 cpu time (seconds, -t) unlimited max user processes (-u) 278528 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited cat /proc/cpufinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 23 model name : Intel(R) Xeon(R) CPU L5240 @ 3.00GHz stepping: 6 cpu MHz : 2992.505 cache size : 6144 KB physical id : 0 siblings: 2 core id : 0 cpu cores : 2 fpu : yes fpu_exception : yes cpuid level : 10 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl est tm2 cx16 xtpr lahf_lm bogomips: 5989.00 clflush size: 64 cache_alignment : 64 address sizes : 38 bits physical, 48 bits virtual power management: processor : 1 vendor_id : GenuineIntel cpu family : 6 model : 23 model name : Intel(R) Xeon(R) CPU L5240 @ 3.00GHz stepping: 6 cpu MHz : 2992.505 cache size : 6144 KB physical id : 3 siblings: 2 core id : 6 cpu cores : 2 fpu : yes fpu_exception : yes cpuid level : 10 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl est tm2 cx16 xtpr lahf_lm bogomips: 5985.03 clflush size: 64 cache_alignment : 64 address sizes : 38 bits physical, 48 bits virtual power management: processor : 2 vendor_id : GenuineIntel cpu family : 6 model : 23 model name : Intel(R) Xeon(R) CPU L5240 @ 3.00GHz stepping: 6 cpu MHz : 2992.505 cache size : 6144 KB physical id : 0 siblings: 2 core id : 1 cpu cores : 2 fpu : yes fpu_exception : yes cpuid level : 10 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl est tm2 cx16 xtpr lahf_lm bogomips: 5984.96 clflush size: 64 cache_alignment : 64 address sizes : 38 bits physical, 48 bits virtual power management: processor : 3 vendor_id : GenuineIntel cpu family : 6 model : 23 model name : Intel(R) Xeon(R) CPU L5240 @ 3.00GHz stepping: 6 cpu MHz : 2992.505 cache size : 6144 KB physical id : 3 siblings: 2 core id : 7 cpu cores : 2 fpu : yes fpu_exception : yes cpuid level : 10 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
varnish behind varnish and X-Forwarded-For
Hello, setup is a varnish behind an other varnish - don't ask ;) Is there a way to get the X-Forwarded-For from the first varnish to send it to the backend (Apache with mod_rpaf) ? I see in the varnishlog of the second varnish that there are 2 X-Forwarded-For (the client IP and the varnish IP) How would I get rid of the second X-Forward-For but not the first one (the client IP) ? Thanks, Jesse ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
