Re: GRSEC and Varnish
Kristian Lyngstol a écrit :
On Tue, Feb 02, 2010 at 04:44:48PM +0100, Bernardf FRIT wrote:
Hi,
I'am running :
- varnishd (varnish-2.0.4)
Why not 2.0.6?
When a server is running well, I'm a bit reluctant to upgrade. Now, I'm
ok to upgrade as an attempt to fix this.
and it appears that the grsec Kernel repeatedly and unexpectedly sends
signal 11 to the varnishd child.
grsec seems to just report that a segfault occurred. SIGSEG would be a
strange signal to send in any event. You want to fetch yourself a core-dump
of this. However, before we get into that, I'd like to know what parameters
you start Varnish with, and the general setup. VCL too, if possible.
Ok, I just misunderstood the grsec report. I can't find any core dump
file in the system.
I start varnishd using /etc/init.d/varnishd with the following parameters :
# cat /etc/conf.d/varnishd
# /etc/conf.d/varnishd
# options passed to varnish on startup
# please see the varnishd man page for more options
VARNISHD_OPTS=-a 87.98.137.117:80 -f /etc/varnish/yourimmo.vcl -n
/home/varnish/yourimmo -s file,/home/varnish/cache/yourimmo,1G -T
127.0.0.1:
# arguments passed to varnishncsa
# please see the varnishncsa man page for more options
VARNISHNCSA_ARGS=-c -a -n /home/varnish/yourimmo -w
/var/log/varnish/access.log
---
# cat /etc/varnish/yourimmo.vcl
### define backends:
# ha proxy
backend ha_proxy {
.host = 127.0.0.1;
.port = 80;
}
acl purge {
localhost;
111.111.111.111;
}
### Called when a client request is received
sub vcl_recv {
### if there is a purge make sure its coming from $localhost
if (req.request == PURGE) {
if (!client.ip ~ purge) {
error 405 Not allowed.;
}
lookup;
}
# Add a unique header containing the client address
remove req.http.X-Forwarded-For;
setreq.http.X-Forwarded-For = client.ip;
# setreq.http.X-Forwarded-For = req.http.rlnclientipaddr;
# grace settings, note this is also set in vcl_fetch,
set req.grace = 600s;
if (req.http.host ~ ^(www.)?your-immo.fr$) {
set req.backend = ha_proxy;
}
### ne pas mettre en cache:
if (req.request == GET req.url ~ \.(php|html)$) {
pass;
}
if (req.request == GET req.url ~ \.(your-immo\.fr)$) {
pass;
}
### toujours mettre en cache:
if (req.request == GET req.url ~ \.(js)) {
lookup;
}
## images
if (req.request == GET req.url ~
\.(gif|jpg|jpeg|bmp|png|tiff|tif|ico|img|tga|wmf)$) {
lookup;
}
## pages statiques
if (req.request == GET req.url ~ \.(css|pdf|exe)$) {
lookup;
}
## multimedia
if (req.request == GET req.url ~
\.(svg|swf|ico|mp3|mp4|m4a|ogg|mov|avi|wmv)$) {
lookup;
}
## xml
if (req.request == GET req.url ~ \.(xml)$) {
lookup;
}
### regles a ne pas mettre en cache:
if (req.request == GET req.url ~ \/stats) {
pipe;
}
if (req.request != GET req.request != HEAD) {
pipe;
}
if (req.http.Authenticate || req.http.Authorization) {
pass;
}
### ne pas mettre en cache les sessions d'authenticfication
if (req.http.Cookie req.http.Cookie ~ authtoken=) {
pipe;
}
### parse accept encoding rulesets to make it look nice
if (req.http.Accept-Encoding) {
if (req.http.Accept-Encoding ~ gzip) {
set req.http.Accept-Encoding = gzip;
} elsif (req.http.Accept-Encoding ~ deflate) {
set req.http.Accept-Encoding = deflate;
} else {
# unkown algorithm
remove req.http.Accept-Encoding;
}
}
### Modif suite a segfault
pass;
### if it passes all these tests, do a lookup anyway;
###lookup;
### end of vcl_recv
}
### Called when an object is in the cache, its a hit.
sub vcl_hit {
if (req.request == PURGE) {
set obj.ttl = 0s;
error 200 Purged.;
}
if (!obj.cacheable) {
pass;
}
deliver;
}
### Called when the requested object was not found in the cache
sub vcl_miss {
if (req.request == PURGE) {
error 404 Not in cache.;
}
}
### Called when the requested object has been retrieved from the
backend, or the request to the backend has failed
sub vcl_fetch {
## If the request to the backend returns a code other than 200, restart
the loop
## If the number of restarts reaches the value of the parameter
max_restarts,
## the request will be error'ed. max_restarts defaults to 4. This
prevents
## an eternal loop in the event that, e.g., the object does not exist
at all.
## this rule also allows for 301's and 302's redirects
GRSEC and Varnish
Hi, I'am running : - varnishd (varnish-2.0.4) - linux kernel 2.6.27.10-grsec--grs-ipv4-64 and it appears that the grsec Kernel repeatedly and unexpectedly sends signal 11 to the varnishd child. .../... Feb 2 12:01:02 XX varnishd[17111]: segfault at 1000 ip 0043abf0 sp 47d89ae0 error 4 in varnishd[40+5] Feb 2 12:01:02 XX grsec: From 82.67.39.69: signal 11 sent to /usr/sbin/varnishd[varnishd:17111] uid/euid:65534/65534 gid/egid:65534/65534, parent /usr/sbin/varnishd[varnishd:28927] uid/euid:0/0 gid/egid:0/0 Feb 2 13:45:44 XX varnishd[22187]: segfault at f5000 ip 0043abf0 sp 48538ae0 error 4 in varnishd[40+5] Feb 2 13:45:44 XX grsec: From 80.13.19.228: signal 11 sent to /usr/sbin/varnishd[varnishd:22187] uid/euid:65534/65534 gid/egid:65534/65534, paren t /usr/sbin/varnishd[varnishd:28927] uid/euid:0/0 gid/egid:0/0 Feb 2 13:54:57 XX varnishd[22236]: segfault at 1000 ip 0043abf0 sp 45445ae0 error 4 in varnishd[40+5] Feb 2 13:54:57 XX grsec: From 80.13.19.228: signal 11 sent to /usr/sbin/varnishd[varnishd:22236] uid/euid:65534/65534 gid/egid:65534/65534, paren t /usr/sbin/varnishd[varnishd:28927] uid/euid:0/0 gid/egid:0/0 Feb 2 14:13:41 XX varnishd[22595]: segfault at ae000 ip 0043abf0 sp 40ff4ae0 error 4 in varnishd[40+5] Feb 2 14:13:41 XX grsec: From 83.145.80.130: signal 11 sent to /usr/sbin/varnishd[varnishd:22595] uid/euid:65534/65534 gid/egid:65534/65534, pare nt /usr/sbin/varnishd[varnishd:28927] uid/euid:0/0 gid/egid:0/0 Feb 2 14:31:08 XX varnishd[23547]: segfault at 1000 ip 0043abf0 sp 45b40ae0 error 4 in varnishd[40+5] Feb 2 14:31:08 XX grsec: From 81.49.118.48: signal 11 sent to /usr/sbin/varnishd[varnishd:23547] uid/euid:65534/65534 gid/egid:65534/65534, paren t /usr/sbin/varnishd[varnishd:28927] uid/euid:0/0 gid/egid:0/0 Feb 2 16:19:05 XX varnishd[24256]: segfault at f7000 ip 0043abf0 sp 473bcae0 error 4 in varnishd[40+5] Feb 2 16:19:05 XX grsec: From 192.196.142.20: signal 11 sent to /usr/sbin/varnishd[varnishd:24256] uid/euid:65534/65534 gid/egid:65534/65534, par ent /usr/sbin/varnishd[varnishd:28927] uid/euid:0/0 gid/egid:0/0 Then the parent varnishd process starts immediately a new child process which lasts some time. Is there any way to fix this. Remocve the GRSEC kernel ? Upgrade the kernel ? Varnish ? or whatever ? Thanks in advance. -- Bernard FRIT ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Strange different behavior
Hi,
I'm facing a pretty strange issue. URLs not supposed to be cached are in
cache when
requested by firefox but not by lwp.
I'm suspecting cookie management... but I don't realy know how to avoid
caching
these URLs. Any help would be apreciated.
URL like XX.html are XXX.php rewrited.
In vcl I have :
if (req.request == GET req.url ~ \.(html|php)) {
pass;
}
And when requesting pages :
1. Under Linux/GET
# GET -m HEAD
_*http://XXX/annonces-immobilier/appartement-roanne/11.html*_
200 OK
Cache-Control: no-store, no-cache, must-revalidate, post-check=0,
pre-check=0
Connection: close
_*Date: Wed, 16 Dec 2009 07:26:46 GMT*_
Pragma: no-cache
Via: 1.1 varnish
Age: 0
Server: Apache-NSCA
Vary: Accept-Encoding,User-Agent
Content-Type: text/html
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Client-Date: Wed, 16 Dec 2009 07:26:20 GMT
Client-Response-Num: 1
Set-Cookie: PHPSESSID=36e4f91a2e8ae9b1aa00d819c06e03a8; path=/
Set-Cookie: DYNSRV=s0; path=/
_*X-Cache: MISS*_
X-Served-By: Server 203
X-Varnish: 1411446250
2. Under Windows/Firefox
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Vary: Accept-Encoding,User-Agent
Content-Encoding: gzip
Content-Type: text/html
Content-Length: 11758
X-Cacheable: YES
_*Date: Wed, 16 Dec 2009 07:30:21 GMT*_
X-Varnish: 1411446251 1411446175
Age: 570
Via: 1.1 varnish
Connection: keep-alive
X-Served-By: Server 203
_*X-Cache: HIT*_
X-Cache-Hits: 1
Server: Apache-NSCA
3. Under Linux/GET
# GET -m HEAD
_*http://XXX/annonces-immobilier/appartement-roanne/11.html*_
200 OK
Cache-Control: no-store, no-cache, must-revalidate, post-check=0,
pre-check=0
Connection: close
_*Date: Wed, 16 Dec 2009 07:31:34 GMT*_
Pragma: no-cache
Via: 1.1 varnish
Age: 0
Server: Apache-NSCA
Vary: Accept-Encoding,User-Agent
Content-Type: text/html
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Client-Date: Wed, 16 Dec 2009 07:31:08 GMT
Client-Response-Num: 1
Set-Cookie: PHPSESSID=fd043b9a9207d7137f79777970a54735; path=/
Set-Cookie: DYNSRV=s0; path=/
_*X-Cache: MISS*_
X-Served-By: Server 203
Regards
--
Bernard FRIT
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Strange different behavior
Hi,
I'm facing a pretty strange issue. URLs not supposed to be cached are in
cache when
requested by firefox but not by lwp.
I'm suspecting cookie management... but I don't realy know how to avoid
caching
these URLs. Any help would be apreciated.
URL like XX.html are XXX.php rewrited.
In vcl I have :
if (req.request == GET req.url ~ \.(html|php)) {
pass;
}
And when requesting pages :
1. Under Linux/GET
# GET -m HEAD
_*http://XXX/annonces-immobilier/appartement-roanne/11.html*_
200 OK
Cache-Control: no-store, no-cache, must-revalidate, post-check=0,
pre-check=0
Connection: close
_*Date: Wed, 16 Dec 2009 07:26:46 GMT*_
Pragma: no-cache
Via: 1.1 varnish
Age: 0
Server: Apache-NSCA
Vary: Accept-Encoding,User-Agent
Content-Type: text/html
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Client-Date: Wed, 16 Dec 2009 07:26:20 GMT
Client-Response-Num: 1
Set-Cookie: PHPSESSID=36e4f91a2e8ae9b1aa00d819c06e03a8; path=/
Set-Cookie: DYNSRV=s0; path=/
_*X-Cache: MISS*_
X-Served-By: Server 203
X-Varnish: 1411446250
2. Under Windows/Firefox
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0,
pre-check=0
Pragma: no-cache
Vary: Accept-Encoding,User-Agent
Content-Encoding: gzip
Content-Type: text/html
Content-Length: 11758
X-Cacheable: YES
_*Date: Wed, 16 Dec 2009 07:30:21 GMT*_
X-Varnish: 1411446251 1411446175
Age: 570
Via: 1.1 varnish
Connection: keep-alive
X-Served-By: Server 203
_*X-Cache: HIT*_
X-Cache-Hits: 1
Server: Apache-NSCA
3. Under Linux/GET
# GET -m HEAD
_*http://XXX/annonces-immobilier/appartement-roanne/11.html*_
200 OK
Cache-Control: no-store, no-cache, must-revalidate, post-check=0,
pre-check=0
Connection: close
_*Date: Wed, 16 Dec 2009 07:31:34 GMT*_
Pragma: no-cache
Via: 1.1 varnish
Age: 0
Server: Apache-NSCA
Vary: Accept-Encoding,User-Agent
Content-Type: text/html
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Client-Date: Wed, 16 Dec 2009 07:31:08 GMT
Client-Response-Num: 1
Set-Cookie: PHPSESSID=fd043b9a9207d7137f79777970a54735; path=/
Set-Cookie: DYNSRV=s0; path=/
_*X-Cache: MISS*_
X-Served-By: Server 203
Regards
--
Bernard FRIT
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Strange different behavior
Ooouups, sorry for the double... -- BF ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Trying to use X-Backend to select backend
Tollef Fog Heen : Hi, | I'm just wondering which solution is best : | - varnish as frontend to haproxy | - haproxy as frontend to varnish Either should work fine, but test. You could just also move all static content to a different host name and let haproxy be the dynamic one and varnish the static one. I did the tests. Varnish as frontend to haproxy is much more convenient to configure. Just a matter of minutes. On the other side haproxy as frontend to varnish is much more tricky and seems to need fine tuning as I had some timeout problems. IMHO Varnish as frontend to haproxy is actually a KISS solution. -- Bernard FRIT ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Trying to use X-Backend to select backend
Tollef Fog Heen : Thanks for your answer | I downgraded varnish to v 2.0.3 in order to use the | varnish-bereq-hosts.patch Which patch is this? http://varnish.projects.linpro.no/attachment/ticket/481/ | 1. Store the backend name into X-Backend custom header | 2. Force each request with a X-Backend header to be directed to the | stored backend name There is currently no way to look up backends by name, so you would have to write this as a series of if statements Yes it worked but didn't do the job I expected. After first request to the site I want the browser to be always directed to the same backend (due to sessionId management). I think I have to use Cookies to achieve this or just serve static content with varnish and use haproxy as backend for dynamic content. I'm just wondering which solution is best : - varnish as frontend to haproxy - haproxy as frontend to varnish -- Bernard FRIT ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
