Malformed varnishncsa output
I'm getting a lot of lines line this from varnishncsa: 10.151.1.1 - - [25/Nov/2008:19:11:14 +] GET http:// vectordevhttp://vectordev/devsite/diagrams/tn-rev1.png HTTP/1.1 200 60834 - curl/7.16.3 (amd64-portbld-freebsd6.3) libcurl/7.16.3 OpenSSL/0.9.7e zlib/1.2.3 Notice the duplicated http://vectordevhttp://vectordev part after GET. Should I just change it to not print out lp-df_Host in varnishncsa.c or something? Thanks, Skye ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
varnishncsa and liblogging
Hello again, I'm currently working on adding liblogging (reliable syslog over BEEP) support to varnishncsa. http://www.liblogging.org/ Is this something that the project would be interested in adding to trunk when it's done? I presume it would need to be wrapped in a configure --with-liblogging option (default off) Thanks, Skye ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: purge.hash in trunk
Oh I see, right, it matches against the whole concatenated hash string.
Why is it that this purges all files from w0011.example.com
purge.hash w0011.*#.*
But this does nothing?
purge.hash w0011.*#.*#
Flipping the #'s around works:
purge.hash #w0011.*#.*
This also has no effect, is + not supported in the regex?
purge.hash w0011.+#.+
purge.hash w0011.+#.+#
purge.hash #w0011.+#.+
But none of these do anything:
purge.hash #w0011.*#.*html
purge.hash w0011.*#.*html
purge.hash w0011.*#.*html#
So I'm somewhat confused again. I haven't messed with vcl_hash at all.
Thanks,
Skye
On 14-Jul-08, at 12:33 PM, Poul-Henning Kamp wrote:
In message [EMAIL PROTECTED], Skye Poier
Nott writes
:
Updated to r2945 today, is purge.hash changed or broken?
This works:
purge.url .
purge.hash .
But none of these have any effect:
purge.hash .#.#
This would only purge single character urls on single character
host names.
Try:
purge.hash .*#.*#
for wider matching
(The last trunk rev I was using had #host#path but now it's
host#path# ? It would be nice if 'help purge.hash' mentioned the
current expected format)
The expected format is what vcl_hash {} set up, and since the
user can redefine that function, a constant help message would
have good chances of being confusing.
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by
incompetence.
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: purge.hash in trunk
On 14-Jul-08, at 2:58 PM, Poul-Henning Kamp wrote:
Because the default vcl_hash says:
sub vcl_hash {
set req.hash += req.url;
if (req.http.host) {
set req.hash += req.http.host;
} else {
set req.hash += server.ip;
}
hash;
}
*blush* that would explain it, this whole time I've been trying to
'purge host#path#' when its supposed to be 'purge path#host#'
Thanks,
Skye
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
hash.purge
Does hash.purge take a regex in the host section as well, or just the url section? ie is this valid (delete all jpg's for all *.bar.com hosts) hash.purge #.+\.bar\.com#\.jpg$ Thanks, Skye ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Strategy for large cache sets
I want to deploy Varnish with very large cache sizes (200GB or more) for large, long lived file sets. Is it more efficient to use large swap or large mmap in this scenario? According to the FreeBSD lists, even 20GB of swap requires 200MB of kern.maxswzone just to keep track of it, so it doesn't seem like that will scale too well. Is one or the other method better for many small files vs less many big files? Thanks again... when I'm a Varnish expert I'll help the newbs :) Skye ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Strategy for large cache sets
Thanks, that was my hunch. I'll let you know how it goes when I have some performance metrics. Skye On 1-Jul-08, at 11:24 AM, Poul-Henning Kamp wrote: In message [EMAIL PROTECTED], Skye Poier Nott writes : I want to deploy Varnish with very large cache sizes (200GB or more) for large, long lived file sets. Is it more efficient to use large swap or large mmap in this scenario? We have no real-world experience with content of that size, so the answer is: we don't know. Off the bat, I would think files would be better, for exactly the reson you cite: the swap management is pretty thirsty after metadata. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 [EMAIL PROTECTED] | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Trunk build errors
Weird, any thoughts? Or am I just asking for trouble with -trunk :)
$ svn update (last night's)
$ make clean configure make
mkdir .libs
gcc -DVARNISH_STATE_DIR=\/usr/local/var/varnish\ -g -O2 -o .libs/
varnishd varnishd-cache_acceptor.o varnishd-cache_acceptor_epoll.o
varnishd-cache_acceptor_poll.o varnishd-cache_acceptor_kqueue.o
varnishd-cache_backend.o varnishd-cache_ban.o varnishd-cache_center.o
varnishd-cache_cli.o varnishd-cache_dir_random.o varnishd-
cache_dir_simple.o varnishd-cache_expire.o varnishd-cache_fetch.o
varnishd-cache_hash.o varnishd-cache_http.o varnishd-cache_httpconn.o
varnishd-cache_main.o varnishd-cache_panic.o varnishd-cache_pool.o
varnishd-cache_pipe.o varnishd-cache_response.o varnishd-
cache_session.o varnishd-cache_synthetic.o varnishd-cache_vary.o
varnishd-cache_vcl.o varnishd-cache_vrt.o varnishd-cache_vrt_acl.o
varnishd-cache_vrt_re.o varnishd-cache_vrt_esi.o varnishd-cache_ws.o
varnishd-hash_simple_list.o varnishd-hash_classic.o varnishd-
instance.o varnishd-mgt_child.o varnishd-mgt_cli.o varnishd-
mgt_event.o varnishd-mgt_param.o varnishd-mgt_vcc.o varnishd-rfc2616.o
varnishd-shmlog.o varnishd-stevedore.o varnishd-storage_file.o
varnishd-storage_malloc.o varnishd-tcp.o varnishd-varnishd.o -Wl,--
export-dynamic ../../lib/libvarnish/.libs/libvarnish.so ../../lib/
libvarnishcompat/.libs/libvarnishcompat.so ../../lib/libvcl/.libs/
libvcl.so -lthr -lm -Wl,--rpath -Wl,/usr/local/lib
varnishd-cache_http.o(.text+0x5a0): In function `http_GetHdr':
../../include/vct.h:47: undefined reference to `vct_typtab'
varnishd-cache_http.o(.text+0x65e): In function `http_GetHdrField':
../../include/vct.h:49: undefined reference to `vct_typtab'
varnishd-cache_http.o(.text+0x679):../../include/vct.h:49: undefined
reference to `vct_typtab'
varnishd-cache_http.o(.text+0x6a5):../../include/vct.h:49: undefined
reference to `vct_typtab'
varnishd-cache_http.o(.text+0x6cd):../../include/vct.h:49: undefined
reference to `vct_typtab'
varnishd-cache_http.o(.text+0x737):../../include/vct.h:47: more
undefined references to `vct_typtab' follow
*** Error code 1
$ uname -a
FreeBSD XX 7.0-RELEASE FreeBSD 7.0-RELEASE #0: Sun Feb 24 10:35:36 UTC
2008 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/GENERIC
amd64
$
$ gcc --version
gcc (GCC) 4.2.1 20070719 [FreeBSD]
Copyright (C) 2007 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There
is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
PURPOSE.
$ srcgrep vct_typtab~/varnish/
varnish-cache
include/vct.h:extern unsigned char vct_typtab[256];
include/vct.h:return (vct_typtab[x] (y));
lib/libvarnish/vct.c:unsigned char vct_typtab[256] = {
Today's svn barfs on configure:
configure: creating ./config.status
config.status: creating Makefile
config.status: creating bin/Makefile
config.status: creating bin/varnishadm/Makefile
config.status: creating bin/varnishd/Makefile
config.status: creating bin/varnishlog/Makefile
config.status: creating bin/varnishhist/Makefile
config.status: creating bin/varnishncsa/Makefile
config.status: creating bin/varnishreplay/Makefile
config.status: creating bin/varnishstat/Makefile
config.status: error: cannot find input file: bin/varnishtest/
Makefile.in
zsh: exit 1 configure
Thanks,
Skye
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
A couple of simple questions...
Hello... a few beginner questions:
1. When you restart Varnish the cache is always purged?
2. If I have a section like this in vcl_fetch, is the default_ttl
parameter basically ignored? (assuming default_ttl is less than the
value below). So in other words, this VCL code enforces a minimum TTL?
if (obj.ttl 36h) {
set obj.ttl = 36h;
}
3. When I want to use Varnish in a WAN transparent accelerator mode,
if I don't specify any backend default is the default to fetch the
document from the hostname in the URL?
Thanks,
Skye
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Varnish dumping cache?
Hello, I'm a new Varnish user (coming from Squid - glad to be rid of it) but I'm seeing some strange behaviour when I test load a Varnish server. I have a configuration of 4 machines with 250 http client test threads each requesting a set of 46 files from 2,000 virtual hosts (92,000 objects total). But while it's running, I see the following: # while ((1)); do date varnishstat -1 | grep n_object sleep 5 done n_object will climb steadily and then all of a sudden, it all goes away: n_objecthead86765 . N struct objecthead Mon Apr 14 11:51:06 PDT 2008 n_object86780 . N struct object n_objecthead86782 . N struct objecthead Mon Apr 14 11:51:11 PDT 2008 n_object 358 . N struct object n_objecthead 358 . N struct objecthead Mon Apr 14 11:51:16 PDT 2008 n_object 358 . N struct object n_objecthead 358 . N struct objecthead I don't see anything in varnishlog -i Error when this happens but I'm not sure where to look. Here's my config: param.show 200 783 user nobody (65534) groupnogroup (65533) default_ttl 86400 [seconds] thread_pools 1 [pools] thread_pool_max 1000 [threads] thread_pool_min 1 [threads] thread_pool_timeout 120 [seconds] overflow_max 100 [%] http_workspace 8192 [bytes] sess_timeout 5 [seconds] pipe_timeout 60 [seconds] send_timeout 600 [seconds] auto_restart on [bool] fetch_chunksize 128 [kilobytes] sendfile_threshold unlimited [bytes] vcl_traceoff [bool] listen_address :80 listen_depth 1024 [connections] srcaddr_hash 1049 [buckets] srcaddr_ttl 30 [seconds] backend_http11 off [bool] client_http11off [bool] ping_interval3 [seconds] storage is varnishd_storage=file,/var/cache/varnish,128G (pre-alloced) on FreeBSD 6.3-RELEASE Thanks for helping this Varnish newbie. Great daemon! Skye ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Varnish dumping cache?
Update: I ran varnishd in foreground with -d and I'm seeing these periodically, which would explain the cache invalidation... Child not responding to ping Cache child died pid=23899 status=0x9 Clean child Child cleaned start child pid 23914 Child said (2, 23914): Child starts managed to mmap 137438953472 bytes of 137438953472 Ready CLI ready Cache child died pid=23914 status=0xb Clean child Child cleaned start child pid 23916 Child said (2, 23916): Child starts managed to mmap 137438953472 bytes of 137438953472 Ready CLI ready Cache child died pid=23916 status=0xb Clean child Child cleaned start child pid 23917 Child said (2, 23917): Child starts managed to mmap 137438953472 bytes of 137438953472 Ready CLI ready Child said (2, 23917): getnameinfo = 5 ai_family not supported getnameinfo = 5 ai_family not supported I'm running varnish-1.1.2 built out of ports on amd64...? # uname -a FreeBSD 6.3-RELEASE FreeBSD 6.3-RELEASE #0: Wed Jan 16 01:43:02 UTC 2008 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/SMP amd64 Skye On 14-Apr-08, at 11:55 AM, Skye Poier Nott wrote: Hello, I'm a new Varnish user (coming from Squid - glad to be rid of it) but I'm seeing some strange behaviour when I test load a Varnish server. I have a configuration of 4 machines with 250 http client test threads each requesting a set of 46 files from 2,000 virtual hosts (92,000 objects total). But while it's running, I see the following: # while ((1)); do date varnishstat -1 | grep n_object sleep 5 done n_object will climb steadily and then all of a sudden, it all goes away: n_objecthead86765 . N struct objecthead Mon Apr 14 11:51:06 PDT 2008 n_object86780 . N struct object n_objecthead86782 . N struct objecthead Mon Apr 14 11:51:11 PDT 2008 n_object 358 . N struct object n_objecthead 358 . N struct objecthead Mon Apr 14 11:51:16 PDT 2008 n_object 358 . N struct object n_objecthead 358 . N struct objecthead I don't see anything in varnishlog -i Error when this happens but I'm not sure where to look. Here's my config: param.show 200 783 user nobody (65534) groupnogroup (65533) default_ttl 86400 [seconds] thread_pools 1 [pools] thread_pool_max 1000 [threads] thread_pool_min 1 [threads] thread_pool_timeout 120 [seconds] overflow_max 100 [%] http_workspace 8192 [bytes] sess_timeout 5 [seconds] pipe_timeout 60 [seconds] send_timeout 600 [seconds] auto_restart on [bool] fetch_chunksize 128 [kilobytes] sendfile_threshold unlimited [bytes] vcl_traceoff [bool] listen_address :80 listen_depth 1024 [connections] srcaddr_hash 1049 [buckets] srcaddr_ttl 30 [seconds] backend_http11 off [bool] client_http11off [bool] ping_interval3 [seconds] storage is varnishd_storage=file,/var/cache/varnish,128G (pre- alloced) on FreeBSD 6.3-RELEASE Thanks for helping this Varnish newbie. Great daemon! Skye ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
