Re: log format different and CAPTURE_LEN settings
Hi Willy, On 10-08-2012 04:56, Willy Tarreau wrote: Hi Aleks, On Fri, Aug 10, 2012 at 12:56:18AM +0200, Aleksandar Lazic wrote: Hi will, On 09-08-2012 19:21, Willy Tarreau wrote: On Thu, Aug 09, 2012 at 06:57:16PM +0200, Aleksandar Lazic wrote: [snip] after the first minute the log is now as defined ;-) Thanks guys, patch applied. how about the CAPTURE_LEN setting. ### After rebuild I still get the warning that the I can only capture 63 bytes. I have change include/common/defaults.h to ### // reserved buffer space for header capture #ifndef CAPTURE_LEN #define CAPTURE_LEN 64 #endif ### and rebuild it. ### It's unclear to me what is causing you an issue with this one, because it's only used for cookie captures now (you know, the request and response cookie specified in capture cookie). Also, you shouldn't set it too high because this memory is allocated for each session. I think it's a classical Bug. In the Makefile is a comment like ### Memory usage tuning # If small memory footprint is required, you can reduce the buffer size. There # are 2 buffers per concurrent session, so 16 kB buffers will eat 32 MB memory # with 1000 concurrent sessions. Putting it slightly lower than a page size # will prevent the additional parameters to go beyond a page. 8030 bytes is # exactly 5.5 TCP segments of 1460 bytes and is generally good. Useful tuning # macros include : #SYSTEM_MAXCONN, BUFSIZE, MAXREWRITE, REQURI_LEN, CAPTURE_LEN. # Example: SMALL_OPTS = -DBUFSIZE=8030 -DMAXREWRITE=1030 -DSYSTEM_MAXCONN=1024 SMALL_OPTS = ### If you set -DCAPTURE_LEN=xxx nothing will happen, as far as I have understand. Best regards Aleks
log format different and CAPTURE_LEN settings
Hi,
I wanted to add the uniq-id logging to the http-log format, I just
copied the format
string from src/log.c but I got different log entries. (see below)
I also needed to capture more then 63 bytes so I have build HAProxy
like this
make TARGET=linux26 USE_LINUX_SPLICE=1 USE_STATIC_PCRE=1 # and added
SMALL_OPTS = -DCAPTURE_LEN=256
After rebuild I still get the warning that the I can only capture 63
bytes.
I have change include/common/defaults.h to
###
// reserved buffer space for header capture
#ifndef CAPTURE_LEN
#define CAPTURE_LEN 64
#endif
###
and rebuild it.
default format %Ci:%Cp [%t] %f %b/%s %Tq/%Tw/%Tc/%Tr/%Tt %st %B
%cc %cs %tsc
%ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r
my log-format %Ci:%Cp\ [%t]\ %f\ %b/%s\ %Tq/%Tw/%Tc/%Tr/%Tt\ %st\ %B\
%cc\ %cs\ %tsc\
%ac/%fc/%bc/%sc/%rc\ %ID\ %sq/%bq\ %hr\ %hs\ %{+Q}r
Output of default with original CAPTURE_LEN
Aug 8 15:54:00 localhost.localdomain haproxy[7295]:
188.165.15.140:49534 [08/Aug/2012:15:53:58.819]
fe_panomax delivery/nginx 0/0/0/1563/1563 200 214 - - 2/2/0/0/0 32
0/0 {Mozilla/4.0 (compatible;)}
POST REQUEST HTTP/1.1
Output of my log-format with new CAPTURE_LEN
Aug 8 16:03:55 localhost.localdomain haproxy[9810]:
188.165.15.140:42660 [08/Aug/2012:16:03:54.455]
fe_panomax delivery/- 0/0/0/1344/+1344 200 +128 - - 5/5/1/0/0 143
0/0 POST REQUEST HTTP/1.1
now original with new CAPTURE_LEN
Aug 8 16:11:42 localhost.localdomain haproxy[15396]:
188.165.15.140:56583 [08/Aug/2012:16:11:42.199]
fe_panomax delivery/- 0/0/0/716/+716 200 +128 - - 4/4/1/0/0 0/0
POST REQUEST HTTP/1.1
now original with original CAPTURE_LEN = currently running
Aug 8 16:24:52 localhost.localdomain haproxy[22350]:
188.165.15.140:49382 [08/Aug/2012:16:24:50.269]
fe_panomax delivery/- 0/0/0/2690/+2690 200 +128 - - 5/5/1/0/0 0/0
POST REQUEST HTTP/1.1
Currently running
haproxy-1.5-dev11# ./haproxy -vv
HA-Proxy version 1.5-dev11 2012/06/04
Copyright 2000-2012 Willy Tarreau w...@1wt.eu
Build options :
TARGET = linux26
CPU = generic
CC = gcc
CFLAGS = -O2 -g -fno-strict-aliasing
OPTIONS = USE_LINUX_SPLICE=1 USE_STATIC_PCRE=1
Default settings :
maxconn = 2000, bufsize = 16384, maxrewrite = 8192, maxpollevents =
200
Encrypted password support via crypt(3): yes
Available polling systems :
sepoll : pref=400, test result OK
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 4 (4 usable), will use sepoll.
my config:
grep -v '^(#|$)' /home/al/download/haproxy/panomax.cfg
global
daemon
pidfile /var/run/haproxy.pid
maxconn 6 # warning: this has to be 3 times the expected
value!
log 127.0.0.1 local0
ulimit-n 120022
defaults
modehttp
balance roundrobin
option dontlognull
option httplog
option http-server-close
option redispatch
option forwardfor
option http-no-delay
option httpchk GET /favicon.ico HTTP/1.0
unique-id-format %{+X}o\ %Ci:%Cp_%Ts_%rt
unique-id-header X-Unique-ID
balance source
retries 1
maxconn 2000
contimeout 5000
clitimeout 5
srvtimeout 5
frontend fe_panomax
bind IP:80
option forwardfor # add 'X-Forwarded-For: IP'
log global
capture request header User-Agent len 128
capture cookie PHPSESSID len 128
rspdel ^X-Powered-By:.*
acl stat_request url_beg /haproxy_stats
use_backend stats_backend if stat_request
acl fpm_request url_beg /fpm_status
use_backend default if fpm_request
acl host_delivery hdr_beg(host) -i delivery
use_backend delivery if host_delivery
acl host_admin hdr_beg(host) -i admin
use_backend admin if host_admin
acl host_static hdr_beg(host) -i static
use_backend static if host_static
acl host_panodata hdr_beg(host) -i panodata
use_backend panodata if host_panodata
# send everything to next stage
default_backend default
backend default
log 127.0.0.1 local1
option httplog
option httpchk GET /favicon.ico HTTP/1.0
server nginx IP:81 maxconn 500 check inter 5s fall 3
backend panodata
#127.0.0.3:8002
log global
option httplog
option httpchk GET /crossdomain.xml HTTP/1.0
server nginx IP:80 check inter 5s fall 3
backend delivery
#127.0.0.3:8002
log global
option httplog
option httpchk GET /crossdomain.xml HTTP/1.0
server nginx IP:81 check inter 5s fall 3
backend static
#127.0.0.3:8002
log global
option httplog
option httpchk GET /crossdomain.xml HTTP/1.0
server nginx IP:80 check inter 5s fall 3
backend admin
#127.0.0.3:8002
log global
option httplog
option httpchk GET /img/css-nav.gif HTTP/1.0
server nginx IP:81 check inter 5s fall 3
backend stats_backend
mode http
balance
timeout connect 4000
timeout server 3
stats uri
Re: log format different and CAPTURE_LEN settings
On Thu, Aug 09, 2012 at 03:16:07PM +0200, Aleksandar Lazic wrote:
Hi,
Hello,
[...]
As you can see I have not 'option logasap' but get the '+'-sign?!
Please can anybody help me to find the error, thanks.
Best regards
Aleks
It looks like a bug with the option unique-id-format.
Can you try this patch ?
--
William Lallemand
From 7d40e9f6d3f8f1c5ce09e264226a1e5e369d70a0 Mon Sep 17 00:00:00 2001
From: William Lallemand wlallem...@exceliance.fr
Date: Thu, 9 Aug 2012 16:41:35 +0200
Subject: [PATCH] BUG/MINOR: to_log erased with unique-id-format
curproxy-to_log was reset to LW_INIT when using unique-id-format,
so logs looked like option logasap
---
src/log.c |2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/log.c b/src/log.c
index 2a3cd16..b1f532a 100644
--- a/src/log.c
+++ b/src/log.c
@@ -309,7 +309,7 @@ void parse_logformat_string(char *str, struct proxy *curproxy, struct list *list
struct logformat_node *tmplf, *back;
int options = 0;
- curproxy-to_log = LW_INIT;
+ curproxy-to_log |= LW_INIT;
/* flush the list first. */
list_for_each_entry_safe(tmplf, back, list_format, list) {
--
1.7.9.5
Re: log format different and CAPTURE_LEN settings
Hi William, On 09-08-2012 16:52, William Lallemand wrote: On Thu, Aug 09, 2012 at 03:16:07PM +0200, Aleksandar Lazic wrote: Hi, Hello, [...] As you can see I have not 'option logasap' but get the '+'-sign?! Please can anybody help me to find the error, thanks. Best regards Aleks It looks like a bug with the option unique-id-format. Can you try this patch ? after the first minute the log is now as defined ;-) Thanks. Cheers Aleks
Re: log format different and CAPTURE_LEN settings
On Thu, Aug 09, 2012 at 06:57:16PM +0200, Aleksandar Lazic wrote: Hi William, On 09-08-2012 16:52, William Lallemand wrote: On Thu, Aug 09, 2012 at 03:16:07PM +0200, Aleksandar Lazic wrote: Hi, Hello, [...] As you can see I have not 'option logasap' but get the '+'-sign?! Please can anybody help me to find the error, thanks. Best regards Aleks It looks like a bug with the option unique-id-format. Can you try this patch ? after the first minute the log is now as defined ;-) Thanks guys, patch applied. Willy
Re: log format different and CAPTURE_LEN settings
Hi will, On 09-08-2012 19:21, Willy Tarreau wrote: On Thu, Aug 09, 2012 at 06:57:16PM +0200, Aleksandar Lazic wrote: [snip] after the first minute the log is now as defined ;-) Thanks guys, patch applied. how about the CAPTURE_LEN setting. ### After rebuild I still get the warning that the I can only capture 63 bytes. I have change include/common/defaults.h to ### // reserved buffer space for header capture #ifndef CAPTURE_LEN #define CAPTURE_LEN 64 #endif ### and rebuild it. ### Best regards Aleks
Re: log format different and CAPTURE_LEN settings
Hi Aleks, On Fri, Aug 10, 2012 at 12:56:18AM +0200, Aleksandar Lazic wrote: Hi will, On 09-08-2012 19:21, Willy Tarreau wrote: On Thu, Aug 09, 2012 at 06:57:16PM +0200, Aleksandar Lazic wrote: [snip] after the first minute the log is now as defined ;-) Thanks guys, patch applied. how about the CAPTURE_LEN setting. ### After rebuild I still get the warning that the I can only capture 63 bytes. I have change include/common/defaults.h to ### // reserved buffer space for header capture #ifndef CAPTURE_LEN #define CAPTURE_LEN 64 #endif ### and rebuild it. ### It's unclear to me what is causing you an issue with this one, because it's only used for cookie captures now (you know, the request and response cookie specified in capture cookie). Also, you shouldn't set it too high because this memory is allocated for each session. Regards, Willy
