Hello,
I looked briefly in relay.c file and it seems that in the function
void relay_read_http/2 - which is called only in ssl context - the following
piece of code produces the error:
} else if ((cre->method ==
HTTP_METHOD_DELETE ||
cre->method == HTTP_METHOD_GET ||
cre->method == HTTP_METHOD_HEAD ||
cre->method == HTTP_METHOD_OPTIONS ||
cre->method ==
HTTP_METHOD_POST ||
cre->method == HTTP_METHOD_PUT ||
cre->method == HTTP_METHOD_RESPONSE) &&
strcasecmp("Content-Length", pk.key) == 0) {
/*
* Need to read data from
the client after the
* HTTP header.
* XXX What about non-standard clients not using
* the carriage return? And some browsers seem to
* include the line length in the content-length.
*/
cre->toread =
strtonum(pk.value, 0, ULLONG_MAX, &errstr);
if
(errstr) {
relay_close_http(con, 500, errstr,
0);
goto abort;
}
pk.value contains a value that cannot be converted to a number, hence the
function strtonum sets the error "invalid" in errstr, which appears in this
log message:
relayd www_ssl, session 1 (1 active), 0, 10.10.11.66 ->
127.0.0.1:8080, invalid
I think the problem is that the variable pk.value
contains whatever follows after the header Content-Length.
For example, curl
sends this header to the server:
$ curl -XPOST -k
-vhttps://server/cgi-bin/query -d'param1=val1¶m2=val2'
** SSL
handshake***
POST /cgi-bin/query HTTP/1.1
User-Agent: curl/7.27.0
Host:
server
Accept: */*
Content-Length: 23
Content-Type:
application/x-www-form-urlencoded
The code stops reading further key:value
header entries when encounters Content-Length, and any entry, like
Content-Type: application/x-www-form-urlencoded, that follows is accumulated
in pk.value, and cannot be converted to number becasue contains alfanumeric
characters
yielding the error invalid, in conversion, while pk.key remains
with value "Content-Length".
What is curious enough is that a plain http
request does not even calls this function, and that is why is working.
Bogdan
________________________________
From: Bogdan Andu <bo...@yahoo.com>
To:
Sebastian Benoit <be...@openbsd.org>; "misc@openbsd.org" <misc@openbsd.org>
Cc: "r...@openbsd.org" <r...@openbsd.org>
Sent: Thursday, November 15, 2012
9:36 AM
Subject: Re: relayd and header directives
Hello,
In the meanwhile I
have discovered the following issues:
[WITH SSL]:
1) No headers directives
are allowed - the session is reported as invalid
2)
If the POST arguments are
sent as usual, like this:
$ curl -XPOST -k -v
https://server/cgi-bin/query
-d'param1=val1¶m2=val2'
relayd reports the
session invalid:
relayd
www_ssl, session 1 (1 active), 0, 10.10.11.66 ->
127.0.0.1:8080, invalid
and
the local web server is not accessed
3) If the
POST argumenst are converted
into GET like this:
$ curl -XPOST -k -v
https://server/cgi-bin/query?param1=val1¶m2=val2'
everything work ok.
Although there are sessions reported as invalid, the dialog with local web
server
works, and the respons returns to the client
[WITHOUT SSL]
Everything
work as expected with and without header directives
So, if the
relayd does
not makes ssl offloading seems that everything work ok. I suspect
there must
be something with ssl processing.
The machine is in trunk0 setup
with link
failover in dual stack. So the relayd listens on both IPv4 and IPv6.
With or
without SSL offloading I cannot change response headers.
The local
web server
is system web server Apache 1.3 with mod_perl 1.3.
The web server
is not
chroot-ed.
dmseg follows:
OpenBSD 5.2 (GENERIC.MP) #368: Wed Aug 1
10:04:49 MDT 2012
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem
= 4292550656 (4093MB)
avail mem = 4155912192 (3963MB)
mainbus0 at root
bios0
at mainbus0: SMBIOS rev. 2.5 @ 0xcff9c000 (46 entries)
bios0: vendor Dell Inc.
version "1.4.3" date 05/15/2009
bios0: Dell Inc. PowerEdge R200
acpi0 at
bios0: rev 2
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP APIC SPCR
HPET MCFG WD__ SLIC ERST HEST BERT EINJ SSDT SSDT SSDT
acpi0: wakeup devices
PCI0(S5)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr
0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0:
Intel(R) Xeon(R) CPU E3110 @ 3.00GHz, 3000.60 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,SMX,EST,TM2,S
SSE3,CX16,xTPR,PDCM,SSE4.1,XSAVE,NXE,LONG,LAHF
cpu0: 6MB 64b/line 16-way L2
cache
cpu0: apic clock running at 333MHz
cpu1 at mainbus0: apid 1 (application
processor)
cpu1: Intel(R) Xeon(R) CPU E3110 @ 3.00GHz, 3000.21 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,SMX,EST,TM2,S
SSE3,CX16,xTPR,PDCM,SSE4.1,XSAVE,NXE,LONG,LAHF
cpu1: 6MB 64b/line 16-way L2
cache
ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 20, 24 pins
ioapic0:
misconfigured as apic 0, remapped to apid 2
acpihpet0 at acpi0: 14318179 Hz
acpimcfg0 at acpi0 addr 0xe0000000, bus 0-255
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 1 (PEX1)
acpiprt2 at acpi0: bus 2 (SBE0)
acpiprt3 at
acpi0: bus 3 (SBE4)
acpiprt4 at acpi0: bus 4 (SBE5)
acpiprt5 at acpi0: bus 5
(COMP)
acpicpu0 at acpi0: PSS
acpicpu1 at acpi0: PSS
ipmi at mainbus0 not
configured
cpu0: Enhanced SpeedStep 3000 MHz: speeds: 3000, 2667, 2333, 2000
MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel 3200/3210
Host" rev 0x01
ppb0 at pci0 dev 1 function 0 "Intel 3200/3210 PCIE" rev 0x01:
msi
pci1 at ppb0 bus 1
mpi0 at pci1 dev 0 function 0 "Symbios Logic SAS1068E"
rev 0x08: msi
scsibus0 at mpi0: 112 targets
sd0 at scsibus0 targ 0 lun 0:
<Dell, VIRTUAL DISK, 1028> SCSI3 0/direct fixed
naa.600508e000000000b29cee969c7b3107
sd0: 152064MB, 512 bytes/sector,
311427072 sectors
ppb1 at pci0 dev 28 function 0 "Intel 82801I PCIE" rev 0x02:
msi
pci2 at ppb1 bus 2
ppb2 at pci0 dev 28 function 4 "Intel 82801I PCIE" rev
0x02
pci3 at ppb2 bus 3
bge0 at pci3 dev 0 function 0 "Broadcom BCM5721" rev
0x21, BCM5750 C1 (0x4201): apic 2 int 16, address 00:25:64:3b:a6:e6
brgphy0 at
bge0 phy 1: BCM5750 10/100/1000baseT PHY, rev. 0
ppb3 at pci0 dev 28 function
5 "Intel 82801I PCIE" rev 0x02
pci4 at ppb3 bus 4
bge1 at pci4 dev 0 function
0 "Broadcom BCM5721" rev 0x21, BCM5750 C1 (0x4201): apic 2 int 17, address
00:25:64:3b:a6:e7
brgphy1 at bge1 phy 1: BCM5750 10/100/1000baseT PHY, rev. 0
uhci0 at pci0 dev 29 function 0 "Intel 82801I USB" rev 0x02: apic 2 int 21
uhci1 at pci0 dev 29 function 1 "Intel 82801I USB" rev 0x02: apic 2 int 20
uhci2 at pci0 dev 29 function 2 "Intel 82801I USB" rev 0x02: apic 2 int 21
ehci0 at pci0 dev 29 function 7 "Intel 82801I USB" rev 0x02: apic 2 int 21
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "Intel EHCI root hub" rev
2.00/1.00 addr 1
ppb4 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev
0x92
pci5 at ppb4 bus 5
vga1 at pci5 dev 5 function 0 "ATI ES1000" rev 0x02
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen
1-5 added (80x25, vt100 emulation)
radeondrm0 at vga1: apic 2 int 19
drm0 at
radeondrm0
pcib0 at pci0 dev 31 function 0 "Intel 82801IR LPC" rev 0x02
pciide0 at pci0 dev 31 function 2 "Intel 82801I SATA" rev 0x02: DMA, channel 0
configured to native-PCI, channel 1 configured to native-PCI
pciide0: using
apic 2 int 23 for native-PCI interrupt
atapiscsi0 at pciide0 channel 0 drive 1
scsibus1 at atapiscsi0: 2 targets
cd0 at scsibus1 targ 0 lun 0: <TEAC, DVD-ROM
DV28SV, D.0K> ATAPI 5/cdrom removable
cd0(pciide0:0:1): using PIO mode 4,
Ultra-DMA mode 5
usb1 at uhci0: USB revision 1.0
uhub1 at usb1 "Intel UHCI
root hub" rev 1.00/1.00 addr 1
usb2 at uhci1: USB revision 1.0
uhub2 at usb2
"Intel UHCI root hub" rev 1.00/1.00 addr 1
usb3 at uhci2: USB revision 1.0
uhub3 at usb3 "Intel UHCI root hub" rev 1.00/1.00 addr 1
isa0 at pcib0
isadma0
at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
pckbc0 at isa0
port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
mtrr: Pentium Pro MTRR support
uhub4 at uhub0 port 5 "Cypress
Semiconductor USB2 Hub" rev 2.00/0.0b addr 2
uhidev0 at uhub2 port 1
configuration 1 interface 0 "Avocent Dell 03R874" rev 1.10/1.00 addr 2
uhidev0: iclass 3/1
ukbd0 at uhidev0: 8 variable keys, 6 key codes, country
code 33
wskbd1 at ukbd0 mux 1
wskbd1: connecting to wsdisplay0
uhidev1 at
uhub2 port 1 configuration 1 interface 1 "Avocent Dell 03R874" rev 1.10/1.00
addr 2
uhidev1: iclass 3/1, 3 report ids
ums0 at uhidev1 reportid 1: 5
buttons, Z dir
wsmouse0 at ums0 mux 0
uhid0 at uhidev1 reportid 2: input=2,
output=0, feature=0
uhid1 at uhidev1 reportid 3: input=1, output=0, feature=0
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at
softraid0: 256 targets
root on sd0a (14433713b3e1a5dd.a) swap on sd0b dump on
sd0b
________________________________
From: Sebastian Benoit
<be...@openbsd.org>
To: Bogdan Andu <bo...@yahoo.com>
Cc: r...@openbsd.org
Sent: Thursday, November 15, 2012 12:47 AM
Subject: Re: relayd and header
directives
Hi,
in a quick test i could not reproduce your problem. I will
look into it a bit
more, in the meantime can you please send me the dmesg of
the machine you
are testing on?
/Benno
Hello,
I have the follwing
setup on a single machine:
RELAYD[PUBLIC IP]:443
->
WEB_SERVER[127.0.0.1]:8080
pf is disbaled for testing purposes
relayd is
configured like this (snip):
/etc/relayd.conf:
###############################
table <webhosts> { 127.0.0.1}
http
protocol
www_ssl_prot {
# header append "$REMOTE_ADDR" to
"X-Forwarded-For"
# header append "$SERVER_ADDR:$SERVER_PORT" to
"X-Forwarded-By"
# header change "Keep-Alive" to "$TIMEOUT"
# Various TCP
performance options
tcp { nodelay, sack, socket
buffer 65536, backlog
128 }
ssl { sslv3, tlsv1, ciphers "HIGH"
}
ssl session cache
disable
}
relay www_ssl {
#
Run as a SSL accelerator
listen
on $ext_addr port 443 ssl
protocol www_ssl_prot
# Forward to
hosts in the webhosts
table using a src/dst hash
forward to <webhosts>
port 8080
}
###############################
The problem is that when I want
to
append or modify a header, this results in the error below
relay
www_ssl, session 1 (1 active), 0, 10.10.11.66 -> 127.0.0.1:8080, invalid
A
failed tcpdump session looks like this:
$ sudo tcpdump -A -i lo0
port 8080
tcpdump: listening on lo0, link-type LOOP
09:15:56.710348
localhost.24156 >
localhost.8080: S 2366115149:2366115149(0) win 65535 <mss
33112,nop,nop,sackOK,nop,wscale 3,nop,nop,timestamp 611410478 0> (DF)
M.........v.....X...........
$qb.....
09:15:56.710356 localhost.8080 >
localhost.24156: S 1050504178:1050504178(0) ack 2366115150 win 16384 <mss
33112,nop,nop,sackOK,nop,wscale 3,nop,nop,timestamp 184181294 611410478>
(DF)
N..@........X...........>.k...
.b.$qb.
09:15:56.710362
localhost.24156 >
localhost.8080: . ack 1 win 8192 <nop,nop,timestamp
611410478 184181294>
(DF)
N>.k... ............^\....
$qb.
.b.
tcpdump: WARNING: compensating for
unaligned libpcap packets
09:15:56.711365 localhost.24156 > localhost.8080: F
1:1(0) ack 1 win 8192
<nop,nop,timestamp 611410478 184181294> (DF)
N>.k...
........^\....
$qb.
.b.....
09:15:56.711373 localhost.8080 > localhost.24156:
. ack 2 win
2048 <nop,nop,timestamp 184181294 611410478> (DF)
O.....................^\>.k...
.b.$qb.
09:15:56.711390 localhost.8080 >
localhost.24156: F 1:1(0) ack 2 win 2048 <nop,nop,timestamp 184181294
611410478> (DF)
O.................^\>.k...
.b.$qb.....
09:15:56.711398
localhost.24156 > localhost.8080: . ack 2 win 8192
<nop,nop,timestamp
611410478 184181294> (DF)
O>.k... ............^\....
$qb.
.b.
It seems that
after the connection is established, the
client side of the relayd instead
of
Pushing data and send at least the
HTTP header it sends the FIN flag and the
handshake of closing the
connection with local web server begins.
If all
header directives are
commented out, then everything works fine.
A successful
tcpdump session
looks like this:
$ sudo tcpdump -A -i lo0 port 8080
tcpdump:
listening on lo0, link-type LOOP
09:27:05.334568 localhost.14030 >
localhost.8080: S 2866784757:2866784757(0) win 65535 <mss
33112,nop,nop,sackOK,nop,wscale 3,nop,nop,timestamp 2152179840 0> (DF)
E..@.2@.@...........6................[.....X...........
.G......
09:27:05.334576 localhost.8080 > localhost.14030: S 3002945289:3002945289(0)
ack 2866784758 win 16384 <mss 33112,nop,nop,sackOK,nop,wscale
3,nop,nop,timestamp 669666639 2152179840> (DF)
E..@..@.@.{...........6...O
......@.1
.....X...........
'.MO.G..
09:27:05.334582
localhost.14030 >
localhost.8080: . ack 1 win 8192 <nop,nop,timestamp
2152179840 669666639>
(DF)
E..4.n@.@.]S........6.........O
}.....
.G..'.MO
tcpdump: WARNING: compensating
for unaligned libpcap packets
09:27:05.335528 localhost.14030 >
localhost.8080: P 1:199(198) ack 1 win
8192 <nop,nop,timestamp 2152179840
669666639> (DF)
q]@.@...........6.........O
.. ........
.G..'.MOPOST
/cereri/noi/cgi-bin/query?lang=ro HTTP/1.1
User-Agent:....
09:27:05.335535
localhost.8080 > localhost.14030: . ack 199 win 2023 <nop,nop,timestamp
669666639 2152179840> (DF)
.C@.@..~..........6...O
........$......
'.MO.G..POST
09:27:05.671832 localhost.8080 > localhost.14030: P
1:11455(11454) ack 199 win 2048 <nop,nop,timestamp 669666639 2152179840>
(DF)
E.,..9@.@.............6...O
........
e.....
'.MO.G..HTTP/1.1 200 OK
Date: Tue, 13 Nov 2012 07:27:05 GMT
Server
09:27:05.671851 localhost.14030 >
localhost.8080: . ack 11455 win 6760
<nop,nop,timestamp 2152179840
669666639>
(DF)
E..4..@.@.93........6.........{....h.......
.G..'.MO
09:27:05.673411
localhost.8080 > localhost.14030: P 11455:11460(5) ack 199 win 2048
<nop,nop,timestamp 669666640 2152179840> (DF)
..@.@.............6...{................
'.MP.G..0
/cer
09:27:05.673418
localhost.14030 > localhost.8080: . ack 11460 win 8191
<nop,nop,timestamp
2152179841 669666640> (DF)
E..4.K@.@.cv........6.........{............
.G..'.MP
09:27:05.675649
localhost.14030 > localhost.8080: F 199:199(0) ack
11460 win 8192
<nop,nop,timestamp 2152179841 669666640> (DF)
b.@.@...........6.........{...
........
.G..'.MP0
09:27:05.675658
localhost.8080 > localhost.14030:
. ack 200 win 2048 <nop,nop,timestamp
669666640 2152179841> (DF)
E..4..@.@.w...........6...{................
'.MP.G..
09:27:05.675688
localhost.8080 > localhost.14030: F 11460:11460(0)
ack 200 win 2048
<nop,nop,timestamp 669666640 2152179841> (DF)
..@.@.Q...........6...{................
'.MP.G..0
09:27:05.675697
localhost.14030 > localhost.8080: . ack 11461 win 8192 <nop,nop,timestamp
2152179841 669666640> (DF)
E..4x @.@...........6.........{... ........
.G..'.MP
Here the client side of the relayd does not begins to close the
connections, but actually Pushes data to the local web server and the dialog
carries on normally
Please advice me what should I do.
Where
is the
problem?
The digital certificate is issued by GeoTrust, if this
matters.
Why
relaying the HTTP headers has this effect?
Thank you,
Bogdan
P.S. Sorry
for this long post
relay
www_ssl, session 1 (1 active), 0, 10.10.11.66 ->
127.0.0.1:8080, invalid
--
