Hi Thierry,
Op 18-10-2015 om 21:37 schreef thierry.fourn...@arpalert.org:
On Sun, 18 Oct 2015 00:07:13 +0200
PiBa-NL <piba.nl....@gmail.com> wrote:
Hi haproxy list,
For testing purposes i am trying to 'modify' a response of a webserver
but only having limited success. Is this supposed to work?
As a more usefull goal than the current LAL to TST replacement i imagine
rewriting absolute links on a webpage could be possible which is
sometimes problematic with 'dumb' webapplications..
Or is it outside of the current scope of implemented functionality? If
so, it on the 'lua todo list' ?
I tried for example a configuration like below. And get several
different results in the browser.
-Sometimes i get 4 times TSTA
-Sometimes i see after the 8th TSTA- Connection: keep-alive << this
happens most of the time..
-Sometimes i get 9 times TSTA + STOP << this would be the desired
outcome (only seen very few times..)
Probably due to the response-buffer being filled differently due to
'timing'..
The "connection: keep-alive" text is probably from the actual server
reply which is 'appended' behind the response generated by my lua
script.?. However shouldn't the .done() prevent that from being send to
the client?
Ive tried putting a loop into the lua script to call res:get() multiple
times but that didnt seem to work..
Also to properly modify a page i would need to know all changes before
sending the headers with changed content-length back to the client..
Can someone confirm this is or isn't (reliably) possible? Or how this
can be scripted in lua differently?
Hello,
Your script replace 3 bytes by 3 bytes, this must run with HTTP, but if
your replacement change the length of the response, you can have some
difficulties with clients, or with keepalive.
Yes i started with replacing with the same number of bytes to avoid some
of the possible troubles caused by changing the length.. And as seen in
the haproxy.cfg it is configured with 'mode http'.
The res:get(), returns the current content of the response buffer.
Maybe it not contains the full response. You must execute a loop with
regular "core.yield()" to get back the hand to HAProxy and wait for new
Calling yield does allow to 'wait' for more data to come in.. No
guarantee that it only takes 1 yield for data to 'grow'..
[info] 278/055943 (77431) : luahttpresponse Content-Length XYZ: 14115
[info] 278/055943 (77431) : luahttpresponse SIZE: 2477
[info] 278/055943 (77431) : luahttpresponse LOOP
[info] 278/055943 (77431) : luahttpresponse SIZE: 6221
[info] 278/055943 (77431) : luahttpresponse LOOP
[info] 278/055943 (77431) : luahttpresponse SIZE: 7469
[info] 278/055943 (77431) : luahttpresponse LOOP
[info] 278/055943 (77431) : luahttpresponse SIZE: 7469
[info] 278/055943 (77431) : luahttpresponse LOOP
[info] 278/055943 (77431) : luahttpresponse SIZE: 7469
[info] 278/055943 (77431) : luahttpresponse LOOP
[info] 278/055943 (77431) : luahttpresponse SIZE: 7469
[info] 278/055943 (77431) : luahttpresponse LOOP
[info] 278/055943 (77431) : luahttpresponse SIZE: 7469
[info] 278/055943 (77431) : luahttpresponse LOOP
[info] 278/055943 (77431) : luahttpresponse SIZE: 7469
[info] 278/055943 (77431) : luahttpresponse LOOP
[info] 278/055943 (77431) : luahttpresponse SIZE: 7469
[info] 278/055943 (77431) : luahttpresponse LOOP
[info] 278/055943 (77431) : luahttpresponse SIZE: 8717
[info] 278/055943 (77431) : luahttpresponse LOOP
[info] 278/055943 (77431) : luahttpresponse SIZE: 14337
[info] 278/055943 (77431) : luahttpresponse DONE?: 14337
data. When all the data are read, res:get() returns an error.
Not sure when/how this error would happen.? The result of res:get only
seems to get bigger while the webserver is sending the response..
The res:send() is dangerous because it send data directly to the client
without the end of haproxy analysis. Maybe it is the cause o your
problem.
Try to use res:set().
Ok tried that, new try with function below.
The difficulty is that another "res:get()" returns the same data that
these you put.
I don't known if you can modify an http response greater than one
buffer.
Would be nice if that was somehow possible. But my current lua script
cannot..
The function res:close() closes the connection even if HAProxy want to
keep the connection alive. I suggest that you don't use this function.
It seems txn.res:close() does not exist? txn:done()
I reproduce the error message using curl. By default curl tries
to transfer data with keepalive, and it is not happy if the all the
announced data are not transfered.
Connection: keep-alive curl: (18) transfer closed with outstanding
read data remaining
It seems that i reproduce a bug. I'm looking for.
Ok if you can create a patch, let me know. Happy to test if it solves
some of the issues i see.
Thierry
This function seems to work for responses up to +-15KB.
Sometimes the number of loops it runs is different, and it seems kinda
in-efficient to just run loops until the response is 'complete', another
strange observation is that the res:set inside the loop is required,
even though it doesn't set a modified response, eventually the complete
response is modified in the browser result. Second request over a
keep-alive connection also fails. Adding http-server-close also closes
the client connection, but does avoid the problem with the second
request.. In the lua script I dont account for headers size yet when
checking if the response is completely read, but i dont think thats
affected the test..
Is there anything i can do to improve this function? (Besides removing
the txn:Info() lines.)
function luahttpresponse(txn)
local resheaders = txn.http:res_get_headers()
local contentlength = tonumber(resheaders["content-length"][0])
local response2 = txn.res:get()
txn:Info("luahttpresponse Content-Length XYZ: " .. contentlength)
txn:Info("luahttpresponse SIZE: " .. string.len(response2))
while string.len(response2) < contentlength do
txn:Info("luahttpresponse LOOP")
txn.res:set(response2)
core.yield()
response2 = txn.res:get()
txn:Info("luahttpresponse SIZE: " .. string.len(response2))
end
response2 = string.gsub(response2,"LAL","TST")
txn.res:set(response2)
txn:Info("luahttpresponse DONE?: " .. string.len(response2))
end
Regards,
PiBa-NL
Thanks in advance,
PiBa-NL
## haproxy.cfg
listen proxyresponse
bind :10006
mode http
http-response lua.luahttpresponse
server x 192.168.0.40:302
## script.lua
function luahttpresponse(txn)
local response2 = txn.res:get()
response2 = string.gsub(response2,"LAL","TST")
txn.res:send(response2)
txn.done()
end
core.register_action("luahttpresponse" , { "http-res" }, luahttpresponse);
## webpage.aspx , with 2.7KB output.
------------------
<%@ Page Language="C#" %>
<html><body>START
<% var x = new String('x', 250);
x = "<input type=\"hidden\" value=\""+x+"\" />"; %>
1 -LALA- <% Response.Write(x); %>
2 -LALA- <% Response.Write(x); %>
3 -LALA- <% Response.Write(x); %>
4 -LALA- <% Response.Write(x); %>
5 -LALA- <% Response.Write(x); %>
6 -LALA- <% Response.Write(x); %>
7 -LALA- <% Response.Write(x); %>
8 -LALA- <% Response.Write(x); %>
9 -LALA- <% Response.Write(x); %>
STOP
--------------------
