Re: balance url_param with POST
I'm not seeing how to use reqrep to alter a POST uri by appending a 'a=1' parameter to the end since there is no support for substitution groups. Any pointers? We can't modify the contents of a POST request but we can indeed alter the URI. And yes it does support substitution groups. For instance, you could duplicate the only url param you currently have, eg something approximately like this : reqrep ^(.*)\?([^]*)$ \1?\2\2 Your'e right of course. I don't know why I was thinking that I couldn't use substitution groups. Thanks for the pointer and this will work for me as a work-around until a proper fix can be made. I only need to alter the request line and not POST content. -Bryan
Re: balance url_param with POST
Hi Bryan, just to keep you updated, I'm pushing the fix into 1.4.12. While fixing it I discovered that check_post got broken when I implemented client-side keep-alive, because the content-length we're relying on is reset by the forwarding code before the LB code is called. So I had to fix that too. Regards, Willy
Re: balance url_param with POST
I'd like to find a work-around to this issue until it can be fixed. Is it possible to add a dummy query string parameter in a rewrite rule which would then cause the balance url_param to work with a POST? I'm not seeing how to use reqrep to alter a POST uri by appending a 'a=1' parameter to the end since there is no support for substitution groups. Any pointers? -Bryan On Fri, Feb 25, 2011 at 11:57 PM, Willy Tarreau w...@1wt.eu wrote: Hi Bryan, On Fri, Feb 25, 2011 at 11:40:00PM -0800, Bryan Talbot wrote: Maybe this is the problem? Line 548 of backend.c from 1.4.11: if (s-txn.meth == HTTP_METH_POST memchr(s-txn.req.sol + s-txn.req.sl.rq.u, '', s-txn.req.sl.rq.u_l ) == NULL) s-srv = get_server_ph_post(s); else s-srv = get_server_ph(s-be, s-txn.req.sol + s-txn.req.sl.rq.u, s-txn.req.sl.rq.u_l); It looks to me like when the method is a POST, that the url is searched for a '' character and if it's not found then the post body might be checked. Of course, it's quite likely that there is just one query string parameter so the uri would not contain a ''. I believe this should check for the existence of a '?' instead. If this is the case, then I think there is a documentation bug as well since the first line for url_param claims it only works for GET. I agree with all your points. Looks like this needs fixing. Fortunately I have not released 1.4.12 yet ;-) And yes, the reference to GET in the doc really meant everything but POST. I think we could improve that to look for the URI first, then switch to the body if there is a content-length or transfer-encoding. That would be a lot cleaner and would not rely anymore on the method. Regards, Willy
Re: balance url_param with POST
Hi Bryan, On Mon, Feb 28, 2011 at 11:34:53AM -0800, Bryan Talbot wrote: I'd like to find a work-around to this issue until it can be fixed. Is it possible to add a dummy query string parameter in a rewrite rule which would then cause the balance url_param to work with a POST? I'm not seeing how to use reqrep to alter a POST uri by appending a 'a=1' parameter to the end since there is no support for substitution groups. Any pointers? We can't modify the contents of a POST request but we can indeed alter the URI. And yes it does support substitution groups. For instance, you could duplicate the only url param you currently have, eg something approximately like this : reqrep ^(.*)\?([^]*)$ \1?\2\2 Regards, Willy
Re: balance url_param with POST
In general, I need to load balance based on a url param for any
standard HTTP method, especially the RESTful ones, not just GET. I
need it to work for GET, HEAD, PUT, DELETE, POST at the very least. It
would be great to work with custom methods like PURGE as well as that
is commonly used with proxy-caches.
Why limit balance url_param to only work with GET? Why not allow it
to work with any method that contains a URI?
-Bryan
On Thu, Feb 24, 2011 at 11:12 AM, Bryan Talbot btal...@aeriagames.com wrote:
I'm not sure I understand how the url_param option for balance is
supposed to work. From reading the description, it sounded like it
might work for both GET and POST methods when either method includes a
query string section in the URI. However, that doesn't seem to be
working as I expected with 1.4.10.
listen foo
bind: *.80
balance url_param id
server one a.b.c.d ...
server two a.b.c.e ...
GET /foo/bar?id=1 -- works fine and always sends traffic to the same
server
POST /foo/bar?id=1 -- uses round robin
The docs say:
url_param The URL parameter specified in argument will be looked up in
the query string of each HTTP GET request.
If the modifier check_post is used, then an HTTP POST
request entity will be searched for the parameter argument,
when the question mark indicating a query string ('?') is not
present in the URL. ...
which confuses me on whether or not POST query string params are
searched or not. The first statement says it only works with GET
methods. The second section says it can work with POST entity content
only when a query string is not present.
I would like to use url_param balancing for POST query string
parameters. How can I do that?
-Bryan
Re: balance url_param with POST
Maybe this is the problem? Line 548 of backend.c from 1.4.11:
if (s-txn.meth == HTTP_METH_POST
memchr(s-txn.req.sol + s-txn.req.sl.rq.u,
'',
s-txn.req.sl.rq.u_l ) == NULL)
s-srv = get_server_ph_post(s);
else
s-srv = get_server_ph(s-be,
s-txn.req.sol +
s-txn.req.sl.rq.u,
s-txn.req.sl.rq.u_l);
It looks to me like when the method is a POST, that the url is
searched for a '' character and if it's not found then the post body
might be checked. Of course, it's quite likely that there is just one
query string parameter so the uri would not contain a ''. I believe
this should check for the existence of a '?' instead.
If this is the case, then I think there is a documentation bug as well
since the first line for url_param claims it only works for GET.
-Bryan
On Fri, Feb 25, 2011 at 12:46 PM, Bryan Talbot btal...@aeriagames.com wrote:
In general, I need to load balance based on a url param for any
standard HTTP method, especially the RESTful ones, not just GET. I
need it to work for GET, HEAD, PUT, DELETE, POST at the very least. It
would be great to work with custom methods like PURGE as well as that
is commonly used with proxy-caches.
Why limit balance url_param to only work with GET? Why not allow it
to work with any method that contains a URI?
-Bryan
On Thu, Feb 24, 2011 at 11:12 AM, Bryan Talbot btal...@aeriagames.com wrote:
I'm not sure I understand how the url_param option for balance is
supposed to work. From reading the description, it sounded like it
might work for both GET and POST methods when either method includes a
query string section in the URI. However, that doesn't seem to be
working as I expected with 1.4.10.
listen foo
bind: *.80
balance url_param id
server one a.b.c.d ...
server two a.b.c.e ...
GET /foo/bar?id=1 -- works fine and always sends traffic to the same
server
POST /foo/bar?id=1 -- uses round robin
The docs say:
url_param The URL parameter specified in argument will be looked up in
the query string of each HTTP GET request.
If the modifier check_post is used, then an HTTP POST
request entity will be searched for the parameter argument,
when the question mark indicating a query string ('?') is
not
present in the URL. ...
which confuses me on whether or not POST query string params are
searched or not. The first statement says it only works with GET
methods. The second section says it can work with POST entity content
only when a query string is not present.
I would like to use url_param balancing for POST query string
parameters. How can I do that?
-Bryan
Re: balance url_param with POST
Hi Bryan, On Fri, Feb 25, 2011 at 11:40:00PM -0800, Bryan Talbot wrote: Maybe this is the problem? Line 548 of backend.c from 1.4.11: if (s-txn.meth == HTTP_METH_POST memchr(s-txn.req.sol + s-txn.req.sl.rq.u, '', s-txn.req.sl.rq.u_l ) == NULL) s-srv = get_server_ph_post(s); else s-srv = get_server_ph(s-be, s-txn.req.sol + s-txn.req.sl.rq.u, s-txn.req.sl.rq.u_l); It looks to me like when the method is a POST, that the url is searched for a '' character and if it's not found then the post body might be checked. Of course, it's quite likely that there is just one query string parameter so the uri would not contain a ''. I believe this should check for the existence of a '?' instead. If this is the case, then I think there is a documentation bug as well since the first line for url_param claims it only works for GET. I agree with all your points. Looks like this needs fixing. Fortunately I have not released 1.4.12 yet ;-) And yes, the reference to GET in the doc really meant everything but POST. I think we could improve that to look for the URI first, then switch to the body if there is a content-length or transfer-encoding. That would be a lot cleaner and would not rely anymore on the method. Regards, Willy
