Re: balance url_param with POST
Hi Bryan, On Fri, Feb 25, 2011 at 11:40:00PM -0800, Bryan Talbot wrote: > Maybe this is the problem? Line 548 of backend.c from 1.4.11: > > > if (s->txn.meth == HTTP_METH_POST && > memchr(s->txn.req.sol + s->txn.req.sl.rq.u, > '&', > s->txn.req.sl.rq.u_l ) == NULL) > s->srv = get_server_ph_post(s); > else > s->srv = get_server_ph(s->be, > s->txn.req.sol + > s->txn.req.sl.rq.u, > > s->txn.req.sl.rq.u_l); > > > It looks to me like when the method is a POST, that the url is > searched for a '&' character and if it's not found then the post body > might be checked. Of course, it's quite likely that there is just one > query string parameter so the uri would not contain a '&'. I believe > this should check for the existence of a '?' instead. > > If this is the case, then I think there is a documentation bug as well > since the first line for url_param claims it only works for GET. I agree with all your points. Looks like this needs fixing. Fortunately I have not released 1.4.12 yet ;-) And yes, the reference to GET in the doc really meant "everything but POST". I think we could improve that to look for the URI first, then switch to the body if there is a content-length or transfer-encoding. That would be a lot cleaner and would not rely anymore on the method. Regards, Willy
Re: balance url_param with POST
Maybe this is the problem? Line 548 of backend.c from 1.4.11:
if (s->txn.meth == HTTP_METH_POST &&
memchr(s->txn.req.sol + s->txn.req.sl.rq.u,
'&',
s->txn.req.sl.rq.u_l ) == NULL)
s->srv = get_server_ph_post(s);
else
s->srv = get_server_ph(s->be,
s->txn.req.sol +
s->txn.req.sl.rq.u,
s->txn.req.sl.rq.u_l);
It looks to me like when the method is a POST, that the url is
searched for a '&' character and if it's not found then the post body
might be checked. Of course, it's quite likely that there is just one
query string parameter so the uri would not contain a '&'. I believe
this should check for the existence of a '?' instead.
If this is the case, then I think there is a documentation bug as well
since the first line for url_param claims it only works for GET.
-Bryan
On Fri, Feb 25, 2011 at 12:46 PM, Bryan Talbot wrote:
> In general, I need to load balance based on a url param for any
> standard HTTP method, especially the RESTful ones, not just GET. I
> need it to work for GET, HEAD, PUT, DELETE, POST at the very least. It
> would be great to work with custom methods like PURGE as well as that
> is commonly used with proxy-caches.
>
> Why limit balance url_param to only work with GET? Why not allow it
> to work with any method that contains a URI?
>
> -Bryan
>
>
>
> On Thu, Feb 24, 2011 at 11:12 AM, Bryan Talbot wrote:
>> I'm not sure I understand how the url_param option for balance is
>> supposed to work. From reading the description, it sounded like it
>> might work for both GET and POST methods when either method includes a
>> query string section in the URI. However, that doesn't seem to be
>> working as I expected with 1.4.10.
>>
>> listen foo
>> bind: *.80
>> balance url_param id
>> server one a.b.c.d ...
>> server two a.b.c.e ...
>>
>>
>> GET /foo/bar?id=1 --> works fine and always sends traffic to the same
>> server
>> POST /foo/bar?id=1 --> uses round robin
>>
>>
>> The docs say:
>>
>> url_param The URL parameter specified in argument will be looked up in
>> the query string of each HTTP GET request.
>>
>> If the modifier "check_post" is used, then an HTTP POST
>> request entity will be searched for the parameter argument,
>> when the question mark indicating a query string ('?') is
>> not
>> present in the URL. ...
>>
>>
>> which confuses me on whether or not POST query string params are
>> searched or not. The first statement says it only works with GET
>> methods. The second section says it can work with POST entity content
>> only when a query string is not present.
>>
>> I would like to use url_param balancing for POST query string
>> parameters. How can I do that?
>>
>> -Bryan
>>
>
About multi url_param
hi: How can I dispatch a request like http://www.mysite.com/getthing?key=ad565cbb6e9d63f640a5f7202d9f7661&q=whatisup&a1=how&a2=when&a3=who according to "q=whatisup&a1=how&a2=when", that is, the values of "q", "a1", "a2" simultaneously. And the length of "q=whatisup&a1=how&a2=when" is not fixed. thanks a lot.
Re: proto_ftp.c
2011/2/25 Krzysztof Olędzki : > Proxing FTP is much more complicated than simply providing one additional > command for passing client's IP address. > > Please note that FTP is based on two independent TCP connections: control > and data. You need to analyze a control stream and modify on-fly data (port > numbers and ip addresses) and set up additional sockets and initiate > additional connections to handle data stream. To do this you also need to > handle both PASV/EPSV (passive) and PORT/EPRT (active) modes. > > It is of course doable but the amount of work is quite big. I even was > recently asked to implement such function as a sponsored feature. After a > short conversation with my possible employer we decided that it would took > too much time to be profitable and cost effective. Instead another solution > was chosen - LVS DR. I have all of that figured out. I simply would like to have the client's IP address. I only use HAProxy for the command channel. Data channel is handled simply by choosing a different PASV port range for each backend server, and NATing the right range to the right server. Outbound Active connections are similarly S-NAT'd to the appropriate outbound address. I just want the last piece of the puzzle. As always, in parallel I am building a mainline kernel 2.6.37.2, while I am investigating other options.
Re: balance url_param with POST
In general, I need to load balance based on a url param for any
standard HTTP method, especially the RESTful ones, not just GET. I
need it to work for GET, HEAD, PUT, DELETE, POST at the very least. It
would be great to work with custom methods like PURGE as well as that
is commonly used with proxy-caches.
Why limit balance url_param to only work with GET? Why not allow it
to work with any method that contains a URI?
-Bryan
On Thu, Feb 24, 2011 at 11:12 AM, Bryan Talbot wrote:
> I'm not sure I understand how the url_param option for balance is
> supposed to work. From reading the description, it sounded like it
> might work for both GET and POST methods when either method includes a
> query string section in the URI. However, that doesn't seem to be
> working as I expected with 1.4.10.
>
> listen foo
> bind: *.80
> balance url_param id
> server one a.b.c.d ...
> server two a.b.c.e ...
>
>
> GET /foo/bar?id=1 --> works fine and always sends traffic to the same
> server
> POST /foo/bar?id=1 --> uses round robin
>
>
> The docs say:
>
> url_param The URL parameter specified in argument will be looked up in
> the query string of each HTTP GET request.
>
> If the modifier "check_post" is used, then an HTTP POST
> request entity will be searched for the parameter argument,
> when the question mark indicating a query string ('?') is not
> present in the URL. ...
>
>
> which confuses me on whether or not POST query string params are
> searched or not. The first statement says it only works with GET
> methods. The second section says it can work with POST entity content
> only when a query string is not present.
>
> I would like to use url_param balancing for POST query string
> parameters. How can I do that?
>
> -Bryan
>
Re: proto_ftp.c
On 2011-02-25 18:29, Ben Timby wrote: First of all, sorry for the previous list spam. I pasted the wrong address while subscribing. I am setting up FTP load balancing using HAProxy. The rub is that I want something similar to the X-Forwarded-For header supported in HTTP. I am aware of TPROXY, but I don't wish to maintain my own packages for the kernel, xen and all the dependencies this entails. TPROXY is now included in the mainline. Howevwe, I'm not sure if it is the solution you are interested in. A simpler user-space solution would suit me much better. I would like to patch HAProxy so that it provides specialized FTP handling in the form of an FTP SITE command. Such that when optionally enabled, it will inject the following FTP command at the beginning of the TCP stream. SITE IP=XXX.XXX.XXX.XXX My backend FTP server will know how to deal with this site command and store the IP address for use internally. This would negate the need for TPROXY and seems fairly straightforward. Any feedback or thoughts on this topic? Proxing FTP is much more complicated than simply providing one additional command for passing client's IP address. Please note that FTP is based on two independent TCP connections: control and data. You need to analyze a control stream and modify on-fly data (port numbers and ip addresses) and set up additional sockets and initiate additional connections to handle data stream. To do this you also need to handle both PASV/EPSV (passive) and PORT/EPRT (active) modes. It is of course doable but the amount of work is quite big. I even was recently asked to implement such function as a sponsored feature. After a short conversation with my possible employer we decided that it would took too much time to be profitable and cost effective. Instead another solution was chosen - LVS DR. Best regards, Krzysztof Olędzki
proto_ftp.c
First of all, sorry for the previous list spam. I pasted the wrong address while subscribing. I am setting up FTP load balancing using HAProxy. The rub is that I want something similar to the X-Forwarded-For header supported in HTTP. I am aware of TPROXY, but I don't wish to maintain my own packages for the kernel, xen and all the dependencies this entails. A simpler user-space solution would suit me much better. I would like to patch HAProxy so that it provides specialized FTP handling in the form of an FTP SITE command. Such that when optionally enabled, it will inject the following FTP command at the beginning of the TCP stream. SITE IP=XXX.XXX.XXX.XXX My backend FTP server will know how to deal with this site command and store the IP address for use internally. This would negate the need for TPROXY and seems fairly straightforward. Any feedback or thoughts on this topic? Thanks.
subscribe
subscribe
