Re: mod_remoteip and mod_http2 combined
Try modules/http2/h2_h2.c line 550 add "mod_remoteip.c" after "mod_ssl.c". This reminds me since remoteip is being updated, maybe it should also specify some before and after mods to avoid. On 15 February 2017 at 12:50, Sander Hoentjen wrote: > Hey guys, > > I am trying to use both mod_remoteip with ProxyProtocol and mod_http2. > It looks like mod_http2 gets handed the connection before mod_remoteip, > so things don't work as they should. ProxyProtocol should always be > handled first, since it is prepended to the stream. Any pointers to > where in the code I can look to change things around? > > -- > Sander
Re: mod_remoteip and mod_http2 combined
On 02/15/2017 12:19 PM, Jordan Gigov wrote: > On 15 February 2017 at 12:50, Sander Hoentjen wrote: >> Hey guys, >> >> I am trying to use both mod_remoteip with ProxyProtocol and mod_http2. >> It looks like mod_http2 gets handed the connection before mod_remoteip, >> so things don't work as they should. ProxyProtocol should always be >> handled first, since it is prepended to the stream. Any pointers to >> where in the code I can look to change things around? >> >> -- >> Sander > Try modules/http2/h2_h2.c line 550 add "mod_remoteip.c" after "mod_ssl.c". > > This reminds me since remoteip is being updated, maybe it should also > specify some before and after mods to avoid. It seems I was wrong, the issue is that for some reason remoteip_hook_pre_connection is called twice for a h2 connection: [Wed Feb 15 14:32:08.048968 2017] [remoteip:debug] [pid 8521] mod_remoteip.c(1015): [client 192.168.122.1:35136] AH03503: RemoteIPProxyProtocol: enabled on connection to 192.168.122.249:84 [Wed Feb 15 14:32:08.049082 2017] [remoteip:debug] [pid 8521] mod_remoteip.c(1406): [client 192.168.122.1:35136] AH03511: RemoteIPProxyProtocol: received valid PROXY header: 192.168.122.1:35136 [Wed Feb 15 14:32:08.049554 2017] [http2:debug] [pid 8521] h2_session.c(994): [client 192.168.122.1:35136] AH03200: h2_session(4) created, max_streams=100, stream_mem=32768, workers_limit=6, workers_max=1, push_diary(type=1,N=256) [Wed Feb 15 14:32:08.049579 2017] [http2:debug] [pid 8521] h2_session.c(1088): [client 192.168.122.1:35136] AH03201: h2_session(4): start, INITIAL_WINDOW_SIZE=65535, MAX_CONCURRENT_STREAMS=100 [Wed Feb 15 14:32:08.049589 2017] [http2:debug] [pid 8521] h2_session.c(2067): [client 192.168.122.1:35136] AH03079: h2_session(4): started on localhost.localdomain:84 [Wed Feb 15 14:32:08.049596 2017] [http2:debug] [pid 8521] h2_session.c(1721): [client 192.168.122.1:35136] AH03078: h2_session(4): transit [INIT] -- init --> [BUSY] [Wed Feb 15 14:32:08.049616 2017] [http2:debug] [pid 8521] h2_session.c(441): [client 192.168.122.1:35136] AH03066: h2_session(4): recv FRAME[SETTINGS[length=0, stream=0]], frames=0/0 (r/s) [Wed Feb 15 14:32:08.049628 2017] [http2:debug] [pid 8521] h2_stream.c(189): [client 192.168.122.1:35136] AH03082: h2_stream(4-1): opened [Wed Feb 15 14:32:08.049655 2017] [http2:debug] [pid 8521] h2_session.c(441): [client 192.168.122.1:35136] AH03066: h2_session(4): recv FRAME[HEADERS[length=31, hend=1, stream=1, eos=1]], frames=1/0 (r/s) [Wed Feb 15 14:32:08.049706 2017] [http2:debug] [pid 8521] h2_session.c(677): [client 192.168.122.1:35136] AH03068: h2_session(4): sent FRAME[SETTINGS[length=6, stream=0]], frames=2/0 (r/s) [Wed Feb 15 14:32:08.049768 2017] [remoteip:debug] [pid 8521] mod_remoteip.c(1015): [client 192.168.122.1:35136] AH03503: RemoteIPProxyProtocol: enabled on connection to 192.168.122.249:84 [Wed Feb 15 14:32:08.049810 2017] [remoteip:warn] [pid 8521] [client 192.168.122.1:35136] AH03496: RemoteIPProxyProtocol data is missing, but required! Aborting request. [Wed Feb 15 14:32:08.049840 2017] [http2:debug] [pid 8521] h2_request.c(271): [client 192.168.122.1:35136] AH03367: h2_request: access_status=400, request_create failed mod_remote ip has: /* mod_proxy creates outgoing connections - we don't want those */ if (!remoteip_is_server_port(c->local_addr->port)) { return DECLINED; } I am guessing something similar is needed for h2 connections? -- Sander
Re: mod_remoteip and mod_http2 combined
On Wed, Feb 15, 2017 at 9:02 AM, Sander Hoentjen wrote: > > mod_remote ip has: > /* mod_proxy creates outgoing connections - we don't want those */ > if (!remoteip_is_server_port(c->local_addr->port)) { > return DECLINED; > } > I am guessing something similar is needed for h2 connections? I suspect that the mod_remoteip logic is wrong, that it should be guarding against any subordinate connections and examining only explicitly configured ports / origin IPs. the PROXY protocol is not part of the HTTP protocol and incompatible with it, so the trust list logic isn't directly compatible (this is clearly explained in the PROXY pseudo-RFC.)
Re: mod_remoteip and mod_http2 combined
On 2017-02-15 09:07 (-0600), William A Rowe Jr wrote: > On Wed, Feb 15, 2017 at 9:02 AM, Sander Hoentjen wrote: > > > > mod_remote ip has: > > /* mod_proxy creates outgoing connections - we don't want those */ > > if (!remoteip_is_server_port(c->local_addr->port)) { > > return DECLINED; > > } > > I am guessing something similar is needed for h2 connections? > > I suspect that the mod_remoteip logic is wrong, that it should be guarding > against any subordinate connections and examining only explicitly configured > ports / origin IPs. the PROXY protocol is not part of the HTTP protocol and > incompatible with it, so the trust list logic isn't directly compatible (this > is > clearly explained in the PROXY pseudo-RFC.) > Hi, Bill. That is what the module is doing. The original authors wrote it to have a list of virtual hosts it is explicitly enabled for and explicitly disabled for. I added a third list for optional vhosts. In the pre_connection hook, it checks to see if the connection's local_addr (which should normally be the server's IP) is explicitly configured to enable PROXY handling. It then checks to see if the local port is a server port. Looking at the logs shared, 192.168.122.249:84 is the server IP:Port combo and is also the local IP:Port from mod_h2. If h2 sets the master of this connection, then we could skip the whole ordeal with this patch: Index: modules/metadata/mod_remoteip.c === --- modules/metadata/mod_remoteip.c (revision 1781701) +++ modules/metadata/mod_remoteip.c (working copy) @@ -862,6 +862,10 @@ remoteip_conn_config_t *conn_conf; int optional; +if (c->master != NULL) { +return DECLINED; +} + conf = ap_get_module_config(ap_server_conf->module_config, &remoteip_module); .. but I don't know if that potentially means we are looking at the wrong connection. Sander, would it be possible to try this out?
Re: mod_remoteip and mod_http2 combined
I am away on holiday until the 25th, will try when i get back. On Sat Feb 18 23:25:51 2017 GMT+0100, Daniel Ruggeri wrote: > On 2017-02-15 09:07 (-0600), William A Rowe Jr wrote: > > On Wed, Feb 15, 2017 at 9:02 AM, Sander Hoentjen wrote: > > > > > > mod_remote ip has: > > > /* mod_proxy creates outgoing connections - we don't want those */ > > > if (!remoteip_is_server_port(c->local_addr->port)) { > > > return DECLINED; > > > } > > > I am guessing something similar is needed for h2 connections? > > > > I suspect that the mod_remoteip logic is wrong, that it should be guarding > > against any subordinate connections and examining only explicitly configured > > ports / origin IPs. the PROXY protocol is not part of the HTTP protocol and > > incompatible with it, so the trust list logic isn't directly compatible > > (this is > > clearly explained in the PROXY pseudo-RFC.) > > > > Hi, Bill. That is what the module is doing. The original authors wrote it to > have a list of virtual hosts it is explicitly enabled for and explicitly > disabled for. I added a third list for optional vhosts. In the pre_connection > hook, it checks to see if the connection's local_addr (which should normally > be the server's IP) is explicitly configured to enable PROXY handling. It > then checks to see if the local port is a server port. > > Looking at the logs shared, 192.168.122.249:84 is the server IP:Port combo > and is also the local IP:Port from mod_h2. If h2 sets the master of this > connection, then we could skip the whole ordeal with this patch: > > Index: modules/metadata/mod_remoteip.c > === > --- modules/metadata/mod_remoteip.c (revision 1781701) > +++ modules/metadata/mod_remoteip.c (working copy) > @@ -862,6 +862,10 @@ > remoteip_conn_config_t *conn_conf; > int optional; > > +if (c->master != NULL) { > +return DECLINED; > +} > + > conf = ap_get_module_config(ap_server_conf->module_config, > &remoteip_module); > > .. but I don't know if that potentially means we are looking at the wrong > connection. > > Sander, would it be possible to try this out? > > -- Sent from my Jolla
Re: mod_remoteip and mod_http2 combined
On Sat, Feb 18, 2017 at 4:25 PM, Daniel Ruggeri wrote: > On 2017-02-15 09:07 (-0600), William A Rowe Jr wrote: >> On Wed, Feb 15, 2017 at 9:02 AM, Sander Hoentjen wrote: >> > >> > mod_remote ip has: >> > /* mod_proxy creates outgoing connections - we don't want those */ >> > if (!remoteip_is_server_port(c->local_addr->port)) { >> > return DECLINED; >> > } >> > I am guessing something similar is needed for h2 connections? >> >> I suspect that the mod_remoteip logic is wrong, that it should be guarding >> against any subordinate connections and examining only explicitly configured >> ports / origin IPs. the PROXY protocol is not part of the HTTP protocol and >> incompatible with it, so the trust list logic isn't directly compatible >> (this is >> clearly explained in the PROXY pseudo-RFC.) >> > > Hi, Bill. That is what the module is doing. The original authors wrote it to > have a list of virtual hosts it is explicitly enabled for and explicitly > disabled for. I added a third list for optional vhosts. In the pre_connection > hook, it checks to see if the connection's local_addr (which should normally > be the server's IP) is explicitly configured to enable PROXY handling. It > then checks to see if the local port is a server port. > > Looking at the logs shared, 192.168.122.249:84 is the server IP:Port combo > and is also the local IP:Port from mod_h2. If h2 sets the master of this > connection, then we could skip the whole ordeal with this patch: > > Index: modules/metadata/mod_remoteip.c > === > --- modules/metadata/mod_remoteip.c (revision 1781701) > +++ modules/metadata/mod_remoteip.c (working copy) > @@ -862,6 +862,10 @@ > remoteip_conn_config_t *conn_conf; > int optional; > > +if (c->master != NULL) { > +return DECLINED; > +} > + > conf = ap_get_module_config(ap_server_conf->module_config, > &remoteip_module); > > .. but I don't know if that potentially means we are looking at the wrong > connection. That should be close, but need to ensure c->master is initialized for http as well where there is no master/subordinate. If the 'optional' (unwise) feature were removed, the decision to inject the filter before the http or h2 filter would be trivial, it would step out of the way after the first-pass (and perhaps not need to live on the filter stack at all - if we do a fixed read against the core filter - we hopefully know the number of bytes affected early and can then do a second read to complete the v1 vs v2 read?) --- all before we are in a multiple pipeline state. If we move to a conn_rec oriented one-shot nothing happens during the request phase at all. By looking at the protocol filter stack, we should be able to glean whether we are talking to the core filter, or an 'unexpected' non-network filter, right?
Re: mod_remoteip and mod_http2 combined
On 02/20/2017 07:48 PM, William A Rowe Jr wrote: > On Sat, Feb 18, 2017 at 4:25 PM, Daniel Ruggeri wrote: >> On 2017-02-15 09:07 (-0600), William A Rowe Jr wrote: >>> On Wed, Feb 15, 2017 at 9:02 AM, Sander Hoentjen wrote: mod_remote ip has: /* mod_proxy creates outgoing connections - we don't want those */ if (!remoteip_is_server_port(c->local_addr->port)) { return DECLINED; } I am guessing something similar is needed for h2 connections? >>> I suspect that the mod_remoteip logic is wrong, that it should be guarding >>> against any subordinate connections and examining only explicitly configured >>> ports / origin IPs. the PROXY protocol is not part of the HTTP protocol and >>> incompatible with it, so the trust list logic isn't directly compatible >>> (this is >>> clearly explained in the PROXY pseudo-RFC.) >>> >> Hi, Bill. That is what the module is doing. The original authors wrote it to >> have a list of virtual hosts it is explicitly enabled for and explicitly >> disabled for. I added a third list for optional vhosts. In the >> pre_connection hook, it checks to see if the connection's local_addr (which >> should normally be the server's IP) is explicitly configured to enable PROXY >> handling. It then checks to see if the local port is a server port. >> >> Looking at the logs shared, 192.168.122.249:84 is the server IP:Port combo >> and is also the local IP:Port from mod_h2. If h2 sets the master of this >> connection, then we could skip the whole ordeal with this patch: >> >> Index: modules/metadata/mod_remoteip.c >> === >> --- modules/metadata/mod_remoteip.c (revision 1781701) >> +++ modules/metadata/mod_remoteip.c (working copy) >> @@ -862,6 +862,10 @@ >> remoteip_conn_config_t *conn_conf; >> int optional; >> >> +if (c->master != NULL) { >> +return DECLINED; >> +} >> + >> conf = ap_get_module_config(ap_server_conf->module_config, >> &remoteip_module); >> >> .. but I don't know if that potentially means we are looking at the wrong >> connection. First I'll say that with the "Optional" mode it worked, just not with On I just tried this patch and as far as I have tested this seems to work fine in On mode, as well as in Optional. I do see some other issue, but that is probably in my own code, I'll try to track that down later. > That should be close, but need to ensure c->master is initialized for > http as well > where there is no master/subordinate. I am not sure what this means, how should I test this? > > If the 'optional' (unwise) feature were removed, the decision to > inject the filter before > the http or h2 filter would be trivial, it would step out of the way > after the first-pass > (and perhaps not need to live on the filter stack at all - if we do a > fixed read against > the core filter - we hopefully know the number of bytes affected early > and can then > do a second read to complete the v1 vs v2 read?) --- all before we are > in a multiple > pipeline state. > > If we move to a conn_rec oriented one-shot nothing happens during the request > phase at all. > > By looking at the protocol filter stack, we should be able to glean > whether we are > talking to the core filter, or an 'unexpected' non-network filter, right? > I myself have no use-case at the moment for the "Optional" mode, maybe others do.
Re: mod_remoteip and mod_http2 combined
Thanks, all, for the patience as I finally got back to this. On 2/24/2017 11:05 AM, Sander Hoentjen wrote: > > On 02/20/2017 07:48 PM, William A Rowe Jr wrote: >> On Sat, Feb 18, 2017 at 4:25 PM, Daniel Ruggeri wrote: >>> On 2017-02-15 09:07 (-0600), William A Rowe Jr wrote: On Wed, Feb 15, 2017 at 9:02 AM, Sander Hoentjen wrote: > mod_remote ip has: > /* mod_proxy creates outgoing connections - we don't want those */ > if (!remoteip_is_server_port(c->local_addr->port)) { > return DECLINED; > } > I am guessing something similar is needed for h2 connections? I suspect that the mod_remoteip logic is wrong, that it should be guarding against any subordinate connections and examining only explicitly configured ports / origin IPs. the PROXY protocol is not part of the HTTP protocol and incompatible with it, so the trust list logic isn't directly compatible (this is clearly explained in the PROXY pseudo-RFC.) >>> Hi, Bill. That is what the module is doing. The original authors wrote it >>> to have a list of virtual hosts it is explicitly enabled for and explicitly >>> disabled for. I added a third list for optional vhosts. In the >>> pre_connection hook, it checks to see if the connection's local_addr (which >>> should normally be the server's IP) is explicitly configured to enable >>> PROXY handling. It then checks to see if the local port is a server port. >>> >>> Looking at the logs shared, 192.168.122.249:84 is the server IP:Port combo >>> and is also the local IP:Port from mod_h2. If h2 sets the master of this >>> connection, then we could skip the whole ordeal with this patch: >>> >>> Index: modules/metadata/mod_remoteip.c >>> === >>> --- modules/metadata/mod_remoteip.c (revision 1781701) >>> +++ modules/metadata/mod_remoteip.c (working copy) >>> @@ -862,6 +862,10 @@ >>> remoteip_conn_config_t *conn_conf; >>> int optional; >>> >>> +if (c->master != NULL) { >>> +return DECLINED; >>> +} >>> + >>> conf = ap_get_module_config(ap_server_conf->module_config, >>> &remoteip_module); >>> >>> .. but I don't know if that potentially means we are looking at the wrong >>> connection. > First I'll say that with the "Optional" mode it worked, just not with On > I just tried this patch and as far as I have tested this seems to work > fine in On mode, as well as in Optional. I do see some other issue, but > that is probably in my own code, I'll try to track that down later. This is good news and about what I was expecting to happen. I will add this to the commit I've got coming that incorporates a lot of Ruediger's feedback. >> That should be close, but need to ensure c->master is initialized for >> http as well >> where there is no master/subordinate. > I am not sure what this means, how should I test this? Hi, Bill - also hoping for a bit more input. Since PROXY protocol is not tied to any particular layer 7 protocol, I don't think we'd have to verify it is initialized for HTTP - just that there is no master at all. At least, that's my understanding so I appreciate any corrections. >> If the 'optional' (unwise) feature were removed, the decision to >> inject the filter before >> the http or h2 filter would be trivial, it would step out of the way >> after the first-pass >> (and perhaps not need to live on the filter stack at all - if we do a >> fixed read against >> the core filter - we hopefully know the number of bytes affected early >> and can then >> do a second read to complete the v1 vs v2 read?) --- all before we are >> in a multiple >> pipeline state. I don't think that is the case because the module was also written to support PROXY header consumption in a name-based virtual host context, too, where ServerName abc may require it but ServerName xyz does not. Agreed, if it were simply all or nothing (either it's enabled on this connection or not), we could avoid some of these gymnastics but I think that use case as well as the Optional use case requires it. >> If we move to a conn_rec oriented one-shot nothing happens during the request >> phase at all. >> >> By looking at the protocol filter stack, we should be able to glean >> whether we are >> talking to the core filter, or an 'unexpected' non-network filter, right? >> > I myself have no use-case at the moment for the "Optional" mode, maybe > others do. Sure, to clarify, the Optional use case came from a member on one of our cousin projects (Tomcat) Chris Schultz as well as my own use cases. It is useful for internally accessing the site from behind the loadbalancer. When there is a publicly addressed upstream loadbalancer (Amazon ELB or just HAProxy itself) talking to RFC1918 addressed or non-routeable backend httpd servers, it becomes impossible to enable internal communication on the RFC1918 space to the back
Re: mod_remoteip and mod_http2 combined
On 03/11/2017 07:57 PM, Daniel Ruggeri wrote: > Thanks, all, for the patience as I finally got back to this. > > On 2/24/2017 11:05 AM, Sander Hoentjen wrote: >> On 02/20/2017 07:48 PM, William A Rowe Jr wrote: >>> On Sat, Feb 18, 2017 at 4:25 PM, Daniel Ruggeri wrote: On 2017-02-15 09:07 (-0600), William A Rowe Jr wrote: > On Wed, Feb 15, 2017 at 9:02 AM, Sander Hoentjen > wrote: >> mod_remote ip has: >> /* mod_proxy creates outgoing connections - we don't want those */ >> if (!remoteip_is_server_port(c->local_addr->port)) { >> return DECLINED; >> } >> I am guessing something similar is needed for h2 connections? > I suspect that the mod_remoteip logic is wrong, that it should be guarding > against any subordinate connections and examining only explicitly > configured > ports / origin IPs. the PROXY protocol is not part of the HTTP protocol > and > incompatible with it, so the trust list logic isn't directly compatible > (this is > clearly explained in the PROXY pseudo-RFC.) > Hi, Bill. That is what the module is doing. The original authors wrote it to have a list of virtual hosts it is explicitly enabled for and explicitly disabled for. I added a third list for optional vhosts. In the pre_connection hook, it checks to see if the connection's local_addr (which should normally be the server's IP) is explicitly configured to enable PROXY handling. It then checks to see if the local port is a server port. Looking at the logs shared, 192.168.122.249:84 is the server IP:Port combo and is also the local IP:Port from mod_h2. If h2 sets the master of this connection, then we could skip the whole ordeal with this patch: Index: modules/metadata/mod_remoteip.c === --- modules/metadata/mod_remoteip.c (revision 1781701) +++ modules/metadata/mod_remoteip.c (working copy) @@ -862,6 +862,10 @@ remoteip_conn_config_t *conn_conf; int optional; +if (c->master != NULL) { +return DECLINED; +} + conf = ap_get_module_config(ap_server_conf->module_config, &remoteip_module); .. but I don't know if that potentially means we are looking at the wrong connection. >> First I'll say that with the "Optional" mode it worked, just not with On >> I just tried this patch and as far as I have tested this seems to work >> fine in On mode, as well as in Optional. I do see some other issue, but >> that is probably in my own code, I'll try to track that down later. > This is good news and about what I was expecting to happen. I will add > this to the commit I've got coming that incorporates a lot of Ruediger's > feedback. > >>> That should be close, but need to ensure c->master is initialized for >>> http as well >>> where there is no master/subordinate. >> I am not sure what this means, how should I test this? > Hi, Bill - also hoping for a bit more input. Since PROXY protocol is not > tied to any particular layer 7 protocol, I don't think we'd have to > verify it is initialized for HTTP - just that there is no master at all. > At least, that's my understanding so I appreciate any corrections. Here are my changes by the way: https://github.com/AntagonistHQ/httpd/commit/2d208793b4494e73289477c231c79be9e0030a2b > Sure, to clarify, the Optional use case came from a member on one of our > cousin projects (Tomcat) Chris Schultz as well as my own use cases. It > is useful for internally accessing the site from behind the > loadbalancer. When there is a publicly addressed upstream loadbalancer > (Amazon ELB or just HAProxy itself) talking to RFC1918 addressed or > non-routeable backend httpd servers, it becomes impossible to enable > internal communication on the RFC1918 space to the backend instances. > If the goal is to monitor or probe the site and (httpd proxy) backends > internally for health, this *can* be done by duplicating the virtual > hosts. Depending on the complexity of the virtual hosts, what resources > those virtual hosts have (proxies and whatnot) and their general size, > this could result in a fairly unmanageable httpd configuration having a > vhost that requires PROXY header and a second one on a different port or > IP that does not. > It gets even more complicated when you are aiming to do management > tasks. If you have balancers configured at the vhost as intended, you > can only manage those balancers from the vhost they live in. Further, > you may want to view server statistics, check info about the ldap cache, > etc but permit access to those things only from a trusted network in > addition to the user credentials protecting it. > So for those examples, aside from creating an internal HAProxy to > provide the header, it's not really possible
Re: mod_remoteip and mod_http2 combined
On 03/16/2017 10:34 AM, Sander Hoentjen wrote: > > On 03/11/2017 07:57 PM, Daniel Ruggeri wrote: >> Thanks, all, for the patience as I finally got back to this. >> >> On 2/24/2017 11:05 AM, Sander Hoentjen wrote: >>> On 02/20/2017 07:48 PM, William A Rowe Jr wrote: On Sat, Feb 18, 2017 at 4:25 PM, Daniel Ruggeri wrote: > On 2017-02-15 09:07 (-0600), William A Rowe Jr > wrote: >> On Wed, Feb 15, 2017 at 9:02 AM, Sander Hoentjen >> wrote: >>> mod_remote ip has: >>> /* mod_proxy creates outgoing connections - we don't want those */ >>> if (!remoteip_is_server_port(c->local_addr->port)) { >>> return DECLINED; >>> } >>> I am guessing something similar is needed for h2 connections? >> I suspect that the mod_remoteip logic is wrong, that it should be >> guarding >> against any subordinate connections and examining only explicitly >> configured >> ports / origin IPs. the PROXY protocol is not part of the HTTP protocol >> and >> incompatible with it, so the trust list logic isn't directly compatible >> (this is >> clearly explained in the PROXY pseudo-RFC.) >> > Hi, Bill. That is what the module is doing. The original authors wrote it > to have a list of virtual hosts it is explicitly enabled for and > explicitly disabled for. I added a third list for optional vhosts. In the > pre_connection hook, it checks to see if the connection's local_addr > (which should normally be the server's IP) is explicitly configured to > enable PROXY handling. It then checks to see if the local port is a > server port. > > Looking at the logs shared, 192.168.122.249:84 is the server IP:Port > combo and is also the local IP:Port from mod_h2. If h2 sets the master of > this connection, then we could skip the whole ordeal with this patch: > > Index: modules/metadata/mod_remoteip.c > === > --- modules/metadata/mod_remoteip.c (revision 1781701) > +++ modules/metadata/mod_remoteip.c (working copy) > @@ -862,6 +862,10 @@ > remoteip_conn_config_t *conn_conf; > int optional; > > +if (c->master != NULL) { > +return DECLINED; > +} > + > conf = ap_get_module_config(ap_server_conf->module_config, > &remoteip_module); > > .. but I don't know if that potentially means we are looking at the wrong > connection. >>> First I'll say that with the "Optional" mode it worked, just not with On >>> I just tried this patch and as far as I have tested this seems to work >>> fine in On mode, as well as in Optional. I do see some other issue, but >>> that is probably in my own code, I'll try to track that down later. >> This is good news and about what I was expecting to happen. I will add >> this to the commit I've got coming that incorporates a lot of Ruediger's >> feedback. >> That should be close, but need to ensure c->master is initialized for http as well where there is no master/subordinate. >>> I am not sure what this means, how should I test this? >> Hi, Bill - also hoping for a bit more input. Since PROXY protocol is not >> tied to any particular layer 7 protocol, I don't think we'd have to >> verify it is initialized for HTTP - just that there is no master at all. >> At least, that's my understanding so I appreciate any corrections. > Here are my changes by the way: > https://github.com/AntagonistHQ/httpd/commit/2d208793b4494e73289477c231c79be9e0030a2b >> Sure, to clarify, the Optional use case came from a member on one of our >> cousin projects (Tomcat) Chris Schultz as well as my own use cases. It >> is useful for internally accessing the site from behind the >> loadbalancer. When there is a publicly addressed upstream loadbalancer >> (Amazon ELB or just HAProxy itself) talking to RFC1918 addressed or >> non-routeable backend httpd servers, it becomes impossible to enable >> internal communication on the RFC1918 space to the backend instances. >> If the goal is to monitor or probe the site and (httpd proxy) backends >> internally for health, this *can* be done by duplicating the virtual >> hosts. Depending on the complexity of the virtual hosts, what resources >> those virtual hosts have (proxies and whatnot) and their general size, >> this could result in a fairly unmanageable httpd configuration having a >> vhost that requires PROXY header and a second one on a different port or >> IP that does not. >> It gets even more complicated when you are aiming to do management >> tasks. If you have balancers configured at the vhost as intended, you >> can only manage those balancers from the vhost they live in. Further, >> you may want to view server statistics, check info about the ldap cache, >> etc but permit access to those things only from a trusted network in >
Re: mod_remoteip and mod_http2 combined
On Mon, Mar 27, 2017 at 9:07 AM, Sander Hoentjen wrote: > > On 03/16/2017 10:34 AM, Sander Hoentjen wrote: >> >> On 03/11/2017 07:57 PM, Daniel Ruggeri wrote: >>> Thanks, all, for the patience as I finally got back to this. >>> >>> On 2/24/2017 11:05 AM, Sander Hoentjen wrote: On 02/20/2017 07:48 PM, William A Rowe Jr wrote: > On Sat, Feb 18, 2017 at 4:25 PM, Daniel Ruggeri > wrote: >> On 2017-02-15 09:07 (-0600), William A Rowe Jr >> wrote: >>> On Wed, Feb 15, 2017 at 9:02 AM, Sander Hoentjen >>> wrote: mod_remote ip has: /* mod_proxy creates outgoing connections - we don't want those */ if (!remoteip_is_server_port(c->local_addr->port)) { return DECLINED; } I am guessing something similar is needed for h2 connections? >>> I suspect that the mod_remoteip logic is wrong, that it should be >>> guarding >>> against any subordinate connections and examining only explicitly >>> configured >>> ports / origin IPs. the PROXY protocol is not part of the HTTP protocol >>> and >>> incompatible with it, so the trust list logic isn't directly compatible >>> (this is >>> clearly explained in the PROXY pseudo-RFC.) >>> >> Hi, Bill. That is what the module is doing. The original authors wrote >> it to have a list of virtual hosts it is explicitly enabled for and >> explicitly disabled for. I added a third list for optional vhosts. In >> the pre_connection hook, it checks to see if the connection's local_addr >> (which should normally be the server's IP) is explicitly configured to >> enable PROXY handling. It then checks to see if the local port is a >> server port. >> >> Looking at the logs shared, 192.168.122.249:84 is the server IP:Port >> combo and is also the local IP:Port from mod_h2. If h2 sets the master >> of this connection, then we could skip the whole ordeal with this patch: >> >> Index: modules/metadata/mod_remoteip.c >> === >> --- modules/metadata/mod_remoteip.c (revision 1781701) >> +++ modules/metadata/mod_remoteip.c (working copy) >> @@ -862,6 +862,10 @@ >> remoteip_conn_config_t *conn_conf; >> int optional; >> >> +if (c->master != NULL) { >> +return DECLINED; >> +} >> + >> conf = ap_get_module_config(ap_server_conf->module_config, >> &remoteip_module); >> >> .. but I don't know if that potentially means we are looking at the >> wrong connection. First I'll say that with the "Optional" mode it worked, just not with On I just tried this patch and as far as I have tested this seems to work fine in On mode, as well as in Optional. I do see some other issue, but that is probably in my own code, I'll try to track that down later. >>> This is good news and about what I was expecting to happen. I will add >>> this to the commit I've got coming that incorporates a lot of Ruediger's >>> feedback. >>> > That should be close, but need to ensure c->master is initialized for > http as well > where there is no master/subordinate. I am not sure what this means, how should I test this? >>> Hi, Bill - also hoping for a bit more input. Since PROXY protocol is not >>> tied to any particular layer 7 protocol, I don't think we'd have to >>> verify it is initialized for HTTP - just that there is no master at all. >>> At least, that's my understanding so I appreciate any corrections. >> Here are my changes by the way: >> https://github.com/AntagonistHQ/httpd/commit/2d208793b4494e73289477c231c79be9e0030a2b >>> Sure, to clarify, the Optional use case came from a member on one of our >>> cousin projects (Tomcat) Chris Schultz as well as my own use cases. It >>> is useful for internally accessing the site from behind the >>> loadbalancer. When there is a publicly addressed upstream loadbalancer >>> (Amazon ELB or just HAProxy itself) talking to RFC1918 addressed or >>> non-routeable backend httpd servers, it becomes impossible to enable >>> internal communication on the RFC1918 space to the backend instances. >>> If the goal is to monitor or probe the site and (httpd proxy) backends >>> internally for health, this *can* be done by duplicating the virtual >>> hosts. Depending on the complexity of the virtual hosts, what resources >>> those virtual hosts have (proxies and whatnot) and their general size, >>> this could result in a fairly unmanageable httpd configuration having a >>> vhost that requires PROXY header and a second one on a different port or >>> IP that does not. >>> It gets even more complicated when you are aiming to do management >>> tasks. If you have balancers configured at the vhost as intended, you >>> can only manage those balancers from the vhost they live in. Further
Re: mod_remoteip and mod_http2 combined
On Wed, Mar 29, 2017 at 4:43 PM, William A Rowe Jr wrote: > > It would be nice if the mod_remoteip patch to PROXY protocol followed the > security advisories of the PROXY draft security comments, and we rip out the > 'optional' mode. The remaining objection is around the ambiguity of 'optional' > (which can't exist) and the objection that how PROXY works as an implicit > trust model using mod_remoteip is laughable, since the connection cannot > be established without some PROXY protocol line interceptor yanking the > garbage out of otherwise well-formed HTTP/1.1 - HTTP-TLS - h2c - h2 input. > > There is no 'untrusted PROXY header input' because that isn't part of the > HTTP protocol and that garbage generates a 400 without an interceptor. > No problem declaring that if we are willing to decode it, we will accept the > input as gospel. (By this measure, I'm dropping any objection to also setting HTTPS TLS content trust flags, although mod_ssl would typically provide much more information about which server and client certificates had been presented and what cipher is in use.)
Re: mod_remoteip and mod_http2 combined
Agreed - as many times as I read the spec, I have no idea how I did not see that security advisory. It's flat-out damning to the idea of an "optional" mode. I'll go ahead and rip out the optional processing and will add your suggested idea of a list of subnets to disable parsing. I hope to have a patch later this morning to share. As awful as the name is, I'm thinking RemoteIPProxyProtocolDisableNetworks ARG1 ARG2 ARG3. -- Daniel Ruggeri On 3/29/2017 4:43 PM, William A Rowe Jr wrote: > This is the gist of my remaining objections. > > It would be nice if the mod_remoteip patch to PROXY protocol followed the > security advisories of the PROXY draft security comments, and we rip out the > 'optional' mode. The remaining objection is around the ambiguity of 'optional' > (which can't exist) and the objection that how PROXY works as an implicit > trust model using mod_remoteip is laughable, since the connection cannot > be established without some PROXY protocol line interceptor yanking the > garbage out of otherwise well-formed HTTP/1.1 - HTTP-TLS - h2c - h2 input. > > There is no 'untrusted PROXY header input' because that isn't part of the > HTTP protocol and that garbage generates a 400 without an interceptor. > No problem declaring that if we are willing to decode it, we will accept the > input as gospel.
Re: mod_remoteip and mod_http2 combined
Sorry for the top post. I've committed r1789800 which pulls out Optional handling and adds the ability to disable based on source network. This is more or less the code as it was donated, plus some cleanup and the small addition to disable based on networks (overall a cleaner approach anyway). I went with the directive name RemoteIPProxyProtocolDisableHosts to align more with the fact that a single host or range can be disabled. I've verified this works via haproxy, rejects when hit directly and disables processing when coming from a permitted network. I'm in the process of updating the backport proposal. To be safe, I'm removing @jim's vote given how many times the code has changed since he reviewed and will put it back in the "active" section of STATUS. -- Daniel Ruggeri On 4/1/2017 8:17 AM, Daniel Ruggeri wrote: > Agreed - as many times as I read the spec, I have no idea how I did not > see that security advisory. It's flat-out damning to the idea of an > "optional" mode. I'll go ahead and rip out the optional processing and > will add your suggested idea of a list of subnets to disable parsing. I > hope to have a patch later this morning to share. As awful as the name > is, I'm thinking RemoteIPProxyProtocolDisableNetworks ARG1 ARG2 ARG3. >
Re: mod_remoteip and mod_http2 combined
Hi Daniel, On Sat, Apr 1, 2017 at 3:56 PM, Daniel Ruggeri wrote: > I went with the directive name > RemoteIPProxyProtocolDisableHosts to align more with the fact that a > single host or range can be disabled. How about RemoteIPProxyProtocolExceptions since one can configure either an IP or a network? Host looks a bit ambiguous to me, especially with HTTP... Regards, Yann.
Re: mod_remoteip and mod_http2 combined
On 4/1/2017 11:18 AM, Yann Ylavic wrote: > Hi Daniel, > > On Sat, Apr 1, 2017 at 3:56 PM, Daniel Ruggeri wrote: >> I went with the directive name >> RemoteIPProxyProtocolDisableHosts to align more with the fact that a >> single host or range can be disabled. > How about RemoteIPProxyProtocolExceptions since one can configure > either an IP or a network? > Host looks a bit ambiguous to me, especially with HTTP... > > > Regards, > Yann. Oh, yeah, I like that even better. I'll stage this up and will commit in a day or two unless we have some better suggestions. -- Daniel Ruggeri
Re: mod_remoteip and mod_http2 combined
I can see that a flat directive namespace has its drawbacks... ;-) > Am 01.04.2017 um 19:12 schrieb Daniel Ruggeri : > > > On 4/1/2017 11:18 AM, Yann Ylavic wrote: >> Hi Daniel, >> >> On Sat, Apr 1, 2017 at 3:56 PM, Daniel Ruggeri wrote: >>> I went with the directive name >>> RemoteIPProxyProtocolDisableHosts to align more with the fact that a >>> single host or range can be disabled. >> How about RemoteIPProxyProtocolExceptions since one can configure >> either an IP or a network? >> Host looks a bit ambiguous to me, especially with HTTP... >> >> >> Regards, >> Yann. > > Oh, yeah, I like that even better. I'll stage this up and will commit in > a day or two unless we have some better suggestions. > > -- > Daniel Ruggeri >