[Patch] mod_tcp / mod_proxy_tcp / mod_ssl_tcp
Hi all, The following patch provides support for TCP proxying to httpd. It consists of the following three parts: - mod_tcp: Allows the frontend to receive pure TCP connections - mod_proxy_tcp: Allows the proxy to make pure tcp or tls connections to a backend - mod_ssl_tcp: Allows the proxy to route incoming connections based on the SNI header (tlsext) In the following example config, incoming TCP connections are routed based on their SNI (the tlsext protocol) to given backend servers, which then complete the SSL connections as raw tunnels. This allows you to use client certificates through the httpd proxy balancer all the way to the backend server without the proxy terminating any SSL along the way. Protocol tlsext ServerName jira.example.com ProxyPass / tcp://john.example.com:443 Protocol tlsext ServerName www.example.com ProxyPass / tcp://erica.example.com:443 In order for mod_ssl_tcp to work, it needs to read ahead to see if any client hello message is present, and then set aside any extra data so it could be read again. This is fundamentally incompatible with c->data_in_input_filters which only allows the core filter to set aside unread data. For this reason the ability to set aside data was rolled out to all filters. mod_ssl_tcp just cares about SNI for now, but could conceivably support APLN too, making a configuration something like this: Protocol tlsext ServerName secure.example.com ProxyPass / tcp://imap.example.com:993 ProxyPass / tcp://pop3.example.com:995 Regards, Graham -- httpd-tcp-proxy.patch Description: Binary data
Re: [Patch] mod_tcp / mod_proxy_tcp / mod_ssl_tcp
On Sat, Mar 12, 2016 at 10:46 AM, Graham Leggett wrote: > The following patch provides support for TCP proxying to httpd. > > It consists of the following three parts: > > - mod_tcp: Allows the frontend to receive pure TCP connections > - mod_proxy_tcp: Allows the proxy to make pure tcp or tls connections to a > backend > - mod_ssl_tcp: Allows the proxy to route incoming connections based on the > SNI header (tlsext) Very cool stuff. Only looked on the surface so far, but one thing it reminded me of in the async case is that the MPM knows the "suspended" count but we never gave an API to really "unsuspend" when using a timed or socket callback. Longer term we really need to find a way to share that raw forwarding code as it's now probably in three modules.
Re: [Patch] mod_tcp / mod_proxy_tcp / mod_ssl_tcp
On 12 Mar 2016, at 6:26 PM, Eric Covener wrote: > Very cool stuff. Only looked on the surface so far, but one thing it > reminded me of in the async case is that the MPM knows the "suspended" > count but we never gave an API to really "unsuspend" when using a > timed or socket callback. Longer term we really need to find a way to > share that raw forwarding code as it's now probably in three modules. I am slowly working through this, teasing out each problem that prevents async behaviour. A key part of mod_tcp is the “switch ladder”, which allows tcp_process_connection() to exit at any time and be called again, at which point it will continue where it left off. We need corresponding ladders deeper into the core to make each layer down able to continue where it left off if it got EAGAIN. We also need to fix some of the function signatures - ap_process_request() is void, and so cannot signal to us elegantly that it wants to be called again, or whether it is done and we can move on. Another problem that has become apparent is that the MPMs are too limiting with respect to the events the core is willing to wait on. Right now the only event the core will handle for us is to wait for the underlying socket to be readable/writable. What I have in mind is a more general mechanism, where instead of “one socket” we have “a stack of sockets/pipes/files[1]”, and the event we wait for is the socket/pipe/file on the top of the stack. This gives us AND behaviour. If browser socket is writable AND backend socket is readable then [do stuff]. To state it another way, we wait for the browser socket to be writable, this triggers the normal pattern of events until we need to block on something else, for example we might block waiting for the backend proxy socket to be readable. That socket is added to the stack and the MPM now waits for that event. When that event triggers the top of the stack is popped off and we’re back to waiting on the underlying event (the socket). I just need to find an elegant way to trigger all of this. It would be really nice if we had SOCKET, PIPE, FILE (etc) buckets that pushed themselves onto this stack as soon as they returned EAGAIN via some kind of callback. I need to think this through to work out how to incorporate OR into this. Perhaps a list of stacks. This would allow a set of read events to run concurrently with write events. Then the sense of the connection has to be worked into it too so mod_ssl works non blocking. [1] files are typically not pollable, but we shouldn't restrict our API. It’s pipes we want to block on and in our world that’s apr_file_t. Regards, Graham —
Re: [Patch] mod_tcp / mod_proxy_tcp / mod_ssl_tcp
I've given it a quick look-thru and I. Am. Impressed. This is more Super Cool Mojo! > On Mar 12, 2016, at 10:46 AM, Graham Leggett wrote: > > Hi all, > > The following patch provides support for TCP proxying to httpd. > > It consists of the following three parts: > > - mod_tcp: Allows the frontend to receive pure TCP connections > - mod_proxy_tcp: Allows the proxy to make pure tcp or tls connections to a > backend > - mod_ssl_tcp: Allows the proxy to route incoming connections based on the > SNI header (tlsext) > > In the following example config, incoming TCP connections are routed based on > their SNI (the tlsext protocol) to given backend servers, which then complete > the SSL connections as raw tunnels. > > This allows you to use client certificates through the httpd proxy balancer > all the way to the backend server without the proxy terminating any SSL along > the way. > > > Protocol tlsext > > ServerName jira.example.com > > ProxyPass / tcp://john.example.com:443 > > > > Protocol tlsext > > ServerName www.example.com > > ProxyPass / tcp://erica.example.com:443 > > > In order for mod_ssl_tcp to work, it needs to read ahead to see if any client > hello message is present, and then set aside any extra data so it could be > read again. This is fundamentally incompatible with c->data_in_input_filters > which only allows the core filter to set aside unread data. For this reason > the ability to set aside data was rolled out to all filters. > > mod_ssl_tcp just cares about SNI for now, but could conceivably support APLN > too, making a configuration something like this: > > > Protocol tlsext > ServerName secure.example.com > >ProxyPass / tcp://imap.example.com:993 > > >ProxyPass / tcp://pop3.example.com:995 > > > > Regards, > Graham > -- > >
Re: [Patch] mod_tcp / mod_proxy_tcp / mod_ssl_tcp
+1 Really nice work -- Daniel Ruggeri On 3/13/2016 10:45 AM, Jim Jagielski wrote: > I've given it a quick look-thru and I. Am. Impressed. > > This is more Super Cool Mojo!
Re: [Patch] mod_tcp / mod_proxy_tcp / mod_ssl_tcp
On Sat, Mar 12, 2016 at 4:46 PM, Graham Leggett wrote: > > The following patch provides support for TCP proxying to httpd. > > It consists of the following three parts: > > - mod_tcp: Allows the frontend to receive pure TCP connections It looks like this module is only needed to remove HTTP filters from the chain. Is the goal to have this core module instead of mod_http and make the latter dynamic? > - mod_proxy_tcp: Allows the proxy to make pure tcp or tls connections to a > backend Thanks, this will be very useful. > - mod_ssl_tcp: Allows the proxy to route incoming connections based on the > SNI header (tlsext) Hmm, isn't mod_ssl (underlying-)protocol agnostic? Why couldn't it be used as-is (or adapted), and avoid code duplication? > > In the following example config, incoming TCP connections are routed based on > their SNI (the tlsext protocol) to given backend servers, which then complete > the SSL connections as raw tunnels. > > This allows you to use client certificates through the httpd proxy balancer > all the way to the backend server without the proxy terminating any SSL along > the way. > > > Protocol tlsext Maybe "tcps"? I agree that SNI extension is needed, but "tlsext" looks confusing. I'll look at the patch in more details, I may have missed things... Regards, Yann.
Re: [Patch] mod_tcp / mod_proxy_tcp / mod_ssl_tcp
On 14 Mar 2016, at 11:01 AM, Yann Ylavic wrote: >> The following patch provides support for TCP proxying to httpd. >> >> It consists of the following three parts: >> >> - mod_tcp: Allows the frontend to receive pure TCP connections > > It looks like this module is only needed to remove HTTP filters from the > chain. > Is the goal to have this core module instead of mod_http and make the > latter dynamic? Hmmm - good point. What we need next is a proper protocol handling mechanism to efficiently determine the protocol in use on the connection, the same way we can efficiently determine the HTTP method. Once we have that the core can be free of HTTP modules and we can just use the mod_tcp process_connection() handler. >> - mod_proxy_tcp: Allows the proxy to make pure tcp or tls connections to a >> backend > > Thanks, this will be very useful. > >> - mod_ssl_tcp: Allows the proxy to route incoming connections based on the >> SNI header (tlsext) > > Hmm, isn't mod_ssl (underlying-)protocol agnostic? > Why couldn't it be used as-is (or adapted), and avoid code duplication? It was like that to start with, but I split it all out so it could stand alone. I see the value of mod_ssl just having this as an extra input filter, will simplify this. >> In the following example config, incoming TCP connections are routed based >> on their SNI (the tlsext protocol) to given backend servers, which then >> complete the SSL connections as raw tunnels. >> >> This allows you to use client certificates through the httpd proxy balancer >> all the way to the backend server without the proxy terminating any SSL >> along the way. >> >> >> Protocol tlsext > > Maybe "tcps"? I agree that SNI extension is needed, but "tlsext" looks > confusing. The “tlsext” refers to the TLS extentions which are parsed to determine what the client is trying to talk to. These extensions are SNI and APLN (not yet supported but would be great if we could). “tcps” implies “tcp over ssl”, which we already can do - just turn on SSLEnable. Regards, Graham —
Re: [Patch] mod_tcp / mod_proxy_tcp / mod_ssl_tcp
On 12 Mar 2016, at 5:46 PM, Graham Leggett wrote: > The following patch provides support for TCP proxying to httpd. Here is an updated patch with changes to the ap_mpm_register_poll* functions. Regards, Graham -- httpd-tcp-proxy2.patch Description: Binary data