Re: AWS ELB with SSL backend adds proxy protocol inside SSL stream

[Hector Rivas Gandara]

Hello,

On 10 May 2016 at 14:23, Jonathan Matthews <cont...@jpluscplusm.com> wrote:

> On 5 May 2016 at 12:11, Hector Rivas Gandara
> <hector.rivas.gand...@digital.cabinet-office.gov.uk> wrote:
>>  * If not, is there a better way to 'chain' the config as I did above.
> Take a look at the "abns@" syntax and feature documented here:
> https://cbonte.github.io/haproxy-dconv/configuration-1.6.html#bind.
> It's excellent for HAP->HAP links, as you're using. I'm using it in
> production *inside* Cloud Foundry, for the record :-)

I did not try the `abns@` thing because I did not really understand
it, but I think it is a nice proposal.

Our case is also for Cloud Foundry.

> As an aside, I'd be interested in even a brief summary of how/if you
> resolved your problem, given that I've not seen it described on the
> list before. I wonder if you're the first to run into this specific
> problem ...

As commented, I implemented it first using two frontends chained:

https://github.com/alphagov/paas-haproxy-release/commit/394a7ccf4dfe9b495f671bd3f971e4b91653e58b

Then we discussed it internally and we decided drop the requirement of
encrypting the traffic between ELB and the HAProxy for the time being.



-- 
Regards
Hector Rivas | GDS / Multi-Cloud PaaS

Re: AWS ELB with SSL backend adds proxy protocol inside SSL stream

[Hector Rivas Gandara]

On 5 May 2016 at 23:27, Igor Cicimov <ig...@encompasscorporation.com> wrote:
>
>
> On 5 May 2016 10:39 pm, "Hector Rivas Gandara" 
> <hector.rivas.gand...@digital.cabinet-office.gov.uk> wrote:
> > > https://jve.linuxwall.info/ressources/taf/haproxy-aws/
> > Thank you for your answer, but this article describes a configuration where 
> > the ELB is setup in plain TCP mode
> (no SSL), so it does not do reencryption but passes the stream to HAProxy.
> >
> > But my case is  different ELB terminates SSL and opens a SSL connection to 
> > backend (see my original mail).
>
> Maybe you should think then why do you need tproxy at all.

I am not sure what you refer with 'tproxy' but:

 * If 'tproxy' is ELB, as said: We want to use ELB because they
scalability and HA features provided by AWS, SSL  terminatation and to
restrict access to the end user certificates to only some specific
roles.

 * If 'tproxy' is HAProxy, we want to use use HAProxy to be able to do
some HTTP request rewriting.

 * If 'tproxy' is ELB in TCP/SSL mode, rather than HTTP/HTTPS mode, we
need that because we must support websockets, and ELB does not support
websockets.

Thx


-- 
Regards
Hector Rivas | GDS / Multi-Cloud PaaS

Re: AWS ELB with SSL backend adds proxy protocol inside SSL stream

[Hector Rivas Gandara]

Hi,


> https://jve.linuxwall.info/ressources/taf/haproxy-aws/


Thank you for your answer, but this article describes a configuration where
the ELB is setup in plain TCP mode (no SSL), so it does not do reencryption
but passes the stream to HAProxy.

But my case is  different ELB terminates SSL and opens a SSL connection to
backend (see my original mail).

AWS ELB with SSL backend adds proxy protocol inside SSL stream

[Hector Rivas Gandara]

Hello,

we are trying to configure this architecture:
 
 * ELB terminating SSL, using preconfigured certificates. (this is a
   requirement because so only restricted people has access to the end 
   user certs)
 * ELB connects to HAproxy backend using SSL (also requirement)
 * ELB sends proxy headers as described in http://amzn.to/1YajEG3
  
 * HAproxy listens SSL in 443
 * HAProxy is used for doing some HTTP transformations (modify header, etc).

Once ELB is configured as SSL+Proxy protocol, we tried to configure 
HAProxy by adding accept-proxy in the bind of the HTTPS frontend:

```
frontend https-in
mode http
# Note, I truncated this line because the maillist 80 chars limitations
bind :443 accept-proxy ssl crt \
 /var/vcap/jobs/haproxy/config/cert.pem \
 no-sslv3 ciphers ...
...
```

But it fails: `Received something which does not look like a PROXY 
protocol header`. 

Troubleshooting I found that ELB sends the PROXY header INSIDE of 
the SSL stream. For instance, I run openssl:

```
$ openssl s_server -accept 443 -cert cert.pem
...

ACCEPT
bad gethostbyaddr
-BEGIN SSL SESSION PARAMETERS-
MFUCAQECAgMDBAIAnwQABDBsAWD78V/tz9KhYw4R/kpL5YPBxfF1qcmzxlclNDuz
0KWw9aGojVogjtBkH/zZOLWhBgIEVyoquqIEAgIBLKQGBAQB
-END SSL SESSION PARAMETERS-
Shared
ciphers:...
CIPHER is DHE-RSA-AES256-GCM-SHA384
Secure Renegotiation IS supported
PROXY TCP4 80.194.77.90 192.168.6.14 39220 443
GET / HTTP/1.1
User-Agent: curl/7.35.0
Host: something.com
Accept: */*
```

So I did a "chained" config in haproxy, one to do the SSL termination 
with pure TCP and the other to "extract" the proxy-protocol and do the 
HTTP transformations:

``` 
listen https-in
mode tcp
bind :443 ssl crt /var/vcap/jobs/haproxy/config/cert.pem no-sslv3
ciphers ...
server http 127.0.0.1:8081

frontend http-in-from-ssl
mode http
bind :8081 accept-proxy
option httplog
option forwardfor
reqadd X-Forwarded-Proto:\ https
default_backend http-routers

```

And that works!!!

So my questions are:

 * Is this normal and expected? I cannot find any information about that.
 * Is it possible to change the ELB behaviour to put the proxy-protocol
   header OUTSIDE of the SSL stream? I did not find any info about that.
 * If not. Is it possible to change the behaviour of HAProxy to use one
   frontend but read the proxy-protocol header from inside the SSL 
   stream?
 * If not, is there a better way to 'chain' the config as I did above.
 
Thank you!