Hi Guys, I followed the instructions from Alan, and set up the certs.
> dest_ip=* ssl_cert_name=host.pem ssl_key_name=host.key then I followed the instructions from Susan and tested with --resolve curl -k -v --resolve 'httbin.org:8443:127.0.0.1' > https://httpbin.org/get?answer=42 -v It seems that the ats still lets curl create tls handshake directly with the origin server, which isn't what I'm looking for. > $ curl -k -v --resolve 'httbin.org:8443:127.0.0.1' > https://httpbin.org/get?answer=42 -v > * Added httbin.org:8443:127.0.0.1 to DNS cache > * Trying 184.72.216.47... > * TCP_NODELAY set > * Connected to httpbin.org (184.72.216.47) port 443 (#0) > * ALPN, offering h2 > * ALPN, offering http/1.1 > * Cipher selection: > ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH > * successfully set certificate verify locations: > * CAfile: /etc/ssl/cert.pem > CApath: none > * TLSv1.2 (OUT), TLS handshake, Client hello (1): > * TLSv1.2 (IN), TLS handshake, Server hello (2): > * TLSv1.2 (IN), TLS handshake, Certificate (11): I'm trying to do the following, notice the "origin": "*103.253.27.108*", except that I want to make the request to ATS on *127.0.0.1*, then have the ATS forward this request to *103.253.27.108*, and return the *https* response from *103.253.27.108* back to the client. How can we make this happen with ATS please? $ curl --proxy http://103.253.27.108:80 *https*://httpbin.org/get?answer=42 > -v > * Trying 103.253.27.108... > * TCP_NODELAY set > * Connected to 103.253.27.108 (103.253.27.108) port 80 (#0) > * Establish HTTP proxy tunnel to httpbin.org:443 > > CONNECT httpbin.org:443 HTTP/1.1 > > Host: httpbin.org:443 > > User-Agent: curl/7.54.0 > > Proxy-Connection: Keep-Alive > > > < HTTP/1.1 200 Connection established > < > * Proxy replied OK to CONNECT request > * ALPN, offering h2 > * ALPN, offering http/1.1 > * Cipher selection: > ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH > * successfully set certificate verify locations: > * CAfile: /etc/ssl/cert.pem > CApath: none > * TLSv1.2 (OUT), TLS handshake, Client hello (1): > * TLSv1.2 (IN), TLS handshake, Server hello (2): > * TLSv1.2 (IN), TLS handshake, Certificate (11): > * TLSv1.2 (IN), TLS handshake, Server key exchange (12): > * TLSv1.2 (IN), TLS handshake, Server finished (14): > * TLSv1.2 (OUT), TLS handshake, Client key exchange (16): > * TLSv1.2 (OUT), TLS change cipher, Client hello (1): > * TLSv1.2 (OUT), TLS handshake, Finished (20): > * TLSv1.2 (IN), TLS change cipher, Client hello (1): > * TLSv1.2 (IN), TLS handshake, Finished (20): > * SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256 > * ALPN, server accepted to use h2 > * Server certificate: > * subject: CN=httpbin.org > * start date: Jan 18 00:00:00 2020 GMT > * expire date: Feb 18 12:00:00 2021 GMT > * subjectAltName: host "httpbin.org" matched cert's "httpbin.org" > * issuer: C=US; O=Amazon; OU=Server CA 1B; CN=Amazon > * SSL certificate verify ok. > * Using HTTP2, server supports multi-use > * Connection state changed (HTTP/2 confirmed) > * Copying HTTP/2 data in stream buffer to connection buffer after upgrade: > len=0 > * Using Stream ID: 1 (easy handle 0x7fa826808c00) > > GET /get?answer=42 HTTP/2 > > Host: httpbin.org > > User-Agent: curl/7.54.0 > > Accept: */* > > > * Connection state changed (MAX_CONCURRENT_STREAMS updated)! > < HTTP/2 200 > < date: Sat, 05 Dec 2020 06:05:52 GMT > < content-type: application/json > < content-length: 288 > < server: gunicorn/19.9.0 > < access-control-allow-origin: * > < access-control-allow-credentials: true > < > { > "args": { > "answer": "42" > }, > "headers": { > "Accept": "*/*", > "Host": "httpbin.org", > "User-Agent": "curl/7.54.0", > "X-Amzn-Trace-Id": "Root=1-5fcb2340-1bc9172e5a20c5b55ee319c6" > }, > "origin": "*103.253.27.108*", > "url": "https://httpbin.org/get?answer=42"; > } > * Connection #0 to host 103.253.27.108 left intact On Thu, Dec 3, 2020 at 9:34 PM Lei Sun <[email protected]> wrote: > Guess what, I just realized that all three of us are Yahoo Alums LOL > > On Thu, Dec 3, 2020 at 9:31 PM Lei Sun <[email protected]> wrote: > >> This is super helpful! Thank you so much Alan and Susan! >> >> On Thu, Dec 3, 2020 at 1:36 PM Susan Hinrichs <[email protected]> >> wrote: >> >>> The --resolve option is very helpful for using curl to direct requests >>> to the proxy to terminate. >>> >>> curl -k -v --resolve 'httbin.org:4443:127.0.0.1' >>> https://httpbin.org:4443/get?answer=4a >>> >>> Adding the -k assuming you are using a self-signed cert in ATS for >>> testing. Also assuming your ATS is configured to listen for TLS on port >>> 4443 in this example. >>> >>> On Thu, Dec 3, 2020 at 1:29 PM Alan Carroll < >>> [email protected]> wrote: >>> >>>> You will need to set up the certificates for ATS in that case. Although >>>> it is possible to do this in "records.config", that is (IMHO) deprecated >>>> because it has been superceded by "ssl_multicert.config". I would start >>>> with that directly, it will be easier. >>>> >>>> https://docs.trafficserver.apache.org/en/latest/admin-guide/files/ssl_multicert.config.en.html >>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.trafficserver.apache.org_en_latest_admin-2Dguide_files_ssl-5Fmulticert.config.en.html&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=GAqM_xZpxNbVqsR-aGvQBjOG3d33Y2-i4ynL-JkEouY&m=7bubaML3NbE3IHatnbQQrdelwW56A0ptWueuP2dNGGU&s=L4iVe956pejQQnlOZXgP2jUNJ85P-HmFam8gu5eji0U&e=> >>>> If there is only a single certificate, you will want to use "dest_ip=*" >>>> as the configuration which will use that certificate for all outbound >>>> connections. >>>> You'll need to use different options to curl to test this, as using >>>> "--proxy" with an "https" destination will always bypass TLS on the proxy. >>>> >>>> >>>> On Thu, Dec 3, 2020 at 1:03 PM Lei Sun <[email protected]> wrote: >>>> >>>>> Hi Alan, >>>>> >>>>> Thanks for responding! Yes, I learned about the CONNECT method. >>>>> >>>>> I can confirm that for the "https" method, I don't want the client to >>>>> do TLS directly with the server. Instead, I'd like the trafficserver to >>>>> take that request, decrypt it, then pretend to be a client, and do TLS on >>>>> behalf of the client with the upstream. >>>>> >>>>> Thanks, >>>>> Lei >>>>> >>>>> >>>>> >>>>> On Thu, Dec 3, 2020 at 12:13 PM Alan Carroll < >>>>> [email protected]> wrote: >>>>> >>>>>> The first thing to note is "curl --proxy" behaves very differently if >>>>>> the target URL is "http" vs. "https". In the former case, curl will do a >>>>>> TCP network connection to the proxy and then sends a GET request. In the >>>>>> latter case, as you can see from the output, it does a TCP network >>>>>> connection to the proxy and then sends a CONNECT (not a GET) to the >>>>>> proxy. >>>>>> After this, curl will do TLS negotiation with the upstream, NOT with ATS. >>>>>> It is unclear from your description if this is what you want. >>>>>> >>>>>> So, first question - should ATS do TLS negotiation with the user >>>>>> agent and decrypt the request? Or should it just do a blind tunnel pass >>>>>> the >>>>>> raw bytes to the upstream so the upstream does the TLS with the user >>>>>> agent? >>>>>> >>>>>> On Wed, Dec 2, 2020 at 5:41 PM Lei Sun <[email protected]> wrote: >>>>>> >>>>>>> Hi Kit, >>>>>>> >>>>>>> I'm trying to set up the trafficserver as an intermediary forward >>>>>>> proxy. >>>>>>> >>>>>>> For example, >>>>>>> 1) http client send request to trafficserver. >>>>>>> 2) trafficserver then pass this request to the downstream proxy >>>>>>> 3) downstream proxy then route this request to the origin site >>>>>>> 4) origin site send data back to the downstream proxy >>>>>>> 5) downstream proxy send data back to trafficserver >>>>>>> 6) trafficserver send data back to the http client. >>>>>>> >>>>>>> I was able to make the entire request chain work if the origin site >>>>>>> serves content directly through HTTP. >>>>>>> >>>>>>>> curl --proxy *http*://127.0.0.1:8080 >>>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1-3A8080&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=so4reBHk8fjNcUgFl5Rl6jW1O795FlyMKH-HBzls_yE&e=> >>>>>>>> *http*://httpbin.org/get?answer=4a >>>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org_get-3Fanswer-3D4a&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=Ytt8BUmEjADeQ8BgYA33Srb-fKblsmhS0lnnYYA-7H4&e=> >>>>>>>> -v >>>>>>> >>>>>>> >>>>>>> However, I ran into issues when I was trying to request for content >>>>>>> served from HTTPS. >>>>>>> >>>>>>> $ curl --proxy *http*://127.0.0.1:8080 >>>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1-3A8080&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=so4reBHk8fjNcUgFl5Rl6jW1O795FlyMKH-HBzls_yE&e=> >>>>>>>> *https*://httpbin.org/get?answer=4a >>>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org_get-3Fanswer-3D4a&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=Ytt8BUmEjADeQ8BgYA33Srb-fKblsmhS0lnnYYA-7H4&e=> >>>>>>>> -v >>>>>>>> * Trying 127.0.0.1... >>>>>>>> * TCP_NODELAY set >>>>>>>> * Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0) >>>>>>>> * Establish HTTP proxy tunnel to httpbin.org:443 >>>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org-3A443&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=QVzyvEwxJgKDXVbGsQTcSau-LJcP5X22mrrpyKfksAY&e=> >>>>>>>> > CONNECT httpbin.org:443 >>>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org-3A443&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=QVzyvEwxJgKDXVbGsQTcSau-LJcP5X22mrrpyKfksAY&e=> >>>>>>>> HTTP/1.1 >>>>>>>> > Host: httpbin.org:443 >>>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org-3A443&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=QVzyvEwxJgKDXVbGsQTcSau-LJcP5X22mrrpyKfksAY&e=> >>>>>>>> > User-Agent: curl/7.54.0 >>>>>>>> > Proxy-Connection: Keep-Alive >>>>>>>> > >>>>>>>> < HTTP/1.1 200 OK >>>>>>>> < Date: Wed, 02 Dec 2020 20:53:31 GMT >>>>>>>> < Proxy-Connection: keep-alive >>>>>>>> < Server: ATS/10.0.0 >>>>>>>> < >>>>>>>> * Proxy replied OK to CONNECT request >>>>>>>> * ALPN, offering h2 >>>>>>>> * ALPN, offering http/1.1 >>>>>>>> * Cipher selection: >>>>>>>> ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH >>>>>>>> * successfully set certificate verify locations: >>>>>>>> * CAfile: /etc/ssl/cert.pem >>>>>>>> CApath: none >>>>>>>> * *TLSv1.2 (OUT), TLS handshake, Client hello (1):* >>>>>>>> * error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version >>>>>>>> number >>>>>>>> * stopped the pause stream! >>>>>>>> * Closing connection 0 >>>>>>>> curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong >>>>>>>> version number >>>>>>>> >>>>>>> >>>>>>> From the error message, it seems that curl was able to connect to >>>>>>> the origin server, and even attempted to send the initial TLS handshake, >>>>>>> but somehow the handshake wasn't completed. >>>>>>> >>>>>>> Here are my questions. >>>>>>> 1) What's likely the cause? >>>>>>> 2) How can I fix it. >>>>>>> >>>>>>> Thank you! >>>>>>> Lei >>>>>>> >>>>>>> -- Stay Hungry, Stay Foolish. Lei Sun Cell: 408-306-9199
