On Tue, 5 Dec 2017, Benjamin Morel via curl-library wrote:
However, when querying an HTTPs website, cURL performs a CONNECT instead,
effectively using the proxy as a TCP tunnel, and preventing it from caching
the response:
CONNECT example.com:80 HTTP/1.1
It does this to provide a secure HTTPS connection end-to-end. It is also the
way it has always been done by HTTPS clients as documented and specified since
many years. It is just not something we came up with.
As far as I can see, there is currently no way to force cURL to perform a
GET request for HTTPs websites.
No. Proxies typically have not supported that, users have not asked for it and
developers have not implemented it. Add to that the trust issue.
While I understand the rationale behind this, I don't think it is always
good.
There are plenty of people and organizations who insist HTTPS everywhere is
everywhere from downright evil down to inconvenient and inefficient. And yes,
there's certainly a price to pay for that level of security.
Caching is much harder and requires something else to work, like for example
the "blind caching" effort from earlier this year in the HTTP wg of IETF:
https://tools.ietf.org/html/draft-thomson-http-bc-01. But we don't have
anything like that yet, and I don't think this particular blind cache idea
will go much further than that.
Content scanning, anti-virus, pornfilters etc at the network level are also
commonly mentioned as something people "must have" that HTTPS makes
complicated.
It would be nice to add a cURL option to force a GET request on the proxy,
even for https URLs:
GET https://example.com/ HTTP/1.1
This is perfectly legal.
Oh yes, that's certainly perfectly fine by all the relevant standards. But
it's no longer the same kind of HTTPS as we all are used to, since now your
application no longer has an unbroken end-to-end channel to the remote server.
Now you let your proxy see everything and do anything it wants and you must
trust it uncondionally and wholly.
*Did I miss something? Would you consider adding this feature to cURL?*
I wouldn't object to supporting it, no. If someone wants to implement it,
document it and write a test or two I'm game. But the option name and
documentation would need to be crystal clear about what exactly this means in
terms of it not being "good old HTTPS" anymore - even if the user would use
such a URL together with this option.
--
/ daniel.haxx.se
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
