Re: [Twisted-Python] twisted.web HTTPS client certificate

[Glyph]

> On Jul 14, 2021, at 7:05 AM, Barry Scott  wrote:
> 
> On Tuesday, 13 July 2021 22:48:18 BST Glyph wrote:
>> 
>>> On Jul 13, 2021, at 2:09 AM, Barry Scott  wrote:
>>> 
>>> On Monday, 12 July 2021 09:27:19 BST Glyph wrote:
 FWIW I would avoid calling the pyOpenSSL APIs for this, since hopefully 
 we'll manage to move away from OpenSSL and at least somewhat abstract away 
 the transition. 
>>> 
>>> Are you thinking to use the Windows and macOS provided crypto API and only 
>>> use openssl on Unix systems?
>>> 
>>> Barry
>> 
>> It would be nice to have a system for backends so that we could do this when 
>> certain specific situations call for it (usually related to TLS clients, 
>> rather than servers, although having both would be great), but no, the main 
>> motivation is to drop OpenSSL entirely in favor of Rustls, as recommended by 
>> the ISRG: 
>> > >.
> 
> That is a great goal for Twisted.

I'm glad you think so! I think it's a great goal for everybody, really ;-).

-g
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] twisted.web HTTPS client certificate

[Barry Scott]

On Tuesday, 13 July 2021 22:48:18 BST Glyph wrote:
> 
> > On Jul 13, 2021, at 2:09 AM, Barry Scott  wrote:
> > 
> > On Monday, 12 July 2021 09:27:19 BST Glyph wrote:
> >> FWIW I would avoid calling the pyOpenSSL APIs for this, since hopefully 
> >> we'll manage to move away from OpenSSL and at least somewhat abstract away 
> >> the transition. 
> > 
> > Are you thinking to use the Windows and macOS provided crypto API and only 
> > use openssl on Unix systems?
> > 
> > Barry
> 
> It would be nice to have a system for backends so that we could do this when 
> certain specific situations call for it (usually related to TLS clients, 
> rather than servers, although having both would be great), but no, the main 
> motivation is to drop OpenSSL entirely in favor of Rustls, as recommended by 
> the ISRG: 
>  >.

That is a great goal for Twisted.

Barry




___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] twisted.web HTTPS client certificate

[Glyph]

> On Jul 13, 2021, at 2:09 AM, Barry Scott  wrote:
> 
> On Monday, 12 July 2021 09:27:19 BST Glyph wrote:
>> FWIW I would avoid calling the pyOpenSSL APIs for this, since hopefully 
>> we'll manage to move away from OpenSSL and at least somewhat abstract away 
>> the transition. 
> 
> Are you thinking to use the Windows and macOS provided crypto API and only 
> use openssl on Unix systems?
> 
> Barry

It would be nice to have a system for backends so that we could do this when 
certain specific situations call for it (usually related to TLS clients, rather 
than servers, although having both would be great), but no, the main motivation 
is to drop OpenSSL entirely in favor of Rustls, as recommended by the ISRG: 
>.

-g___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] twisted.web HTTPS client certificate

[Barry Scott]

On Monday, 12 July 2021 09:27:19 BST Glyph wrote:
> FWIW I would avoid calling the pyOpenSSL APIs for this, since hopefully we'll 
> manage to move away from OpenSSL and at least somewhat abstract away the 
> transition. 

Are you thinking to use the Windows and macOS provided crypto API and only use 
openssl on Unix systems?

Barry




___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] twisted.web HTTPS client certificate

[Glyph]

> On Jul 12, 2021, at 1:05 AM, Adi Roiban  wrote:
> 
> On Mon, 12 Jul 2021 at 08:09, Ian Haywood  > wrote:
> I am trying to work out how to retrieve on the server a X.509 
> certificate presented by the HTTPS client.  This code tries to tell me 
> the transport has no peer certificate.
> 
> same error when I use wget as a client so I think my problem is in the 
> server code. I'm using self-signed certificates
> 
> Any clues as to what I'm doing wrong?
> 
> Ian
> 
> 
> As commented by Glyph you will first need  to setup one or a list of accepted 
> CAs.
> 
> Then setup custom server-side context.
> 
> Add the CA via 
> https://www.pyopenssl.org/en/stable/api/ssl.html#OpenSSL.SSL.Context.load_verify_locations
>  
> 
> 
> This is done via 
> https://www.pyopenssl.org/en/stable/api/ssl.html#OpenSSL.SSL.Context.set_verify
>  
> 
>  to define a path or dir
> or implement a custom one via 
> https://www.pyopenssl.org/en/stable/api/ssl.html#OpenSSL.SSL.Context.get_cert_store
>  
> 

FWIW I would avoid calling the pyOpenSSL APIs for this, since hopefully we'll 
manage to move away from OpenSSL and at least somewhat abstract away the 
transition.  These map to the 'caCerts' and 'verify=True' arguments to 
CertificateOptions, if you need more flexibility than the tutorial 
documentation that I linked previously: 
https://twistedmatrix.com/documents/20.3.0/api/twisted.internet.ssl.CertificateOptions.html
 

> Without set_verify, during the TLS/SSL handshake the server will not ask the 
> client to send its own certificate.
> 
> -
>  
> I am using X509 authentication as a  passwordless authentication for 
> automated transfers, similar to the SSH key authentication.
> 
> The x509 certificate authentication is used by the Spanish government across 
> many of their services.
> Taxes, customs, health service ...
> 
> -- 
> Adi Roiban
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] twisted.web HTTPS client certificate

[Adi Roiban]

On Mon, 12 Jul 2021 at 08:09, Ian Haywood  wrote:

> I am trying to work out how to retrieve on the server a X.509
> certificate presented by the HTTPS client.  This code tries to tell me
> the transport has no peer certificate.
>
> same error when I use wget as a client so I think my problem is in the
> server code. I'm using self-signed certificates
>
> Any clues as to what I'm doing wrong?
>
> Ian
>
>
As commented by Glyph you will first need  to setup one or a list of
accepted CAs.

Then setup custom server-side context.

Add the CA via
https://www.pyopenssl.org/en/stable/api/ssl.html#OpenSSL.SSL.Context.load_verify_locations

This is done via
https://www.pyopenssl.org/en/stable/api/ssl.html#OpenSSL.SSL.Context.set_verify
to define a path or dir
or implement a custom one via
https://www.pyopenssl.org/en/stable/api/ssl.html#OpenSSL.SSL.Context.get_cert_store

Without set_verify, during the TLS/SSL handshake the server will not ask
the client to send its own certificate.

-

I am using X509 authentication as a  passwordless authentication for
automated transfers, similar to the SSH key authentication.

The x509 certificate authentication is used by the Spanish government
across many of their services.
Taxes, customs, health service ...

-- 
Adi Roiban
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] twisted.web HTTPS client certificate

[Glyph]

> On Jul 12, 2021, at 12:07 AM, Ian Haywood  wrote:
> 
> I am trying to work out how to retrieve on the server a X.509 certificate 
> presented by the HTTPS client.  This code tries to tell me the transport has 
> no peer certificate.
> 
> same error when I use wget as a client so I think my problem is in the server 
> code. I'm using self-signed certificates
> 
> Any clues as to what I'm doing wrong?

It's been quite a while since I tried to do this, but:

For starters, you need to specify the certificate authority that will be 
validating your client's certificates.  I forget the precise specifics, but I 
believe if you don't specify a CA that will be used, it won't prompt the client 
to present a certificate in the first place, and also there's nothing for your 
endpoint to validate against, so what would it even do if the client did 
present one, other than let you retrieve it? The point is not to inspect the 
certificate but to authenticate it, an API that let you inspect it without 
first validating it against a CA would be a security nightmare.  And generally 
client certs are not understood to be validated by webtrust CAs, so "just 
validate it like usual" doesn't apply, since you can't check the SANs for a 
hostname.

There's a guide to doing this programmatically here: 
https://docs.twistedmatrix.com/en/twisted-21.2.0/core/howto/ssl.html#tls-server-with-client-authentication-via-client-certificate-verification
 

 - I believe that this edge-case is not supported by endpoints.serverFromString.

If it's within your control to avoid, don't use client certificate 
authentication from HTTPS clients.  It unnecessarily leaks a bunch of 
peripheral information to the service you're authenticating to, the UX is a 
disaster on basically every browser, and most of the big players have stopped 
caring about this use-case in favor of things like webauthn that properly exist 
entirely outside of the setup process for a secure channel.  I confess I used 
to think client cert auth was really cool myself, but luckily nothing I did 
with it ever took off :-).

That said: as long as the use-case exists, Twisted should have good support for 
it.  Adding a clientCA argument to twisted.internet.endpoints._parseSSL so that 
serverFromString would support it would be a pretty simple PR to put together, 
so if you've got to bite the bullet on this, a contribution to close the gap 
here would be appreciated.  Twisted is nothing if not a tool that should make 
it seamless to integrate with every bad idea anyone's had in a protocol design 
;-).

-g

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] twisted.web HTTPS client certificate

[Ian Haywood]

I am trying to work out how to retrieve on the server a X.509 
certificate presented by the HTTPS client.  This code tries to tell me 
the transport has no peer certificate.


same error when I use wget as a client so I think my problem is in the 
server code. I'm using self-signed certificates


Any clues as to what I'm doing wrong?

Ian

from twisted.internet import reactor, endpoints
from twisted.internet.ssl import Certificate
from twisted.web.server import Site
from twisted.web.resource import Resource


class CertPage(Resource):
isLeaf = True

def render_GET(self, request):
HTML = """

getPeerCertificate %r %r
"""
cert = Certificate.peerFromTransport(request.transport)
return bytes(HTML % (type(cert), cert), "ascii")


resource = CertPage()
site = Site(resource)
e = endpoints.serverFromString(
reactor, "ssl:8443:certKey=server.crt:privateKey=server.key"
)
e.listen(site)
reactor.run()
from zope.interface import implementer

from OpenSSL.crypto import FILETYPE_PEM
from twisted.internet.ssl import (
optionsForClientTLS,
Certificate,
PrivateCertificate,
KeyPair,
)
from twisted.trial import unittest
from twisted.internet import reactor
from twisted.web.iweb import IPolicyForHTTPS
from twisted.web.client import Agent, ResponseFailed, readBody


@implementer(IPolicyForHTTPS)
class LoadClientCert:
def __init__(self, hostmap, server_cert, client_cert, client_key):
self.hostmap = hostmap
with open(server_cert) as fd:
self.server_cert = Certificate.loadPEM(fd.read())
with open(client_cert) as fd:
client_cert = Certificate.loadPEM(fd.read())
with open(client_key) as fd:
client_key = KeyPair.load(fd.read(), FILETYPE_PEM)
self.client_cert = PrivateCertificate.fromCertificateAndKeyPair(
client_cert, client_key
)

def creatorForNetloc(self, hostname, port):
hostname = hostname.decode("ascii")
if hostname in self.hostmap:
hostname = self.hostmap[hostname]

import pdb

pdb.set_trace()
return optionsForClientTLS(hostname, self.server_cert, self.client_cert)


def getPage(url, server_cert, client_cert, client_key, hostmap={}):
a = Agent(reactor, LoadClientCert(hostmap, server_cert, client_cert, client_key))
d = a.request(b"GET", url.encode("ascii"))

def cb_getBody(response):
return readBody(response)

return d.addCallback(cb_getBody)


class HubTest(unittest.TestCase):
def test_getpage(self):
def cb_print(s):

print(s)

DIR = "/home/ian/athen/hub/"
d = getPage(
"https://localhost:8443/;,
DIR + "server.crt",
DIR + "client.crt",
DIR + "client.key",
{"localhost": "mintbox"},
)
d.addCallback(cb_print)
return d
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python