Re: [pylons-discuss] Problems when using Gunicorn and Postgresql together

[Mike Orr]

On Mon, Nov 20, 2023 at 4:14 PM Jonathan Vanasco  wrote:
>
> SQLAlchemy supports this via `Engine.dispose()`, which is the documented way 
> of handling a post-fork connection:
>
>https://docs.sqlalchemy.org/en/13/core/connections.html#engine-disposal

Yes, that sounds familiar.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/CAH9f%3DuoNxgs5ufQT7p8%2BPobzeS0COJPrEP-8o%3D%2BQciCrbyFU%2BQ%40mail.gmail.com.

Re: [pylons-discuss] Re: Using SSL client certificate in a Pyramid application

[Jonathan Vanasco]

100% agree with Michael's comments about signed headers.  When dealing with 
credentials, a lot of people distrust proxies and testing them - so on top 
of removing all headers, anything added will be signed (or the aggregate 
signed) with a low-cost method, which can easily be tested.  
On Friday, November 17, 2023 at 2:30:12 PM UTC-5 Michael Merickel wrote:

> You don’t need a signed header. You just need to make sure your proxy is 
> configured to remove that header from an incoming request and only put 
> validated data into it before sending it to your app. This is standard for 
> other headers set by your proxy as well. Definitely never trust a header 
> without checking those properties of your deployment though. 
>
> - Michael
>
> On Nov 17, 2023, at 10:02, Jonathan Vanasco  wrote:
>
> 
>
> I do a lot of work with SSL Certs.  If you are using publicly trusted 
> client certificates (i.e. LetsEncrypt, Digisign, etc), then you basically 
> just need to do what Michael suggested - ensure your gateway or server 
> populates the headers or environment variables with the information for the 
> client.  Most implementations for that stuff will also insert a 
> secret-signed header field to ensure the client did not try and stuff 
> headers themselves.  You'd then need to validate the certificate again 
> within python (see workaround #3 below)
>
> I think this is going to be a bit of a headache if you are using untrusted 
> CAs for the client certificates (ie. you control the CA and allocate the 
> certs to your clients/partners) and trying to implement the sort of policy 
> that PostgreSQL and other systems offer with client verification.
>
> The complexity and level of work required come down to the following 
> factors:
>
> * Who will issue the client certificates (a public trusted CA or your own 
> private CA)
> * How much validation can you split between the gateway/server and Python 
> (based on application design and business logic constraints), and where can 
> the errors pop-up?
>
> If you are talking about custom CA certs, the two approaches I know of for 
> this sort of stuff:
>
> 1- (Easy for developer, harder for client) The client application stuffs 
> the certificate into a header.  Everything happens as normal Pyramid app.  
> You just write pyramid code to require and validate the certificate 
> header.  This isn't actually mTLS, but uses the client certificate similar 
> to ssl keychains.
>
> 2. (Hard for developer, easy for client) Actually use TLS Mutual Auth, 
> which is what Theron first brought up.  With mTLS, the client certificate 
> is presented and validated during the TLS handshake - which means you would 
> need to configure the gateway or mod_wsgi to handle the validation.
>
> The large complexity of #2 is that many web servers / gateways can only be 
> configured with a single client CA or a static client CA store - so 
> building this as scalable to multiple clients may not necessarily be 
> possible.
>
> The three workarounds I can think of are:
> 1- Don't terminate TLS or validate the client certificate on the gateway, 
> instead pass it all back to a python server like Waitress and see if you 
> can handle everything via middleware to process the SSL connection.
>
> 2- Terminate TLS on OpenResty(nginx) and use a Lua script to handle the 
> client validation.  Then you can proxy that back to mod_wsgi.  I 
> opensourced our dynamic SSL certificate tool for OpenResty here - 
> https://github.com/aptise/lua-resty-peter_sslers - it does not do client 
> certificates, but IMHO is a good reference for what you want to 
> accomplish.it looks like OpenResty has some builtin stuff for this, but 
> if not - those are the correct hooks.  
>
> 3- Terminate TLS on the gateway but make it very lax for the client 
> certificate to pass - basically just checking to see if the certificate is 
> valid and from the CA.  Then validate the client a second time in 
> middleware/tween to ensure the certificate is active and allowed.  
>
>
> On Wednesday, November 15, 2023 at 9:13:39 AM UTC-5 Thierry Florac wrote:
>
>> Hi,
>> My problem is probably quite simple: I would like to be able, in a 
>> Pyramid application, to create a custom security policy which could use an 
>> SSL client certificate as a request credential to handle authentication 
>> (authorized certificates being referenced in a database or stored in a 
>> specific server directory).
>> This application is then supposed to be published via mod_wsgi in an 
>> Apache server located behind an HAProxy.
>> I tried to search here and there but didn't find any information about 
>> this...
>> Any hint?
>>
>> Best regards,
>> Thierry
>> -- 
>>   https://www.ulthar.net -- http://pyams.readthedocs.io
>>
> -- 
>
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discus...@googlegroups.com.
>

Re: [pylons-discuss] Problems when using Gunicorn and Postgresql together

[Jonathan Vanasco]

SQLAlchemy supports this via `Engine.dispose()`, which is the documented 
way of handling a post-fork connection:

   https://docs.sqlalchemy.org/en/13/core/connections.html#engine-disposal

Invoking `Pool.recreate()` should be fine, but the documented pattern is to 
call `Engine.dispose`

> How should I modify my app to make it compatible with both Waitress and 
Gunicorn?

I've been trying to standardize this with pyramid_forkfsafe when I was 
deploying projects on uwsgi and gunicorn simultaneously:

https://pypi.org/project/pyramid-forksafe/
https://github.com/jvanasco/pyramid_forksafe

I don't know if the gunicorn is currently working, as I only work with 
uwsgi now and never got around to writing tests for gunicorn.

Ideally, my package works like this:

* it defines a new `ApplicationPostFork` event
* you write your post-fork code in an event subscriber
* the package invokes the event at the right time

This allows you to just define the post-fork routine, and if everything 
goes correctly pyramid_forksafe will detect the correct post-fork event 
and invoke your routine.
On Thursday, November 16, 2023 at 12:18:59 PM UTC-5 Theron Luhn wrote:

> If you aren’t using `—preload` then gunicorn should load the application 
> fresh for each worker and you shouldn’t have any issues.
>
> If you are using preload, you have to recreate any existing connections on 
> fork.  For SQLAlchemy I use:
>
> def after_fork(registry):
> registry['db_engine'].pool.recreate()
>
> def includeme(config):
> os.register_at_fork(
> after_in_child=functools.partial(after_fork, config.registry),
> )
>
>
> — Theron
>
>
>
> On Nov 16, 2023, at 7:41 AM, Laurent Daverio  wrote:
>
> Hello list,
>
> this page seems to describe perfectly a problem I've stumbled on: 
>
>
> https://stackoverflow.com/questions/64995178/decryption-failed-or-bad-record-mac-in-multiprocessing
>
> Namely, if you deploy with Gunicorn a Pyramid + PostgreSQL app based on 
> the standard cookiecutter, you will run into problems, because the 
> connection to the DB can't be shared between the processes, so each process 
> needs to have its own connection to the DB.
>
> Before I start trying to develop a workaround, has anybody encountered the 
> problem? How should I modify my app to make it compatible with both 
> Waitress and Gunicorn?
>
> Thanks in advance,
>
> Laurent.
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discus...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/CAB7cU6z1DqHpEazrrJ1sPHmSPQvYtfkmeKfsJP_jLmsDyPA96w%40mail.gmail.com
>  
> 
> .
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/459deec2-f880-494d-afc4-bfc300f15093n%40googlegroups.com.