Re: [pylons-discuss] request.resource_url or request.resource_path

[Michael Merickel]

It really doesn’t make a difference imo. If you are using path to get around a 
proxy misconfig in your WSGI env then stop and go fix the misconfig because as 
Mike said, certain things like a Location header in a redirect are required to 
be absolute and will be incorrectly computed if your env’s http scheme, host, 
and script_name aren’t right. 

- Michael

> On May 29, 2024, at 09:04, Mike Orr  wrote:
> 
> I prefer the '*_path' methods to get relative URLs that automatically
> adjust to more environments, and to minimize the chance of mismatches
> behind reverse proxies. Except in redirects where I've read the target
> is supposed to be an absolute URL. Some other Pyramid developers
> prefer the '*_url' methods.
> 
>> On Tue, May 28, 2024 at 11:32 PM Petr Blahoš  wrote:
>> 
>> Hi,
>> I have been using pyramid for many years and I have always been
>> using
>> 
>> request.resource_url
>> request.static_url
>> or similar ending with _url.
>> 
>> Yesterday when trying to make IIS work as a reverse proxy I noticed
>> that the links in my pages contain scheme://hostname:port part of the
>> URL too. So I am thinking: Should I always use the resource_path / 
>> static_path
>> functions to make the URLs?
>> 
>> What are your thoughts?
>> Thanks,
>> Petr
>> 
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "pylons-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to pylons-discuss+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/pylons-discuss/CA%2ByMeXVZ%3DxN6yYCC-oRU3W%3D7J52xzAuQfVwJSyyKep4gpg%3Do5Q%40mail.gmail.com.
> 
> 
> 
> --
> Mike Orr 
> 
> --
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/CAH9f%3Duqh7TCu2e5Upy2hSt7gh8BxxMdv8b6gRqwVeYoZEnF11Q%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/565FE6B2-EDE7-4B29-AEBB-7FD332629E57%40gmail.com.

Re: [pylons-discuss] Question about creating routes

[Michael Merickel]

Laurent, what error do you get? I'm not immediately aware of limitations
around using lambdas in Pyramid. This example below works fine with a
lambda as a view:

from pyramid.config import Configurator
from waitress import serve

config = Configurator()
config.add_route('foo', '/')
config.add_view(lambda req: 'foo', route_name='foo', renderer='string')
app = config.make_wsgi_app()
serve(app, listen='*:8080')

On Wed, Mar 13, 2024 at 2:26 PM Mike Orr  wrote:

> I'm not an expert in this, but I think the maybe-resolve functions
> only do lookups when the spec is a string, and pass non-strings
> through unchanged. As far as I know lambdas are regular functions, so
> I'd expect it to pass them through like view callable functions. Maybe
> another exception is occurring, and when Python tries to print the
> traceback, it can't find the lambda's source.
>
> I generally use regular functions instead of lambdas, but I realize
> this is a stylistic choice and some think otherwise. A partial might
> work in your use case:
> config.add_view( functools.partial(_delete_multi,
> arg2_keyword=Doc2Resource) , ...
>
> On Wed, Mar 13, 2024 at 10:50 AM Laurent Daverio 
> wrote:
> >
> > Hello list,
> >
> > it seems that Google is allowing me to post on here again. A couple of
> weeks ago, I was banned, both from emailing the list, AND from posting on
> the web group 💩
> >
> > I am trying to create a series of routes based on lambda functions,
> something like:
> >
> > ```
> > config.add_route('doc1.delete_selection', '/doc1/-/delete_selection')
> > config.add_route('doc2.delete_selection', '/doc2/-/delete_selection')
> > config.add_view(
> > lambda request: _delete_multi(request, Doc1Resource),
> > route_name='doc1.delete_selection', request_method='POST',
> renderer='json')
> > config.add_view(
> > lambda request: _delete_multi(request, Doc2Resource),
> > route_name='doc2.delete_selection', request_method='POST',
> renderer='json')
> > ```
> >
> > (I also have `archive_selection` and `unarchive_selection` routes)
> >
> > I'm under the impression that `config.add_view` can't take lambda
> functions as arguments, as it tries to resolve their names ("maybe_dotted",
> "maybe_resolve", etc.). All I can say it that the views seem to be calling
> the wrong functions. This prevents me from using for loops and lambda for
> generating the routes in a more concise way :/
> >
> > Could anyone confirm if I'm correct?
> >
> > Many thanks,
> >
> > Laurent.
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups "pylons-discuss" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to pylons-discuss+unsubscr...@googlegroups.com.
> > To view this discussion on the web visit
> https://groups.google.com/d/msgid/pylons-discuss/CAB7cU6y56gx3fDEGsdx3qFJKTVPOM5yN1maohu8tEESRnY2ygQ%40mail.gmail.com
> .
>
>
>
> --
> Mike Orr 
>
> --
> You received this message because you are subscribed to the Google Groups
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pylons-discuss/CAH9f%3Duq3ENBFg13AfwbH%2BivU2yzP7zORkRAREJefS0Wg1MAJaQ%40mail.gmail.com
> .
>


-- 

Michael

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwECMJJyV1C9PcHET0%3DuGiQSbz09%2BP%2B8hyyPx4wL6W%3DUPg%40mail.gmail.com.

Re: [pylons-discuss] request for help upgrading the cookiecutter to support sqlalchemy 2.0

[Michael Merickel]

I've chatted a little bit with Laurent Daverio on the side but yeah for the
most part:

- I upgraded the cookiecutter-starter to use pyproject.toml and SQLAlchemy
2.0 patterns. That branch is here
https://github.com/Pylons/pyramid-cookiecutter-starter/tree/main
- I skimmed alembic changes and didn't see something worth fixing in our
setup, but open to feedback.
- I upgraded the wiki2 tutorial in Pyramid to use the new SQLA 2.0
patterns. That PR is here https://github.com/Pylons/pyramid/pull/3747 and I
left it open for a bit in case someone wanted to look.

I would love if someone synced the pyproject.toml and SQLA changes into the
quick tutorial and the zodb tutorial (should be a lot easier than the wiki2
tutorial I did above).

Other than that - I'm trying to evaluate the feasibility of making
pkg_resources optional in Pyramid before cutting a new release. My
ramblings about that topic are here if you want to track it for any reason.
https://github.com/Pylons/pyramid/issues/3731

Thanks!

On Mon, Feb 5, 2024 at 2:55 PM Jonathan Vanasco  wrote:

> I don't have time to do this, but would be happy to be looped in for code
> review if there is another PR.
>
> It looks like you've gotten most things done already.
>
> I think the alembic integration may be out of date, as those files haven't
> been touched for a while.  I can ask Federico on the SqlAlchemy team to
> take a look at the current setup, he's been the main force behind
> maintaining and expanding Alembic.  The core devs meet on Wednesdays on
> gitter.
>
>
> On Monday, January 29, 2024 at 5:08:54 AM UTC-5 Laurent Daverio wrote:
>
>> Hello list,
>>
>> I think I could do that, unless someone has volunteered already (I have A
>> LOT to do these days, wouldn't want to duplicate someone else's work).
>>
>> Laurent.
>>
>>
>> Le lun. 29 janv. 2024 à 07:51, Michael Merickel  a
>> écrit :
>>
>>> Hey folks,
>>>
>>> I'd really appreciate it if someone was willing to take the time to
>>> upgrade the pyramid-cookiecutter-starter to use SQLAlchemy 2.0 best
>>> practices. Things like rewriting the queries away from Query to simple
>>> select(), and using the more modern declarative mapper that is typing
>>> friendly.
>>>
>>> https://github.com/Pylons/pyramid-cookiecutter-starter/tree/main
>>>
>>> Once we get that upgraded, we'd need to fix up the wiki2 tutorial to
>>> reference the updated files accurately - but first step is just getting the
>>> cookiecutter updated.
>>>
>>> https://github.com/Pylons/pyramid/tree/main/docs/tutorials/wiki2
>>>
>>> Thanks!
>>>
>>> - Michael
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "pylons-discuss" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to pylons-discus...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/pylons-discuss/54913600-1D9F-47E9-B991-136D0B5A73F3%40gmail.com
>>> <https://groups.google.com/d/msgid/pylons-discuss/54913600-1D9F-47E9-B991-136D0B5A73F3%40gmail.com?utm_medium=email&utm_source=footer>
>>> .
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pylons-discuss/ea574da1-7b7e-4a88-ac4b-83950161a5dcn%40googlegroups.com
> <https://groups.google.com/d/msgid/pylons-discuss/ea574da1-7b7e-4a88-ac4b-83950161a5dcn%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>


-- 

Michael

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwHHL1NKsmJ7VemuKSAiKM3sXKm6-1M9JJ1B5iQ9CcQgLw%40mail.gmail.com.

[pylons-discuss] pyramid_debugtoolbar 4.12 released

[Michael Merickel]

pyramid-debugtoolbar 4.12 has been released.

- Fixed annoying deprecation warnings on Pyramid 2.0.
- Removed dependency on setuptools.

Full change history:
https://docs.pylonsproject.org/projects/pyramid_debugtoolbar/en/latest/changes.html

You can install it via PyPI:

 pip install pyramid-debugtoolbar==4.12

Enjoy, and please report any issues you find to the issue tracker at
https://github.com/Pylons/pyramid_debugtoolbar/issues

 
Thanks!

- Pylons Project core developers

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/48727353-D51F-4192-8FA1-FDDBF6A37A59%40gmail.com.

[pylons-discuss] Enabling Github Discussions

[Michael Merickel]

Hey folks,

I've enabled the discussions feature on the pyramid repository as an 
alternative form of communication if it simplifies communication for anyone. We 
shall see!

https://github.com/Pylons/pyramid/discussions

- Michael

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/E85AAE9E-C47A-4724-A605-FFE46A90837E%40gmail.com.

[pylons-discuss] request for help upgrading the cookiecutter to support sqlalchemy 2.0

[Michael Merickel]

Hey folks,

I'd really appreciate it if someone was willing to take the time to upgrade the 
pyramid-cookiecutter-starter to use SQLAlchemy 2.0 best practices. Things like 
rewriting the queries away from Query to simple select(), and using the more 
modern declarative mapper that is typing friendly.

https://github.com/Pylons/pyramid-cookiecutter-starter/tree/main

Once we get that upgraded, we'd need to fix up the wiki2 tutorial to reference 
the updated files accurately - but first step is just getting the cookiecutter 
updated.

https://github.com/Pylons/pyramid/tree/main/docs/tutorials/wiki2

Thanks!

- Michael

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/54913600-1D9F-47E9-B991-136D0B5A73F3%40gmail.com.

[pylons-discuss] pyramid-debugtoolbar 4.11 released

[Michael Merickel]

pyramid-debugtoolbar 4.11 has been released.

- Added support for Python 3.12.
- Added support for SQLAlchemy 2.x.

Full change history:
https://docs.pylonsproject.org/projects/pyramid_debugtoolbar/en/latest/changes.html

You can install it via PyPI:

 pip install pyramid-debugtoolbar==4.11

Enjoy, and please report any issues you find to the issue tracker at
https://github.com/Pylons/pyramid_debugtoolbar/issues

 
Thanks!

- Pylons Project core developers

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/A3B18DBA-B80A-4F66-BD50-65420806F8FC%40gmail.com.

[pylons-discuss] hupper 1.12.1 released

[Michael Merickel]

hupper 1.12.1 has been released.

Full change history:
https://docs.pylonsproject.org/projects/hupper/en/latest/changes.html

You can install it via PyPI:

 pip install hupper==1.12.1

Enjoy, and please report any issues you find to the issue tracker at
https://github.com/Pylons/hupper/issues

Thanks!

- Pylons Project core developers

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/956FB1D8-A638-4601-8E77-7695FDBCBD43%40gmail.com.

Re: [pylons-discuss] Re: Using SSL client certificate in a Pyramid application

[Michael Merickel]

You don’t need a signed header. You just need to make sure your proxy is configured to remove that header from an incoming request and only put validated data into it before sending it to your app. This is standard for other headers set by your proxy as well. Definitely never trust a header without checking those properties of your deployment though. - MichaelOn Nov 17, 2023, at 10:02, Jonathan Vanasco  wrote:I do a lot of work with SSL Certs.  If you are using publicly trusted client certificates (i.e. LetsEncrypt, Digisign, etc), then you basically just need to do what Michael suggested - ensure your gateway or server populates the headers or environment variables with the information for the client.  Most implementations for that stuff will also insert a secret-signed header field to ensure the client did not try and stuff headers themselves.  You'd then need to validate the certificate again within python (see workaround #3 below)I think this is going to be a bit of a headache if you are using untrusted CAs for the client certificates (ie. you control the CA and allocate the certs to your clients/partners) and trying to implement the sort of policy that PostgreSQL and other systems offer with client verification.The complexity and level of work required come down to the following factors:* Who will issue the client certificates (a public trusted CA or your own private CA)* How much validation can you split between the gateway/server and Python (based on application design and business logic constraints), and where can the errors pop-up?If you are talking about custom CA certs, the two approaches I know of for this sort of stuff:1- (Easy for developer, harder for client) The client application stuffs the certificate into a header.  Everything happens as normal Pyramid app.  You just write pyramid code to require and validate the certificate header.  This isn't actually mTLS, but uses the client certificate similar to ssl keychains.2. (Hard for developer, easy for client) Actually use TLS Mutual Auth, which is what Theron first brought up.  With mTLS, the client certificate is presented and validated during the TLS handshake - which means you would need to configure the gateway or mod_wsgi to handle the validation.The large complexity of #2 is that many web servers / gateways can only be configured with a single client CA or a static client CA store - so building this as scalable to multiple clients may not necessarily be possible.The three workarounds I can think of are:1- Don't terminate TLS or validate the client certificate on the gateway, instead pass it all back to a python server like Waitress and see if you can handle everything via middleware to process the SSL connection.2- Terminate TLS on OpenResty(nginx) and use a Lua script to handle the client validation.  Then you can proxy that back to mod_wsgi.  I opensourced our dynamic SSL certificate tool for OpenResty here - https://github.com/aptise/lua-resty-peter_sslers - it does not do client certificates, but IMHO is a good reference for what you want to accomplish.    it looks like OpenResty has some builtin stuff for this, but if not - those are the correct hooks.  3- Terminate TLS on the gateway but make it very lax for the client certificate to pass - basically just checking to see if the certificate is valid and from the CA.  Then validate the client a second time in middleware/tween to ensure the certificate is active and allowed.  On Wednesday, November 15, 2023 at 9:13:39 AM UTC-5 Thierry Florac wrote:Hi,My problem is probably quite simple: I would like to be able, in a Pyramid application, to create a custom security policy which could use an SSL client certificate as a request credential to handle authentication (authorized certificates being referenced in a database or stored in a specific server directory).This application is then supposed to be published via mod_wsgi in an Apache server located behind an HAProxy.I tried to search here and there but didn't find any information about this...Any hint?Best regards,Thierry--   https://www.ulthar.net -- http://pyams.readthedocs.io




-- 
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/39f47b19-0e08-4936-aaff-3da5822dc90fn%40googlegroups.com.




-- 
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/F3D4E89A-A27F-4923-8ACA-DF1AF2B6821B%40gmail.com.

Re: [pylons-discuss] Using SSL client certificate in a Pyramid application

[Michael Merickel]

Another example is that Envoy as a proxy supports adding the x-forwarded-client-cert header to your requests after it validates it.HTTP header manipulation — envoy 1.29.0-dev-bbb4e6 documentationenvoyproxy.io- MichaelOn Nov 15, 2023, at 13:25, Delta Regeer  wrote:mod_ssl can stuff the information about the mTLS information into server environment variables. If you are using mod_wsgi you should be able to retrieve those. mod_ssl will validate the certificate is valid, and place the information (such as subject name) in the environment and you can use that in your Pyramid application by pulling it out of the environ.See https://httpd.apache.org/docs/trunk/mod/mod_ssl.html#envvars for more information.You can then write a Pyramid authentication provider that validates the information in the environment.On Nov 15, 2023, at 11:06, Thierry Florac  wrote:Hi Theron,I'm not sure of the exact naming of this!The common idea behind it is just to use an SSL client certificate as a credential to authenticate an incoming request; this is generally used to authenticate a remote application more than a common user...Regards,Thierry--   https://www.ulthar.net -- http://pyams.readthedocs.ioLe mer. 15 nov. 2023 à 18:43, Theron Luhn  a écrit :I’m unsure what this “request credential” is.  Are you talking about TLS Mutual Auth?
— Theron

On Nov 15, 2023, at 6:13 AM, Thierry Florac  wrote:Hi,My problem is probably quite simple: I would like to be able, in a Pyramid application, to create a custom security policy which could use an SSL client certificate as a request credential to handle authentication (authorized certificates being referenced in a database or stored in a specific server directory).This application is then supposed to be published via mod_wsgi in an Apache server located behind an HAProxy.I tried to search here and there but didn't find any information about this...Any hint?Best regards,Thierry--   https://www.ulthar.net -- http://pyams.readthedocs.io

-- 
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWBosR7p%3DLb%2BzEXWKuwuuENy6CORPrVpHaRMU9qWV4uW4g%40mail.gmail.com.


-- 
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/57B4950A-F6F9-432B-81C8-81566502F94C%40luhn.com.


-- 
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWD9vFNoJ1tqUs3_PoP7AB6P%3D6cDiawLQ66FYy2NJR-fSA%40mail.gmail.com.




-- 
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/9D206192-A136-41A4-AB5D-0F899F055B0F%400x58.com.




-- 
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/705EFD2B-6A66-4504-A76B-207CD14BA0FD%40gmail.com.

Re: [pylons-discuss] Getting error when running inspect in python 3.11, win11

[Michael Merickel]

Just fyi there is a pull request on pastedeploy that patches the inspect issue 
on 3.12. I’m planning to get that released when I can find time this week. 

- Michael

> On Nov 14, 2023, at 15:34, Laurent Daverio  wrote:
> 
> Glad to hear that, and many thanks for offering :) I'll let you know if I 
> need.
> 
> You just mentioned a database, and it may be totally
> irrelevant/unrelated, but it occurs to me that SQLAlchemy also has an
> "inspect" function, which you use for introspecting your database. I
> do use it, unlike the standard "inspect" :)
> 
> All the best,
> 
> Laurent.
> 
>> Le mar. 14 nov. 2023 à 22:29, Oberdan Santos  a écrit :
>> 
>> 
>> Thanks for the support. The error is gone, now I can see communication with 
>> the database.
>> My specialty is developing prediction models (Machine learning). If you need 
>> anything in this field, I am at your disposal.
>>> Em terça-feira, 14 de novembro de 2023 às 17:17:54 UTC-3, Laurent Daverio 
>>> escreveu:
>>> 
>>> Our messages crossed, my previous one contains the answer to your question:
>>> 
>>> https://github.com/Pylons/pastedeploy/blame/main/src/paste/deploy/util.py#L36
>>> 
>>> Le mar. 14 nov. 2023 à 21:13, Oberdan Santos  a écrit :
 
 Checking the literature on some sites, I see that I have two options ( 
 inspect.signature() or inspect.getfullargspec() to solve the problem. The 
 question is: How to use inspect.signature() or the other option, where to 
 change it ? .The following is the function that was throwing error.
 
 import inspect
 import sys
 
 try:
 import importlib.metadata as importlib_metadata # noqa F401
 except ImportError: # pragma: no cover
 # bw-compat shim for py37
 import importlib_metadata # noqa F401
 
 
 def fix_type_error(exc_info, callable, varargs, kwargs):
 """
 Given an exception, this will test if the exception was due to a
 signature error, and annotate the error with better information if
 so.
 
 Usage::
 
 try:
 val = callable(*args, **kw)
 except TypeError:
 exc_info = fix_type_error(None, callable, args, kw)
 raise exc_info[0], exc_info[1], exc_info[2]
 """
 if exc_info is None:
 exc_info = sys.exc_info()
 if (
 exc_info[0] != TypeError
 or str(exc_info[1]).find('arguments') == -1
 or getattr(exc_info[1], '_type_error_fixed', False)
 ):
 return exc_info
 exc_info[1]._type_error_fixed = True
 argspec = inspect.formatargspec(*inspect.getargspec(callable))
 args = ', '.join(map(_short_repr, varargs))
 if kwargs and args:
 args += ', '
 if kwargs:
 kwargs = sorted(kwargs.items())
 args += ', '.join(['%s=...' % n for n, v in kwargs])
 gotspec = '(%s)' % args
 msg = f'{exc_info[1]}; got {gotspec}, wanted {argspec}'
 exc_info[1].args = (msg,)
 return exc_info
 
 
 
 Desde já agradeço,
 ]
 Oberdan Costa
 
 Em terça-feira, 14 de novembro de 2023 às 16:15:50 UTC-3, Oberdan Santos 
 escreveu:
> 
> Thanks again for your attention. I will provide a brief contextualization.
> I have a script based on FASTAPI for predicting the risk of chronic 
> non-communicable diseases to aid clinical decision making in primary 
> health care (backend), for now I will leave it like that. it works 
> perfectly.
> On the frontend I use streamlit to obtain the results of up to 10 chronic 
> diseases simultaneously. This ended up growing to include other modules 
> (routing and intelligent management). At the moment I'm looking for a 
> robust framework for this application purpose and the pyramid environment 
> has allowed me to move in this direction (I'm based on cookiecutter) for 
> my first experiences. I have a modular division already structured.
> Now I'm trying to connect to my database structure and I'm encountering 
> this error when I run my application. From the request: I need help to 
> take this step and fix this problem.
> 
> Em terça-feira, 14 de novembro de 2023 às 16:00:06 UTC-3, Laurent Daverio 
> escreveu:
>> 
>>> The application I am requesting support for is based on the Pyramd 
>>> Cookiecutter application framework. In this sense, my question is 
>>> pertinent.
>> 
>> Oh, I see why you posted here, sorry then. For my defence, you
>> provided no context, no traceback, nothing indicating you were indeed
>> referring to Pyramid. You only wrote "When trying to run my code I am
>> receiving an error", and your code was only standard Python.
>> 
>>> Why does pyramid continue to carry this module in its base application 
>>> structures, such as cookiecutter?
>> 
>> Actually, I'm not sure what problem you encountered :
>> 
>> - it could be in the cookiecutter module, which is not related to
>> Pyramid at all. It's definitely not a "base application st

[pylons-discuss] Pyramid 2.0.2 released

[Michael Merickel]

Pyramid 2.0.2 has been released.

This release fixes a security advisory affecting static views on Pyramid 2.0 
and 2.0.1 installed on Python 3.11.x prior to 3.11.5. If this affects you, 
please read the changelog for more details!

The full changelog is here:
https://docs.pylonsproject.org/projects/pyramid/en/2.0-branch/changes.html

What's New In Pyramid 2.0:
https://docs.pylonsproject.org/projects/pyramid/en/2.0-branch/whatsnew-2.0.html

2.0 release documentation:
https://docs.pylonsproject.org/projects/pyramid/en/2.0-branch/

You can install it via PyPI:

  pip install Pyramid==2.0.2

Enjoy, and please report any issues you find to the issue tracker at
https://github.com/Pylons/pyramid/issues

Thanks!

- Pyramid core developers

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/78F8D59B-A014-44AC-AF7C-37C6812D98B0%40gmail.com.

[pylons-discuss] hupper 1.12 released

[Michael Merickel]

hupper 1.12 has been released.

Full change history:
https://docs.pylonsproject.org/projects/hupper/en/latest/changes.html

You can install it via PyPI:

 pip install hupper==1.12

Enjoy, and please report any issues you find to the issue tracker at
https://github.com/Pylons/hupper/issues

Thanks!

- Pylons Project core developers

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/74F1AE84-A97F-4E1B-9B46-63AD28D12EA1%40gmail.com.

[pylons-discuss] Pyramid 2.0.1 released

[Michael Merickel]

Pyramid 2.0.1 has been released.

The full changelog is here:
https://docs.pylonsproject.org/projects/pyramid/en/2.0-branch/changes.html

What's New In Pyramid 2.0:
https://docs.pylonsproject.org/projects/pyramid/en/2.0-branch/whatsnew-2.0.html

2.0 release documentation (across all alphas and betas, as well as when it gets
to final release):
https://docs.pylonsproject.org/projects/pyramid/en/2.0-branch/

You can install it via PyPI:

  pip install Pyramid==2.0.1

Enjoy, and please report any issues you find to the issue tracker at
https://github.com/Pylons/pyramid/issues

Thanks!

- Pyramid core developers

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/D9EC13FF-E9B3-4015-ADCC-1ECE0FB95573%40gmail.com.

[pylons-discuss] hupper 1.11 released

[Michael Merickel]

hupper 1.11 has been released.

Full change history:
https://docs.pylonsproject.org/projects/hupper/en/latest/changes.html

You can install it via PyPI:

 pip install hupper==1.11

Enjoy, and please report any issues you find to the issue tracker at
https://github.com/Pylons/pyramid_debugtoolbar/issues

Thanks!

- Pylons Project core developers

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/81DA74C1-6689-4C0F-9CFE-C3444D8C4E20%40gmail.com.

[pylons-discuss] pyramid_debugtoolbar 4.10 released

[Michael Merickel]

Pyramid Debugtoolbar 4.10 has been released.

Full change history:
https://docs.pylonsproject.org/projects/pyramid-debugtoolbar/en/latest/changes.html

You can install it via PyPI:

  pip install pyramid_debugtoolbar==4.10

Enjoy, and please report any issues you find to the issue tracker at
https://github.com/Pylons/pyramid_debugtoolbar/issues

Thanks!

- Pylons Project core developers

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/18FA0D78-4B6F-448A-9D59-B6B3AA32D94C%40gmail.com.

[pylons-discuss] colander 2.0 released

[Michael Merickel]

colander 2.0 has been released.

The full changelog is here:
https://docs.pylonsproject.org/projects/colander/en/latest/changes.html

Documentation:
https://docs.pylonsproject.org/projects/colander/en/latest/

You can install it via PyPI:

  pip install colander==2.0

Enjoy, and please report any issues you find to the issue tracker at
https://github.com/Pylons/colander/issues

Thanks!

- colander core developers

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/722AF3F1-931C-4F8C-9F82-B6463F0A0D48%40gmail.com.

Re: [pylons-discuss] SQLAlchemy 2.0 support

[Michael Merickel]

I'm unaware of any incompatibilities with SQLAlchemy 2.0 after reading the 
upgrade guide pretty extensively when I upgraded to 1.4. Nothing stood out in 
the changes to me that wouldn't work well with the existing pyramid_tm pattern.

> On Nov 24, 2022, at 12:38, zsol...@gmail.com  wrote:
> 
> Hi,
> 
> I'm starting a new project from cookiecutter and I wanted to use SQLAlchemy 
> 2.0 (pre-release currently). 
> 
> So far I changed meta.py to 
> 
> class Base(DeclarativeBase):
> metadata = my_metadata
> 
> as well as changing the model to the new Mapped / mapped_column syntax.
> 
> My question is that is there anything in pyramid_tm, transaction or 
> zope.sqlalchemy packages which needs to be changed, or this was all I needed?
> 
> SQLAlchemy changed a lot in 2.0, that's why I think the whole transaction 
> related part might be affected.
> 
> If it's the case then I'll just go back to 1.4 and wait a year or so till it 
> matures. What do you think?
> 
> Zsolt
> 
> 
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/7db23f68-cdef-42e4-8b71-3279c4db1c61n%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/AA15D550-A6B8-497E-A08D-A55C2459B4E2%40gmail.com.

[pylons-discuss] Re: plaster 1.1.1 and 1.1.2 released

[Michael Merickel]

Yeah ok, found another small thing and cut 1.1.2 as well. :-)

Check out https://pypi.org/project/plaster/1.1.2/ for more info!

- Michael

> On Nov 20, 2022, at 19:26, Michael Merickel  wrote:
> 
> Hey folks,
> 
> I've released 1.1.1 for plaster. This fixes a bug that seemed to affect folks 
> on Fedora more than most.
> 
> Check out https://pypi.org/project/plaster/1.1.1/ for more info!
> 
> - Michael

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/8ACA7051-E8B6-4E85-8E94-1DC4BD536633%40gmail.com.

[pylons-discuss] plaster 1.1.1 released

[Michael Merickel]

Hey folks,

I've released 1.1.1 for plaster. This fixes a bug that seemed to affect folks 
on Fedora more than most.

Check out https://pypi.org/project/plaster/1.1.1/ for more info!

- Michael

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/CD7FDC88-C2B0-41A5-A8C1-9B129E84EAFA%40gmail.com.

Re: [pylons-discuss] plaster 1.1 and plaster_pastedeploy 1.0/1.0.1 released

[Michael Merickel]

Thanks for the info. This has been reported here 
https://github.com/Pylons/plaster/issues/25 and I've been trying to repro it. 
Something in the new release is not deduplicating things and just need to track 
that down and I'll get a fix out. :-) I have not yet had this happen in my envs 
but I think I have some ideas.

> On Nov 19, 2022, at 05:58, Lele Gaifax  wrote:
> 
> Hi,
> 
> I tried to update one of my applications to these latest versions, and I
> got some problem figuring out how to launch my app with pserve.
> 
> I have the "usual" dev.ini/prod.ini twin configuration files, and I'm
> used to launch the development app using
> 
>  pserve dev.ini
> 
> With latest version that command errors out with the following traceback
> 
>  Traceback (most recent call last):
>File "/nix/store/.../bin/.pserve-wrapped", line 9, in 
>  sys.exit(main())
>File 
> "/nix/store/.../lib/python3.10/site-packages/pyramid/scripts/pserve.py", line 
> 30, in main
>  return command.run()
>File 
> "/nix/store/.../lib/python3.10/site-packages/pyramid/scripts/pserve.py", line 
> 189, in run
>  loader = self._get_config_loader(config_uri)
>File 
> "/nix/store/.../lib/python3.10/site-packages/pyramid/scripts/common.py", line 
> 23, in get_config_loader
>  return plaster.get_loader(config_uri, protocols=['wsgi'])
>File "/nix/store/.../lib/python3.10/site-packages/plaster/loaders.py", 
> line 115, in get_loader
>  raise MultipleLoadersFound(
>  plaster.exceptions.MultipleLoadersFound: Multiple plaster loaders were found 
> for scheme "file+ini", protocol "wsgi". Please specify a more specific 
> config_uri. Matched loaders: plaster-pastedeploy+file+ini, 
> plaster-pastedeploy+file+ini
> 
> After several attempts, I was able to launch the app using
> 
>  pserve plaster-pastedeploy+file+ini:dev.ini
> 
> Is that expected, or is there some problem with my configuration/usage?
> 
> Thanks in advance,
> bye, lele.
> -- 
> nickname: Lele Gaifax | Dire che Emacs è "conveniente" è come
> real: Emanuele Gaifas | etichettare l'ossigeno come "utile"
> l...@etour.tn.it  |   -- Rens Troost
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/87o7t3duls.fsf%40metapensiero.it.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/3A794C94-9F9E-4732-A4D9-4C810F97313C%40gmail.com.

[pylons-discuss] plaster 1.1 and plaster_pastedeploy 1.0/1.0.1 released

[Michael Merickel]

Hey folks,

I've released new versions of plaster and plaster_pastedeploy. The major 
changes to both libraries are:

- Drop support for Python 2.

- Drop dependencies on setuptools.

Check out https://pypi.org/project/plaster/1.1/ and 
https://pypi.org/project/plaster-pastedeploy/1.0.1/ for more info!

- Michael

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/809DBD7B-5FF9-47BE-95B4-50455F957392%40gmail.com.

[pylons-discuss] Re: PasteDeploy 3.0

[Michael Merickel]

I just released 3.0.1 now as well because 3.0 had the incorrect
python_requires metadata which could cause problems on unpinned
environments with older Python versions. I'm trying to pull the 3.0 release
from PyPI if I can get access.

On Sun, Oct 16, 2022 at 6:48 PM Michael Merickel  wrote:

> Hey folks,
>
> I just released PasteDeploy 3.0.
>
> There are only a couple small changes, but it felt safest to issue a major
> version bump just incase. The primary change is the removal of a dependency
> on setuptools by instead using importlib.metadata going forward.
>
> See the full changelog here:
>
> https://docs.pylonsproject.org/projects/pastedeploy/en/latest/news.html
>
> Thanks!
>
> - Michael
>


-- 

Michael

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwH1uNcpjpVwiiAt3JFf-P%2Bj5%2BLKODd0HKKNmLyJ1-AAVg%40mail.gmail.com.

[pylons-discuss] PasteDeploy 3.0

[Michael Merickel]

Hey folks,

I just released PasteDeploy 3.0.

There are only a couple small changes, but it felt safest to issue a major 
version bump just incase. The primary change is the removal of a dependency on 
setuptools by instead using importlib.metadata going forward.

See the full changelog here:

https://docs.pylonsproject.org/projects/pastedeploy/en/latest/news.html 


Thanks!

- Michael

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/461F8163-1576-4CB5-A6AC-647405BA6B7F%40gmail.com.

Re: [pylons-discuss] How to pass a complex validation object to a forbidden view?

[Michael Merickel]

Yeah sorry I looked at the ISecurityPolicy after your comments and subclassing 
HTTPForbidden probably isn’t the best option in general because you don’t 
control all of its call sites. For your security policy you likely want to 
return a subclass of Denied and detect that in your forbidden view.

If you have a need to pass that info explicitly from elsewhere in your codebase 
it may be convenient to define a HTTPForbidden subclass for easier raising. 
It’d just be a thin wrapper to set the right Denied subclass. 

- Michael

> On Aug 29, 2022, at 14:22, Sean Hammond  
> wrote:
> 
> 
>> 
>> You’re free to subclass HTTPForbidden or just extend it.
> 
> But if I subclass HTTPForbidden, how do I get an instance of my subclass to 
> be passed to the forbidden view (as request.exception) instead of the 
> standard HTTPForbidden class? I thought it was Pyramid rather than user code 
> that constructs that HTTPForbidden object.
> 
>> You can also 
>> subclass Denied as you noted. The big question to me is whether you can 
>> control the generation of that object to use the subclass and off the 
>> top of my head I think it’s always done in user code right now if 
>> you’re using the new SecurityPolicy interface but I could be wrong!
> 
> Yeah, this looks to me like it would work. I'll give it a try
> 
>> You 
>> just have to program the forbidden view a little defensively or make a 
>> separate one for your subclass so that you can still handle exceptions 
>> generated from code you may not control. 
> 
> Yep, good tip
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/4663be4d-4334-4c7d-b24e-1101866dd571%40www.fastmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/E74AD4C3-8009-411C-A77F-05E45DF09262%40gmail.com.

Re: [pylons-discuss] How to pass a complex validation object to a forbidden view?

[Michael Merickel]

You’re free to subclass HTTPForbidden or just extend it. You can also subclass 
Denied as you noted. The big question to me is whether you can control the 
generation of that object to use the subclass and off the top of my head I 
think it’s always done in user code right now if you’re using the new 
SecurityPolicy interface but I could be wrong! You just have to program the 
forbidden view a little defensively or make a separate one for your subclass so 
that you can still handle exceptions generated from code you may not control. 

- Michael

> On Aug 29, 2022, at 13:53, Sean Hammond  
> wrote:
> 
> If a view raises an exception, say a custom ValidationError class, then that 
> ValidationError object is set as request.exception when the exception view is 
> called (if you have registered a matching custom exception view).
> 
> On the other hand if a security policy denies access (returning a Denied 
> object) then the forbidden view (again: if you've registered one) gets called 
> with an HTTPForbidden object as request.exception.
> 
> Is there a nice way to attach a custom error object to the forbidden view? 
> For example by replacing the HTTPForbidden with a custom exception class, or 
> by attaching an exception or dict to the HTTPForbidden?
> 
> The Denied object that the security policy returned is available to the 
> forbidden view as request.exception.result. The Denied class's only attribute 
> is a msg string. But I suppose the security policy could just attach a custom 
> attribute to the Denied object?
> 
> Or would you implement a custom subclass of Denied? That's what 
> ACLSecurityPolicy seems to do.
> 
> Thanks!
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/bcf4aa9c-205d-4dc1-a684-20fa1f59a889%40www.fastmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/D4BF74AC-83A9-4866-8B95-0BA16BBFEA4D%40gmail.com.

Re: [pylons-discuss] Re: using pyramid without an ORM ?

[Michael Merickel]

I think it's worth noting that zope.sqlalchemy's session registration
supports a "readonly" initial state, similar to active and changed that
we've all harped on in the past. I'd probably just look into using that and
sticking with existing patterns. If you go all-in on readonly as a pattern
I think it could be a lot simpler but hey, this lets you use the existing
pattern.

On Wed, Aug 17, 2022 at 1:26 PM Jonathan Vanasco  wrote:

> If it's read-only, i would not have it join the transaction and just
> create the cleanup subscriber in the @reify .
>
> > # Because this was in the tutorial.
>
> I believe that is in there because of me. This pattern is used to provide
> access to the current request within SqlAlchemy objects and various
> @property or @reify decorated methods.  Without it, you must explicitly
> pass in a request (so no decorators) or use the `get_current_request`
> function which should generally be avoided.  There is a bit of discussion
> on it in the PR, and the cookiecutter template has many notes on it. See
> https://github.com/Pylons/pyramid-cookiecutter-starter/pull/111
>
> On Tuesday, August 16, 2022 at 5:50:02 PM UTC-4 Mike Orr wrote:
>
>> The SQLite database is pregenerated for a release and contains only
>> reference information. It's read only to the web application. So I'm
>> wondering if it's worth even hooking the session into the transaction
>> manager at all. I have a request subclass, and to open a session I use
>> a reified method:
>>
>> @reify
>> def sa_session(self):
>> engine = self.registry.sa_engine # Attribute set during startup
>> configuration.
>> info = {"request": self} # Because this was in the tutorial.
>> sess = sqlalchemy.orm.Session(engine, info=info)
>> zope.sqlalchemy.register(sess) # Is this worth doing for a
>> read-only database?
>> return sess
>>
>> The transaction manager closes the session for me, so without it I
>> guess I'd have to have a subscriber that rolls back and closes the
>> request. I don't want to have to do it in every view because it's not
>> view-specific logic.
>>
>> On Tue, Aug 16, 2022 at 9:19 AM 'Jonathan Vanasco' via pylons-discuss
>>  wrote:
>> >
>> > On Tuesday, August 16, 2022 at 11:45:24 AM UTC-4 Mike Orr wrote:
>> > > It is rolling back in some of my testing when there's no
>> > > insert/delete/update, but I want to make sure it always does, just in
>> > > case something somehow modifies the database when we didn't intend
>> to.
>> > > It's not that big a deal but it's what I'd like. I'm not sure if
>> > > SQLAlchemy is issuing rollback if there were no changes, or if it
>> will
>> > > always do so.
>> >
>> > That's from SQLAlchemy. It will rollback if there were no database
>> writes. SQLAlchemy is unaware of raw sql being a write operation, so you
>> need to use the `mark_changed` function from zope.sqlalchemy. This is a
>> weird idiosyncrasy of SQLAlchemy and transaction - the transaction could be
>> completely successful, but SQLAlchemy will rollback because there was no
>> activity within it's scope.
>> >
>> > It sounds like you're trying to do the opposite of what the
>> `transaction` package is designed to do.
>> >
>> > The way I normally deal with situations like that is to control if
>> SQLAlchemy joins the transaction or not. In most projects, I only use the
>> transaction on specific views that require this type of integration - such
>> as anything that sends an email (pyramid_mailer integrates with
>> pyramid_tm).
>> >
>> > It also sounds like your concern is mostly in testing. The approach
>> I've started to standardize on is to have a database snapshot for tests and
>> just recreate the database from that on every run. If you just want to
>> completely disable database commits though, you could have your test
>> harness set up a SQLAlchemy event listener for "commit", and then issue a
>> "rollback" within the event.
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> Groups "pylons-discuss" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> an email to pylons-discus...@googlegroups.com.
>> > To view this discussion on the web visit
>> https://groups.google.com/d/msgid/pylons-discuss/771e180a-ca5b-4625-baf7-972d237ea45an%40googlegroups.com.
>>
>>
>>
>>
>> --
>> Mike Orr 
>>
> --
> You received this message because you are subscribed to the Google Groups
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pylons-discuss/afd190a0-8b0e-412d-bd52-1ccdfe403994n%40googlegroups.com
> 
> .
>


-- 

Michael

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this

Re: [pylons-discuss] Re: using pyramid without an ORM ?

[Michael Merickel]

transaction.doom() is a good way. Another is to register a commit_veto hook in 
pyramid_tm. It is a hook that is invoked any time it would be about to commit, 
giving you a chance to stop it. Advantage of the veto is that you can register 
it from settings globally. 

- Michael

> On Aug 16, 2022, at 10:42, Mike Orr  wrote:
> 
> On Mon, Aug 15, 2022 at 3:46 PM Jonathan Vanasco  wrote:
>> 
>> 
>> I second what Michael said.  The sqlalchemy starter template is the right 
>> way to go.
>> 
>> The major thing this template does, is provide you with the glue between a 
>> SQLAlchemy "Session" and the pyramid request.  See : 
>> https://github.com/Pylons/pyramid-cookiecutter-starter/blob/latest/%7B%7Bcookiecutter.repo_name%7D%7D/%7B%7Bcookiecutter.repo_name%7D%7D/sqlalchemy_models/__init__.py#L87-L127
>> 
>> If you run pyramid_tm (https://pypi.org/project/pyramid-tm/) you can then 
>> use zope.sqlalchemy (https://pypi.org/project/zope.sqlalchemy/) to bind the 
>> session to the transaction.
>> 
>> You don't need to use SQLAlchemy's ORM.  You can just use SQLAlchemy Core 
>> (https://docs.sqlalchemy.org/en/14/core/) to do everything.  You can also 
>> access the underlying psycopg2 connections through SQLAlchemy when you want.
> 
> Is there a way to make 'pyramid_tm' always roll back even on success?
> I looked in the code and there seemed to be no flag for that, and I'd
> either have to call 'transaction.doom()' in a subscriber or patch the
> obscure code several levels deep.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/CAH9f%3DuqT58npmt6_sbfXJpVZ4hX7PNJdj1WqYXeXo7rJfhwPQQ%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/D770A14C-D365-42CC-9DCB-9D596E7B6B80%40gmail.com.

Re: [pylons-discuss] Re: using pyramid without an ORM ?

[Michael Merickel]

As Laurent said, if I were starting from scratch a new project I would 
personally use the pyramid-cookiecutter-starter sqlalchemy and then rip out 
what I didn’t need but only one part of that cookie cutter is the ORM. It has 
several other features that you need to solve one way or another so you’d at 
least want to know what you’re getting rid of. It is modular and you can pick 
the layers you want.

- connection pooling
- transaction binding to request lifecycle to rollback or commit at optimal 
times
- tables mappings to assist in converting database types into python
- ORM objects a more comprehensive query language on top of those tables
- alembic for autogenerating migrations from those models and helping with 
database versioning

If you go with only the first couple layers you can rip the rest out and just 
use dbsession.execute() like you would in psycopg2 and just set the 
zope.sqlalchemy initial_state=‘changed’. 

You can accomplish everything done here without sqlalchemy as well - but you 
will still want to understand the layers and reasons for the fundamental 
patterns to hook into pyramid. The sqlalchemy part of the config can be 
replaced almost directly with psycopg2 versions of everything and it just ends 
up being more specific to that application. You will need to use the cookbook 
entries to manage your connections/transactions as I don’t think there’s a 
pyramid_tm binding for psycopg2 directly - although it’s likely easy to write 
that DataManager. 

- Michael

> On Aug 11, 2022, at 09:25, Eldav  wrote:
> 
> 
> sorry, database *abstraction* (stupid typo)
> 
> 
>> Le jeudi 11 août 2022 à 18:22:07 UTC+2, Eldav a écrit :
>> Hello,
>> 
>> afaik, nothing forces you to use SQLAlchemy with Pyramid (after all, you 
>> could decide to use a non-relational database such as the ZODB or MongoDB). 
>> 
>> If you are sure that you won't need database attraction later, you can talk 
>> to psycopg2/3 directly. By doing so, you will also lose the possibility to 
>> automatically synchronise SQL transactions with Pyramid. It may or may not 
>> be ok for you, depending on your problem, but it's certainly possible to 
>> implement the required mechanism if required.
>> 
>> On the other hand, you might decide to run your raw SQL queries via 
>> SQLAlchemy, without using the ORM parts. This would give you the transaction 
>> synchronisation, provided you remember to mark your DB session as changed 
>> (look for "mark_changed" in 
>> https://pypi.org/project/zope.sqlalchemy/#full-example) after an 
>> insert/update/delete query.
>> 
>> Hope this helps,
>> 
>> Laurent.
>> 
>>> Le jeudi 11 août 2022 à 17:53:31 UTC+2, grc...@gmail.com a écrit :
>>> Hello , 
>>> 
>>> I am building an application ( more of a tool really ) that have a web 
>>> interface , and I am trying to avoid using ORM all together. plus I want to 
>>> try postgresql features and experiment more with them.
>>> 
>>> I don't want things to be overly complicated on the web interface so I put 
>>> lots of logic , something that I know will not change any time soon , into 
>>> PostgreSQL directly using functions etc.
>>> 
>>> can you please point me to the right direction of using pyramid without it 
>>> ? what might be the pitfalls of not using an ORM with pyramid  ?
>>> 
>>> 
>>> 
>>> Thanks !
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/d1ab4b3d-48be-48d4-aafb-1cc9cc2ef1b5n%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/F3EB17C1-D866-48C9-93C2-56AEA94AA330%40gmail.com.

Re: [pylons-discuss] Pyramid Response with File-like .write() interface

[Michael Merickel]

This harkens back to the discouraged write() callable in WSGI PEP 
returned by the start_response() invocation. The PEP as well as Pyramid as
a framework would encourage you to map the logic into an app_iter as Bert
suggested.

I think you'll want to define a file-like object that you can write to and
set as the app_iter. The question will be whether you try to do this by
writing from a separate thread or in some other way because once you return
control to the WSGI server to iterate on your app_iter then you are no
longer in control - you'll need some buffer between where you're generating
your parquet file and what you're returning from the app_iter. I don't
think a simple tempfile is good enough because you want the app_iter to
wait instead of stopping when it hits an EOF unless you know that you've
reached the end of the buffer.

On Tue, Aug 9, 2022 at 12:59 PM Bert JW Regeer  wrote:

> Some WSGI servers pass you the raw file descriptor as wsgi.input, but this
> is not guaranteed (wsgiref does for instance). Instead you should return an
> iterator that can be read incrementally so that your WSGI server can chunk
> responses.
>
>
>
> On Aug 9, 2022, at 10:38, Mikko Ohtamaa  wrote:
>
> Hi all,
>
> I'd like to stream dynamically generated Parquet-files from Pyramid
> server. Parquet library itself offers writing to any file-like object. I am
> aware of app_iter and FileResponse interfaces in Pyramid. However, does
> Pyramid (or any example or utility class) offer a Python file-like
> interface, where I could just dynamically .write() stuff?
>
> Br,
> Mikko
>
> --
> You received this message because you are subscribed to the Google Groups
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pylons-discuss/CAK8RCUudomfEGYnqAUk57XgOvatDLtFGGk%2Be5tFzu0w81Ez4Lg%40mail.gmail.com
> 
> .
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pylons-discuss/2DB1D478-20A5-4F02-96BE-8F0EA792AFBF%400x58.com
> 
> .
>


-- 

Michael

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwHF%2BJjGydJ28bHneP1_GXvN_P%3DyN3W2FnSAQ_Cea-xbjw%40mail.gmail.com.

Re: [pylons-discuss] pserve run in a child process under Fabric/Invoke + VirtualEnv ?

[Michael Merickel]

Well I might have answered too hastily. Seems the first process is controlled 
by fabric and not virtualenv. 

- Michael

> On Jun 8, 2022, at 19:23, Michael Merickel  wrote:
> 
> I doubt you’ll have the same problem with “python -m pyramid.scripts.pserve 
> dev_admin.ini” but would be good to find out. The process list appears to be 
> due to the virtualenv wrapper around a console script which this should skip. 
> 
> - Michael
> 
>>> On Jun 8, 2022, at 15:31, 'Jonathan Vanasco' via pylons-discuss 
>>>  wrote:
>>> 
>> I doubt anyone here may have experienced this, but I've run out of 
>> resources to explore on this...
>> 
>> We use Fabric (fabfile.org) to automate a lot of things. It is great.
>> 
>> I built a new routine in it this week, and I can't get it to clean up 
>> properly.  The routine simply spins up an admin version of a pyramid app, 
>> then hits some API endpoints to POST some filesystem data to it.
>> 
>> This is executed in a virtualenv. The problematic part of the routine is 
>> roughly...
>> 
>> @task
>> def import_data(c):
>> with c.cd("localpath/to/pyramid_app"):
>> proc_server = c.run("pserve dev_admin.ini", replace_env=False, 
>> asynchronous=True)
>> 
>> The issue is that I see two different processes on the operating system:
>> * cd localpath/to/pyramid_app && pserve dev_admin.ini
>> * /Library/Frameworks...Python /virtualenv/bin/pserve dev_admin.ini
>> 
>> asynchronous is used because running pyramid would otherwise block forever.  
>> i just analyze the process stderr for the "Serving" line, and continue once 
>> it is emitted. 
>> 
>> In fabric, I can access the FIRST process via `proc_server.runner` and I can 
>> stop/kill/terminate that -- but that leaves the second process running.  
>> That second process is one PID higher, and is the actual process that is 
>> running Pyramid and bound to the ports.  
>> 
>> I have a temporary workaround where I increase the PID by 1 and `c.run("kill 
>> %s" % pid2)` that process, but this is janky.
>> 
>> Has anyone encountered this before or have an idea on how to better handle 
>> this?
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "pylons-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to pylons-discuss+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/pylons-discuss/45ca3e17-e8a7-459a-ac69-492b3fadaeccn%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/882F2F42-DAB6-485D-81CA-FD367DD009C7%40gmail.com.

Re: [pylons-discuss] pserve run in a child process under Fabric/Invoke + VirtualEnv ?

[Michael Merickel]

I doubt you’ll have the same problem with “python -m pyramid.scripts.pserve 
dev_admin.ini” but would be good to find out. The process list appears to be 
due to the virtualenv wrapper around a console script which this should skip. 

- Michael

> On Jun 8, 2022, at 15:31, 'Jonathan Vanasco' via pylons-discuss 
>  wrote:
> 
> I doubt anyone here may have experienced this, but I've run out of resources 
> to explore on this...
> 
> We use Fabric (fabfile.org) to automate a lot of things. It is great.
> 
> I built a new routine in it this week, and I can't get it to clean up 
> properly.  The routine simply spins up an admin version of a pyramid app, 
> then hits some API endpoints to POST some filesystem data to it.
> 
> This is executed in a virtualenv. The problematic part of the routine is 
> roughly...
> 
> @task
> def import_data(c):
> with c.cd("localpath/to/pyramid_app"):
> proc_server = c.run("pserve dev_admin.ini", replace_env=False, 
> asynchronous=True)
> 
> The issue is that I see two different processes on the operating system:
> * cd localpath/to/pyramid_app && pserve dev_admin.ini
> * /Library/Frameworks...Python /virtualenv/bin/pserve dev_admin.ini
> 
> asynchronous is used because running pyramid would otherwise block forever.  
> i just analyze the process stderr for the "Serving" line, and continue once 
> it is emitted. 
> 
> In fabric, I can access the FIRST process via `proc_server.runner` and I can 
> stop/kill/terminate that -- but that leaves the second process running.  That 
> second process is one PID higher, and is the actual process that is running 
> Pyramid and bound to the ports.  
> 
> I have a temporary workaround where I increase the PID by 1 and `c.run("kill 
> %s" % pid2)` that process, but this is janky.
> 
> Has anyone encountered this before or have an idea on how to better handle 
> this?
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/45ca3e17-e8a7-459a-ac69-492b3fadaeccn%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/6144B475-0245-4886-9141-2C2414CAFDD9%40gmail.com.

Re: [pylons-discuss] Waitress and multiprocess pooling

[Michael Merickel]

Mikko,

What is your gunicorn config?

> 
> On May 23, 2022, at 04:52, Mikko Ohtamaa  wrote:
> 
> 
> Hi all,
> 
> Thank you for your kind support.
> 
> We have been running a Gunicorn for half a week now and you can clearly where 
> we switched over, as the request latency is much more stable none. We also 
> managed to solve all configure problems with Gunicorn thanks to your kind 
> help.
> 
> Backend latency:
> 
> 
> 
> Even PostgreSQL latency is more controlled now, don't though not sure if I 
> understand why - probably something to do with PSQL, Waitress, psycopg2 and 
> SQLAlchemy connection pooling.
> 
> 
> 
> 
> 
> Br,
> Mikko
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/CAK8RCUvVk43R8HCnDvWWipi3%3D54hKbF%3DjnDosMP%3D%3DTNS%3D2JN-g%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/BFE62D2E-90DF-432E-80CC-A2235375D89C%40gmail.com.

Re: [pylons-discuss] Waitress and multiprocess pooling

[Michael Merickel]

Mikko,

The best agnostic impl I know for url prefix is the PasteDeploy prefix 
middleware.

https://docs.pylonsproject.org/projects/pastedeploy/en/latest/modules/config.html#paste.deploy.config.PrefixMiddleware

You can mount it in your ini pipeline pretty easily:

[pipeline:main]
pipeline =
prefix
myapp

[filter:prefix]
use = egg:PasteDeploy#prefix
prefix = /foo


> On May 18, 2022, at 02:03, Mikko Ohtamaa  wrote:
> 
> 
> 
> On Fri, 13 May 2022 at 00:14, Jonathan Vanasco  > wrote:
> - What's the recommended modern multiprocess enabled web server to do more 
> scaleable Pyramid hosting?
> 
> I like uWSGI, but others like gunicorn and I think there is another popular 
> option. 
> 
> I decided to go with Gunicorn, because I used uWSGI extensively. Gunicorn is 
> somewhat easier to manage with modern Python package management tools like 
> Poetry, plus you do not encounter the potential issues raising from compiling 
> C.
> 
> So far so good, only option I missing from Gunicorn is Waitress's 
> "url_prefix". Is there a generic way for setting the WSGI application root /  
> URL prefix that is not specific to Waitress?
> 
> Opened a StackOverflow question on this one: 
> https://stackoverflow.com/questions/72284506/setting-application-url-prefix-with-gunicorn
>  
> 
> 
> Br,
> Mikko
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/CAK8RCUtMQwaPvG3QdrjGgpt9sB_pHb3ZHzODF_V1TN56T%2BGmOw%40mail.gmail.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/5A6F5EB4-A6DC-4DF3-8A6F-5B6F1CCA035C%40gmail.com.

Re: [pylons-discuss] Downsides of committing by default?

[Michael Merickel]

My understanding is that it’s a micro optimization that a rollback is slightly 
more efficient than a commit. I’m not aware of any good arguments against 
defaulting to changed. Would need to ask the zope.sqlalchemy folks for some 
insights. 

- Michael

> On May 2, 2022, at 17:55, Theron Luhn  wrote:
> 
> I’m using a cookiecutter-based Pyramid+zope.sqlalchemy+SQLAlchemy stack.  
> More and more lately I’ve been skipping the ORM and using Core for write 
> operations, and I frequently run into issues where I forget to mark_changed 
> and zope.sqlalchemy ROLLBACKs by default:
> 
>> By default, zope.sqlalchemy puts sessions in an ‘active’ state when they are 
>> first used. ORM write operations automatically move the session into a 
>> ‘changed’ state. This avoids unnecessary database commits. Sometimes it is 
>> necessary to interact with the database directly through SQL. It is not 
>> possible to guess whether such an operation is a read or a write. Therefore 
>> we must manually mark the session as changed when manual SQL statements 
>> write to the DB.
> 
> 
> The docs go on to describe how I can change the behavior to COMMIT by default:
> 
>> If this is a problem you may register the events and tell them to place the 
>> session in the ‘changed’ state initially.
> 
> 
> My question:  Is there any downside to COMMITing by default?  I assume 
> there’s a good reason why the default is to ROLLBACK.  I’m using PostgreSQL, 
> but I’d be interested in hearing about how this affects RDBMSs too.
> 
> — Theron
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/598CAFEC-A22F-47E1-8AFE-7B06F29EC5CF%40luhn.com.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/A6882305-07EA-4A31-9347-142895CF8296%40gmail.com.

Re: [pylons-discuss] pyramid-jinja2 2.9.1 has been released

[Michael Merickel]

Hey folks, egg on my face. I hadn't realized that some of the changes
required upgrading to Jinja 3. I have "yanked" the release for now which
means you will receive a warning if you install this version from PyPI. We
need to re-evaluate the minimum dependencies and determine how to release
pyramid_jinja2 in a way that is a bit more smooth for everyone. In the
short term 2.8 is still the latest release of pyramid_jinja2.

Thanks!

- Michael

On Sun, Mar 13, 2022 at 3:06 AM Steve Piercy 
wrote:

> pyramid-jinja2 2.9.1 has been released.
>
> The full changelog is here:
>
> https://docs.pylonsproject.org/projects/pyramid-jinja2/en/latest/changes.html
>
> What's New In pyramid-jinja2 2.9.1:
>
> https://docs.pylonsproject.org/projects/pyramid-jinja2/en/latest/changes.html#id2
>
> Documentation:
> https://docs.pylonsproject.org/projects/pyramid-jinja2/en/latest/
>
> You can install it via PyPI:
>
>pip install pyramid-tm==2.9.1
>
> Enjoy, and please report any issues you find to the issue tracker at
> https://github.com/Pylons/pyramid_jinja2/issues
>
> Thanks!
>
> - pyramid-jinja2 core developers
>
> --
> You received this message because you are subscribed to the Google Groups
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pylons-discuss/d700ff87-d0d0-fdec-0e5c-84222ebb46ee%40gmail.com
> .
>


-- 

Michael

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwEPadFW27iNPC8RQV-qgqrqNb%2BOouGHDWaMG2_WKoaBPg%40mail.gmail.com.

Re: [pylons-discuss] Programmatically creating and closing Waitress server: OSError: [Errno 9] Bad file descriptor

[Michael Merickel]

Mikko,

Have you looked at the StopableWSGIServer in webtest? I wonder if it can help 
debug what you've been seeing.

https://docs.pylonsproject.org/projects/webtest/en/latest/api.html#webtest.http.StopableWSGIServer
 


- Michael

> On Jan 22, 2022, at 05:11, Mikko Ohtamaa  wrote:
> 
> Hi,
> 
> I am trying to add webhook support to the otherwise command-line application 
> using Waitress web server. Webhook endpoints are defined as Pyramid routes. I 
> also wish to test this functionality using pytest.
> 
> The server gets created fine, but on pytest shutdown, I am getting an error: 
> OSError: [Errno 9] Bad file descriptor. What would be the nice way to spin 
> Waitress up and down on demand?
> 
> How do I create the server instance:
> 
> def create_webhook_server(host: str, port: int, username: str, password: str, 
> queue: Queue) -> MultiSocketServer:
> app = create_pyramid_app(username, password, queue, production=False)
> server = create_server(app, host=host, port=port)
> logger.info ("Webhook server will spawn at %s:%d", 
> host, port)
> return server
> 
> How do I start and test the server in the unit test:
> 
> def test_auth_ok():
> """Username and password allow to access the webhook"""
> queue = Queue()
> server = create_webhook_server("127.0.0.1", 5000, "test", "test", queue)
> server_url = "http://test:test@127.0.0.1:5000 
> "
> webhook_thread = Thread(target=server.run)
> webhook_thread.start()
> # Test home view
> resp = requests.get(server_url)
> assert resp.status_code == 200
> server.close()
> 
> The exception I am getting:
> 
> tests/test_webhook_auth.py::test_auth_ok
>   
> /Users/moo/Library/Caches/pypoetry/virtualenvs/tradeexecutor-13XfP0tc-py3.9/lib/python3.9/site-packages/_pytest/threadexception.py:75:
>  PytestUnhandledThreadExceptionWarning: Exception in thread Thread-1
>   
>   Traceback (most recent call last):
> File 
> "/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py",
>  line 973, in _bootstrap_inner
>   self.run()
> File 
> "/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py",
>  line 910, in run
>   self._target(*self._args, **self._kwargs)
> File 
> "/Users/moo/Library/Caches/pypoetry/virtualenvs/tradeexecutor-13XfP0tc-py3.9/lib/python3.9/site-packages/waitress/server.py",
>  line 322, in run
>   self.asyncore.loop(
> File 
> "/Users/moo/Library/Caches/pypoetry/virtualenvs/tradeexecutor-13XfP0tc-py3.9/lib/python3.9/site-packages/waitress/wasyncore.py",
>  line 245, in loop
>   poll_fun(timeout, map)
> File 
> "/Users/moo/Library/Caches/pypoetry/virtualenvs/tradeexecutor-13XfP0tc-py3.9/lib/python3.9/site-packages/waitress/wasyncore.py",
>  line 172, in poll
>   r, w, e = select.select(r, w, e, timeout)
>   OSError: [Errno 9] Bad file descriptor
>   
> warnings.warn(pytest.PytestUnhandledThreadExceptionWarning(msg))
> 
> tests/test_webhook_auth.py::test_auth_failed
>   
> /Users/moo/Library/Caches/pypoetry/virtualenvs/tradeexecutor-13XfP0tc-py3.9/lib/python3.9/site-packages/_pytest/threadexception.py:75:
>  PytestUnhandledThreadExceptionWarning: Exception in thread Thread-2
>   
>   Traceback (most recent call last):
> File 
> "/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py",
>  line 973, in _bootstrap_inner
>   self.run()
> File 
> "/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py",
>  line 910, in run
>   self._target(*self._args, **self._kwargs)
> File 
> "/Users/moo/Library/Caches/pypoetry/virtualenvs/tradeexecutor-13XfP0tc-py3.9/lib/python3.9/site-packages/waitress/server.py",
>  line 322, in run
>   self.asyncore.loop(
> File 
> "/Users/moo/Library/Caches/pypoetry/virtualenvs/tradeexecutor-13XfP0tc-py3.9/lib/python3.9/site-packages/waitress/wasyncore.py",
>  line 245, in loop
>   poll_fun(timeout, map)
> File 
> "/Users/moo/Library/Caches/pypoetry/virtualenvs/tradeexecutor-13XfP0tc-py3.9/lib/python3.9/site-packages/waitress/wasyncore.py",
>  line 172, in poll
>   r, w, e = select.select(r, w, e, timeout)
>   OSError: [Errno 9] Bad file descriptor
>   
> warnings.warn(pytest.PytestUnhandledThreadExceptionWarning(msg))
> 
> 
> 
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/CAK8RCUtiHH3N00sUthcUKj6Ur-ZjNsAGkXByo5FE

Re: [pylons-discuss] Plaster status

[Michael Merickel]

plaster and plaster_pastedeploy are around and maintained. :-)

> On Jan 7, 2022, at 09:26, Stéphane Brunner  
> wrote:
> 
> Hello,
> 
> Pyramid depends on plaster: 
> https://github.com/Pylons/pyramid/blob/master/setup.py#L29-L30
> But the alt commits of plaster and plaster_pastdeploy
> https://github.com/Pylons/plaster
> https://github.com/Pylons/plaster_pastedeploy
> has no commits since begin of 2019
> Then what's the status of plaster?
> 
> Sincerely
> Stéphane Brunner
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/17417a79-1c53-4dc2-acb6-a049507888c9n%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/A8FC2365-84A1-4C57-A65B-9BABD969D583%40gmail.com.

Re: [pylons-discuss] Best practices for running Pyramid (docker) and nginx (host) deployment

[Michael Merickel]

The linked waitress docs do work. I deploy waitress behind nginx and envoy and 
heroku regularly using the guides in there. Make sure to configure the trusted 
proxy settings and ensure the upstream is setting the right headers. 

- Michael

> On Dec 13, 2021, at 18:22, Jens Troeger  wrote:
> 
> Hello,
> 
> I’ve seen some conversation here about running a Pyramid app server inside a 
> Docker container, but none has really answered my questions.
> 
> My setup is that nginx runs on the host and currently uses proxy_pass to 
> forward requests to the container’s external port, so that the requests are 
> then processed and responded to by the Pyramid application running inside the 
> container.
> 
> Question: Inside the container I’m running the Pyramid application using 
> pserve which listens on the container’s mapped internal port. Should I switch 
> to gunicorn instead? Does it matter in such a setup?
> 
> The proxy_pass URL is http://127.0.0.1:6543 which means that the external 
> https gets lost. That, in turn, means that within the Pyramid app (inside of 
> the container) calls to e.g. static_url() return a http route instead of the 
> necessary & expected https.
> 
> Question: I currently use prefix WSGI middleware to rewrite responses 
> (discussion) but that feels hacky. Unfortunately, I wasn’t able to make 
> X-Forward-Proto HTTP header work quite yet so what’s the current 
> recommendation here? Is the Using Behind a Reverse Proxy page current and 
> working?
> 
> Question: Are there any benefits to using a UNIX socket for proxy_pass, 
> instead of HTTP?
> 
> Much thanks in advance!
> Jens
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/508f077e-ff7e-47c4-9e8f-ee5f018e9a7en%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/5B6A51E7-2449-47BB-805F-E0ED8B58050D%40gmail.com.

Re: [pylons-discuss] Strange problem with request method

[Michael Merickel]

I think there's a lot of unknowns in helping unpack this - for starters there's 
no code supplied that actually modifies the cache, so of course it is empty!

- There's a few gotchas in python - for example if this is done in the main 
file in your app (foo.py) and you run "python -m foo" then the module may be 
imported twice as "__main__" and as "foo" which can cause surprises.

- I see you're using waitress so this probably isn't a problem, but you're 
using an in-memory cache and so nothing will be shared across processes if you 
ever fork.

- Michael

> On Oct 11, 2021, at 03:34, Gerhard Schmidt  wrote:
> 
> Hi,
> 
> today I encountered a very strange problem with request methods.
> 
> <--- code --->
> 
> cache = {'agents': set(),
> 'time': 0.0}
> cache_lock = Lock()
> 
> 
> def in_agent_cache(request):
>with cache_lock:
>   return request.user_agent in cache['agents']
> 
> <--- end code --->
> 
> If I call the method directly, everything works fine.
> 
> if I register the method as a request method and call it via 
> request.in_agent_cache() the dict is always the default version, the set 
> empty and time .
> 
> If I move the dict and lock to another file and import it. Everything works 
> even as a request method.
> 
> The log messages are also recorded under a wrong module
> 
> 2021-10-11 10:29:33,153 INFO  [agent_cache:96][waitress-0] in_agent_cache 
> called
> 
> agent_cache is the name of the py file and not the package it's in.
> 
> Any hint if I'm doing something wrong.
> 
> Regards
>Estartu
> 
> 
> 
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/90ae76af-8406-3ec5-b490-2f4af254431d%40augusta.de.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/6C2270B2-EA05-4FC6-98F5-A27863A22682%40gmail.com.

Re: [pylons-discuss] Could not find a matching loader for the scheme "file+ini ", protocol "wsgi"?

[Michael Merickel]

Do you have plaster_pastedeploy installed?

- Michael

> On Sep 9, 2021, at 02:28, Simon  wrote:
> 
> 
> Hi there,
> 
> I got an error about 'could not find a matching loader for the scheme' when I 
> want to run my Pyramid web app in PyCharm based on Pyramid server. However, 
> if I directly run pserve myapp.ini in terminal, it can launch my web app 
> locally. I have edit the configuration in PyCharm: the config file is pointed 
> to myapp.ini and the Python interpreter is pointed to python in the virtual 
> environment.
> 
> Traceback (most recent call last):
>   File 
> "/Applications/PyCharm.app/Contents/plugins/python/helpers/pycharm/pycharm_load_entry_point.py",
>  line 12, in 
> sys.exit(f())
>   File 
> "/Users/simon/Documents/venv/lib/python3.6/site-packages/pyramid/scripts/pserve.py",
>  line 34, in main
> return command.run()
>   File 
> "/Users/simon/Documents/venv/lib/python3.6/site-packages/pyramid/scripts/pserve.py",
>  line 193, in run
> loader = self._get_config_loader(config_uri)
>   File 
> "/Users/simon/Documents/venv/lib/python3.6/site-packages/pyramid/scripts/common.py",
>  line 23, in get_config_loader
> return plaster.get_loader(config_uri, protocols=['wsgi'])
>   File 
> "/Users/simon/Documents/venv/lib/python3.6/site-packages/plaster/loaders.py", 
> line 109, in get_loader
> raise LoaderNotFound(requested_scheme, protocols=protocols)
> plaster.exceptions.LoaderNotFound: Could not find a matching loader for the 
> scheme "file+ini ", protocol "wsgi".
> 
> Process finished with exit code 1
> 
> 
> Though I go to the official website about plaster and WSGI, I still cannot 
> figure it out. \
> 
> Do anyone have any suggestion or have any similar problem when configuring 
> Pyramid web app in PyCharm?
> 
> Thanks.
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/d2974f67-3b15-4dbc-9ed5-f9bac8be7046n%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/BEE12912-C20F-4828-96E5-93F04DFEFFD8%40gmail.com.

Re: [pylons-discuss] session.close() no longer needed or even never needed?

[Michael Merickel]

Unfortunately there's no "right" answer to the situation, it all depends on how 
you choose to manage database connections in your app.

It's hard to answer whether what you're doing now is correct or not without 
knowing more - you can have pyramid_tm enabled in your app and not actually use 
it. It'll just be a no-op if things don't hook into request.tm and use it (via 
zope.sqlalchemy or other using request.tm directly).

The cookiecutter provides what I'd consider to be the best option (but I'm 
biased) which sets up pyramid_tm to manage the begin/commit on the transaction. 
However there are perfectly legitimate ways to handle things (such as in your 
first example) that do not use pyramid_tm with zope.sqlalchemy.

- Michael

> On Sep 1, 2021, at 05:53, Petr Blahoš  wrote:
> 
> Hi,
> 
> I am upgrading some old code of mine, and I came across the
> request.db (I think it's called request.session in Pyramid scaffolding)
> method which looks like:
> 
> def db(request):
> session = request.registry.sessionmaker()
> 
> def cleanup(request):
> session.close()
> request.add_finished_callback(cleanup)
> return session
> 
> and calling config.add_request_method(db, reify=True) somewhere, while the 
> sessionmaker was called with ZopeTransactionExtension.
> 
> The "current" version is something like:
> def db(request):
> session = request.registry.sessionmaker()
> zope.sqlalchemy.register(session, transaction_manager=request.tm 
> )
> return session
> 
> Now, I wonder. I expect that pyramid_tm takes care of the cleanup / 
> session.close()
> for the request. But I have always used the transaction manager pyramid_tm. 
> Am I 
> guessing correctly, that adding finished callback(cleanup) was unnecessary?
> 
> Thanks...
> Petr
> -- 
> https://blahos.com/ 
> https://traceability.cz/ 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/CA%2ByMeXWHgTrt%3DTHkK-fgAcdjxxpqe3nQkuy1RBg4ziM61Axazg%40mail.gmail.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/7F3DB6DE-3AFD-4AF3-A769-72358ABAC5BA%40gmail.com.

Re: [pylons-discuss] Serving large authenticated files

[Michael Merickel]

I have some scenarios where I need to do some processing (envelope decryption) 
on a file from s3 prior to download and then let the user download it and this 
is how I do it as well.

1. Download giant file from S3 to temporary file.
2. Process file to another temporary file.
3. Return a FileResponse wrapping the temporary file.
4. via WSGI iterator protocol the server will invoke the close() method on the 
iterator when the request is cleaned up and this will bubble up to delete the 
temporary file being wrapped.

My solution is not ideal, there is a lag while the file is downloaded from 
storage into a temporary file and processed, before I return the iterator. 
However in practice it doesn't blow out memory (yay) and since the throughput 
between S3 and EC2 is great it does >1GB files with only a slight lag (a couple 
seconds iirc).

I would say that if you don't need to do any processing then Theron's S3 
suggestion is definitely better assuming you can expose those endpoint details 
to clients.

- Michael

> On Jul 14, 2021, at 10:57, Theron Luhn  wrote:
> 
> Pyramid has FileResponse 
> https://docs.pylonsproject.org/projects/pyramid/en/latest/api/response.html#pyramid.response.FileResponse
>  
> ,
>  which does use wsgi.file_wrapper you linked to (if available).
> 
> Generally in this situation I offload the file to S3 or similar, and have 
> Pyramid generate a signed URL to redirect to.  Operationally much simpler.
> 
> — Theron
> 
> 
> 
>> On Jul 14, 2021, at 3:17 AM, Mikko Ohtamaa > > wrote:
>> 
>> Hi,
>> 
>> I need to serve large authenticated files (several gigabytes). I will do an 
>> API key check before the user can download a file.
>> 
>> What is the most efficient way to serve these out from Pyramid? Assuming I 
>> do not want to block processes or threads - is it possible? 
>> 
>> - Zope 2 used to have a sendfile ( 
>> https://www.python.org/dev/peps/pep-0333/#id36 
>> ) - is there anything 
>> equivalent for waitress
>> 
>> - Any ideas about caching the file in the frontend servers (Caddy, 
>> Cloudflare) and then just creating a short-lived HTTP redirect (<1h) to the 
>> actual target
>> 
>> - Other ideas
>> 
>> Cheers,
>> Mikko
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "pylons-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to pylons-discuss+unsubscr...@googlegroups.com 
>> .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/pylons-discuss/CAK8RCUuTG7xZ9tFf5raSmotjOWPJx31dcXyDujk4222_GDq74w%40mail.gmail.com
>>  
>> .
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/8B634DFB-996C-47AB-9623-EC0DFF5FD50D%40luhn.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/1A776BE4-DB33-4E9E-A652-9A2B7F76E04B%40gmail.com.

Re: [pylons-discuss] modifying database in webtest based testing

[Michael Merickel]

If you have the app, the registry is attached as "app.registry" and you can use 
the underlying prepare() method with it, same as bootstrap does. See 
https://docs.pylonsproject.org/projects/pyramid/en/latest/api/scripting.html#pyramid.scripting.prepare
 
.

The rule of thumb is: if you have a registry, use prepare. If you have a config 
spec (ini file), use bootstrap. Prepare is way more lightweight because it 
reuses the configured registry whereas bootstrap creates an entirely new 
registry each time.

- Michael

> On Jun 24, 2021, at 12:33, Zsolt Ero  wrote:
> 
> Hi,
> 
> I have my webtest based testing set up like the following.
> 
> @pytest.fixture(scope="session")
> def app():
> testing_config = {
> 'sqlalchemy.url': 'postgresql+psycopg2://x@/x',
> 'redis.sessions.secret': 'x',  # random string
> 'redis.sessions.db': 1,
> 'tm.annotate_user': 'no',
> }
> 
> return main({"testing": True}, **testing_config)
> 
> 
> @pytest.fixture
> def testapp(app):
> return TestApp(app)
> 
> main() is a config.make_wsgi_app().
> 
> My questions is, how can I have direct database connection - normally a 
> request.dbsession in the app - in the testing fixtures? I'd like to run some 
> DB queries in the setup/teardown process.
> 
> I can access testapp.app or "app" from the fixtures, that works. What I don't 
> know is how can I get a request object from there. 
> 
> In command line scripts I use 
> 
> request_dummy = Request.blank('/', base_url=base_url)
> env = bootstrap(config_uri, request=request_dummy)
> request = env['request']
> 
> Do I need to do something similar here?
> 
> Zsolt
> 
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/CAKw-smAqGQjoX1h%2B_rWHjh6Pvi95Vb5RVQ5E%3Djf91B8AW6b1Yg%40mail.gmail.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/93FFEDD1-B06D-4359-9D98-C270826ABE6C%40gmail.com.

Re: [pylons-discuss] minimal auth/security policy implementation

[Michael Merickel]

CSRF has nothing to do with authentication other than that you should rotate it 
at login/logout privilege boundaries at the very least.

You can use the CSRF system without configuring a security/auth policy at all.

- Michael

> On May 24, 2021, at 14:40, Zsolt Ero  wrote:
> 
> Hi Theron,
> 
> Thanks for your reply. It looks indeed simpler. How much more minimal can I 
> make it? I definitely want to "circumvent" the whole security system, I'm 
> perfectly happy with using my new require_admin=True like options.
> 
> I just want CSRF to work and it seems to be dependent on RootFactory being 
> defined, which I don't understand. 
> 
> Zsolt
> 
> 
> 
> 
> 
> 
> On 24. May 2021 at 20:31:45, Theron Luhn  > wrote:
> You may have better luck with the Pyramid 2.0 security system.  It’s much 
> simpler for cases like yours where you don’t need ACL.  For example, your 
> implementation might look like:
> 
> class CustomSecurityPolicy:
>   def identity(self, request):
> return request.user
> 
>   def authenticated_userid(self, request):
> return request.user.id  if request.user else None
> 
>   def permits(self, request, context, permission):
> if permission == ‘user’ and request.user:
>   return Allowed(‘User is signed in.’)
> elif permission == ‘admin’ and request.user and request.user.id 
>  == 1:
>   return Allowed(‘Admin user is signed in.’)
> else:
>   return Denied(‘Access is not allowed.’)
> 
>   def remember(request, userid, **kw):
> …  # Same as before
> 
>   def forget(request, **kw):
> …
> 
> That’s all.  No ACL or root factory, just identity()/authenticated_userid() 
> returning the current user and permits() giving a thumbs up or down if access 
> should be allowed.  Docs:  
> https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/security.html 
> 
>  
> 
> View derivers would certainly work.  After all, the security system itself is 
> implemented with a view deriver.  But personally I would avoid circumventing 
> the entire security system like that.
> 
> — Theron
> 
> 
> 
>> On May 22, 2021, at 2:16 PM, zsol...@gmail.com  
>> mailto:zsolt@gmail.com>> wrote:
>> 
>> Hi,
>> 
>> I've been using the following auth policies for years, it's been working 
>> fine:
>> 
>> authn_policy = CustomSessionAuthenticationPolicy()
>> authz_policy = ACLAuthorizationPolicy()
>> 
>> config = Configurator(
>> settings=settings,
>> root_factory=RootFactory,
>> authentication_policy=authn_policy,
>> authorization_policy=authz_policy,
>> )
>> 
>> 
>> class RootFactory(object):
>> __acl__ = [
>> (Allow, Authenticated, 'user'),
>> (Allow, 'g:admin', 'admin'),
>> (Allow, 'g:superadmin', ALL_PERMISSIONS),
>> ]
>> 
>> def __init__(self, request):
>> pass
>> 
>> 
>> 
>> class CustomSessionAuthenticationPolicy(SessionAuthenticationPolicy):
>> def authenticated_userid(self, request):
>> return request.user.id 
>> 
>> def effective_principals(self, request):
>> principals = [Everyone]
>> if request.user:
>> principals += [Authenticated]
>> 
>> if request.user.id  == 1:
>> principals += ['g:superadmin', 'g:admin']
>> 
>> return principals
>> 
>> ---
>> 
>> I'm trying to migrate off from this, as I simply don't understand what is 
>> happening behind and I prefer a much simpler view deriver based approach.
>> 
>> Basically, with a couple of view derivers I could solve all my problems in a 
>> few hours, and it also allows me much more flexibility. For example for some 
>> views now I can do auth based on API tokens, while most of the views are 
>> using session based auth.
>> 
>> My questions is, how can I make the auth/security policies as simple as 
>> possible? All I need is working CSRF,  remember and forget.
>> 
>> I'm on 1.10 but I'm happy to migrate to 2.0 if that allows a simplified 
>> approach.
>> 
>> So far I was able to get it down to this:
>> 
>> config = Configurator(
>> settings=settings,
>> root_factory=RootFactory,
>> authentication_policy=SessionAuthenticationPolicy(),
>> )
>> 
>> class RootFactory(object):
>> __acl__ = [
>> (Allow, Authenticated, 'user'),
>> ]
>> 
>> def __init__(self, request):
>> pass
>> 
>> Session is via pyramid_session_redis.
>> 
>> Thanks,
>> Zsolt
>> 
>> 
>> 
>> 
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "pylons-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to pylons-discuss+unsubscr...@googlegroups.com 
>> .
>> To view this discussion

Re: [pylons-discuss] Is there a way to hook into the queue depth warning?

[Michael Merickel]

It's using the python stdlib logging library. You can add a handler to that 
logger that sends emails or does anything else. It really depends on how you're 
running your processes how to setup logging properly but in Pyramid (which uses 
logging.config.fileConfig) it would be done by adding another section to your 
ini file to catch the "waitress.queue" logger.

[loggers]
keys = ..., waitress_queue

[handlers]
keys = ..., waitress_queue

[logger_waitress_queue]
handlers = waitress_queue
qualname = waitress.queue

[handler_waitress_queue]
class = handlers.SMTPHandler
args = (('example.com', 578, 'f...@example.com', ['t...@example.com'], 'queue 
depth warning', ('username', 'password'))
formatter = generic

- Michael

> On May 21, 2021, at 15:51, jdavi...@gmail.com  wrote:
> 
> I understand the cause of the warning, I just wonder if there's a way to hook 
> into the process that triggers it to allow it to send a warning email or 
> something when the queue depth reaches a certain threshold? 
> 
> Alternatively is there a way to query it to see what the queue depth is 
> currently?
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/d65251f9-61b4-4551-9c3b-761c193074f8n%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/0297A2C3-93D4-4389-B09E-0AE1C2D4DF5C%40gmail.com.

Re: [pylons-discuss] Use route_url just after the routes are added and without request

[Michael Merickel]

At that config-time in the application there is no active request and no server 
running. In Pyramid, all url-generation APIs rely on creating a url "relative 
to the wsgi environ" or "relative to the current request". This keeps the app 
itself easy to mount into complex existing url hierarchies. This is done using 
properties of the request (script_name, port, host, scheme).

The way to construct a full URL is to create a dummy request object:

from pyramid.request import Request

request = Request.blank(base_url='http://localhost:5900')
request.registry = config.registry
assert request.route_url('home') == 'http://localhost:5900/'

The Request.blank() api constructs a wsgi environ with enough information for 
Pyramid to generate a url relative to it similar to what it would do if a 
request was coming from an actual wsgi server.

- Michael

> On May 12, 2021, at 15:01, QLands Software  wrote:
> 
> I have a Pyramid application that adds routes using:
> 
> config.add_route("home", "/")
> config.add_view(
> homeView,
> route_name="home",
> "home.jinja2",
> )
> 
> The application uses PCA https://github.com/PyUtilib/pyutilib therefore 
> plug-ins can add routes to the main application.
> 
> After the main app and the plugins add the routes I would like to get the URL 
> of a route for example "home". In views I use "request.route_url()" but I 
> need to get the same during the initialization of the App, meaning just after
> 
> config.make_wsgi_app()
> 
> I tried to use the registry introspection:
> 
>  config.add_route("home", "/")
> config.add_view(
> homeView,
> route_name="home",
> "home.jinja2",
> )
>  config.make_wsgi_app()
>  introspector = config.registry.introspector
>  route_intr = introspector.get('routes', "home")
>  print(route_intr)
> 
> But I get a route object with path and name but not with host, port, etc. I 
> also tried:
> 
> from pyramid.interfaces import IRoutesMapper
> mapper = config.registry.getUtility(IRoutesMapper)
> route = mapper.get_route("home")
> 
> I saw some code in Flask one can do something like this:
> 
> from flask.helpers import url_for
> 
> app.register_blueprint(views)
> url = url_for("home")
> 
> Is there something like that for Pyramid? Basically, I need to get for the 
> route "home" the same result as if I would do request.route_url("home") which 
> is "http://localhost:5900/";
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/40b8550f-f57a-4152-a744-0bb4304251een%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/BE8DF89B-C546-4092-A2D7-C2A4C65DF7C6%40gmail.com.

Re: [pylons-discuss] how does pyramid know what "create" , "view", "edit" etc is ?

[Michael Merickel]

The permission strings are arbitrary. For examples that are using ACLs (like 
the wiki tutorials) the only requirement is that the strings should match up to 
ACL entries that you are generating on your context objects. Pyramid does not 
care about the values of the strings and you could use "update" or "edit" or 
"foo". If a principal in the request and the permission on the view do not 
match an ACL entry then an HTTPForbidden exception is raised.

> On May 10, 2021, at 14:32, pzzcc  wrote:
> 
> thank you for the input everyone.
> 
> please correct me if I am wrong , does pyramid know what a ( view ) action  
> is ? 
> 
> does it know that an Edit action ( is a form that is being POSTed or Restful 
> call to update ?
> 
> same goes for create , does it have a way to figure out that create is ( PUT 
> )?
> 
> 
> in other words , if I go to a view and change the view config to have a 
> permssion of ( update ) instead of ( edit ) ,
> 
> and then go to principals  and update them accordingly , Pyramid it self wont 
> care,  would it ?  
> 
> can a view have more than one permission like ( update , create , view ) ? 
> 
> I am trying to figure out how it works so I can write a better code because I 
> have gone through the wiki tutorial , it is great but it leaves you with a 
> lot of question to be able to understand how things are put together .
> 
> 
> On Monday, May 10, 2021 at 12:30:03 PM UTC+3 Eldav wrote:
> Hello, 
> 
> you could have a look at the "Authorization" page of the SQLAlchemy + 
> URL dispatch wiki tutorial: 
> 
> https://pyramid.readthedocs.io/en/latest/tutorials/wiki2/authorization.html 
>  
> 
> Basically : you define your permission as string via an ACL mechanism. 
> Your permissions may be global (e.g. all members of the "managers" 
> group get the "manage" permission), or defined via a route factory. 
> Route factories allow for policies such as: every authenticated user 
> can "view" a page, its author can "edit" it. They also allow you to 
> simplify the code of your views. 
> 
> Hope this helps, 
> 
> Laurent. 
> 
> Le dim. 9 mai 2021 à 20:17, Thierry Florac  > a écrit : 
> > 
> > Hi, 
> > Are you asking about the way to protect a view with a permission, or about 
> > the way to grant this permission to a request? 
> > Best regards, 
> > Thierry 
> > -- 
> >  https://www.ulthar.net  -- 
> > http://pyams.readthedocs.io  
> > 
> > 
> > Le dim. 9 mai 2021 à 19:00, pzzcc  > > a écrit : 
> >> 
> >> Hi, 
> >> 
> >> I am trying to wrap my head around some pyramid concepts and I am trying 
> >> to figure out how does a view config know what a permission like ( view , 
> >> edit , create ) is ? 
> >> 
> >> does it rely on the pyramid_tm r or the routes or what ? 
> >> 
> >> I know how to use them but I need to wrap my head againts some concepts. 
> >> 
> >> thanks. 
> >> 
> >> -- 
> >> You received this message because you are subscribed to the Google Groups 
> >> "pylons-discuss" group. 
> >> To unsubscribe from this group and stop receiving emails from it, send an 
> >> email to pylons-discus...@googlegroups.com 
> >> . 
> >> To view this discussion on the web visit 
> >> https://groups.google.com/d/msgid/pylons-discuss/2b676239-b805-40d6-9ae2-1e4c60a9a7dcn%40googlegroups.com
> >>  
> >> .
> >>  
> > 
> > -- 
> > You received this message because you are subscribed to the Google Groups 
> > "pylons-discuss" group. 
> > To unsubscribe from this group and stop receiving emails from it, send an 
> > email to pylons-discus...@googlegroups.com 
> > . 
> > To view this discussion on the web visit 
> > https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWCYnWP_Rrbgk1ZBP1JBUN8KNztgj5%3DJ_Q_8%2B_uvAXAv_A%40mail.gmail.com
> >  
> > .
> >  
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/97b621fe-4b8b-4a44-884a-079813495ff4n%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons

Re: [pylons-discuss] Pyramid 2.0: Only venusian callbacks with category 'pyramid' are called by default

[Michael Merickel]

Estartu,

Thanks for bringing up the issue.

The initial reason I looked into changing it is because I had other venusian 
packages that I did not want scanned and when I dove into the details I could 
not find a good reason why things were the way they were (you can see I looked 
through the history in the original ticket), and I could find some good reasons 
that it should be "pyramid" which I don't think I explained well at all.

My main reason for changing it is that each scan() does have a contract with 
the decorator that needs to be satisfied. All pyramid-compatible decorators 
expect the scanner to have a "config" attribute that is a Pyramid Configurator 
object. That is the contract you get when you use "config.scan()". So if a 
decorator wants the config object, shouldn't it register as the "pyramid" 
category or another custom category passed to scan to guarantee it gets what it 
needs? This seems to be the only safe way to do it with the scan api that is an 
umbrella search through the code.

I tend to prefer explicit over implicit in these types of things which is why I 
advocated for the change and at the time no one advocated for keeping it. Which 
packages are using different categories?

- Michael

> On Mar 12, 2021, at 05:05, Gerhard Schmidt  wrote:
> 
> Hallo Steve,
> 
> I have found the change but there is not reason why this change is needed. It 
> simple states the change. What is the reason behind the change.
> 
> Are there security issues? Which added benefits does it provide? Why is 
> categories=None a bad idea?
> 
> These question aren't answered in the pull request nor in the commit message.
> 
> In the related issue (which was closed in 2012) the final comment was
> 'I'm going to leave "all categories", as we've come to rely on it being the 
> default. If you want to limit it within your application, please pass a set 
> of limiting categories.'
> The whole issuse is about using venusian outside of pyramid. So why change 
> the default in pyramid.
> 
> The change was 7 years after the issue was closed.
> 
> Regards
>   Estartu
> 
> Am 12.03.21 um 11:26 schrieb Steve Piercy:
>>> From the release notes:
>> https://docs.pylonsproject.org/projects/pyramid/en/latest/changes.html#backward-incompatibilities
>> pyramid.config.Configurator.scan will no longer, by default, execute 
>> Venusian decorator callbacks registered for categories other than 'pyramid'. 
>> To find any decorator regardless of category, specify config.scan(..., 
>> categories=None). See https://github.com/Pylons/pyramid/pull/3510
>> That PR in turn fixed:
>> https://github.com/Pylons/pyramid/issues/3502
>> Which in turn referenced:
>> https://github.com/Pylons/pyramid/issues/495
>> I don't know if that answers your question, but please feel free to 
>> elaborate if it does not.  Thank you!
>> --steve
>> On 3/12/21 1:36 AM, Gerhard Schmidt wrote:
>>> Hi,
>>> 
>>> thanks for the work.
>>> 
>>> is there a reason why Configurator.scan() now has a default value of 
>>> ('pyramid',) for categories. In pyramid 1 the default was None.
>>> 
>>> The change causes that only callbacks in category 'pyramid' are called by 
>>> default. If the scanner called with categories=None, venusian asks the 
>>> object which categories are used and invoke all callbacks of the scanned 
>>> package. venusian.attach() stores all uses categories in an attribute of 
>>> the object. So the object knowns all used categories.
>>> 
>>> Some packages use different categories, so there callbacks are not called 
>>> by default. You have either call config.scan with all the categories used 
>>> by your product and all the packages you include or set categories=None 
>>> when calling the config.scan().
>>> 
>>> So is there a reason for this change, because it causes some issues when 
>>> migrating to pyramid 2.0.
>>> 
>>> Regards
>>>Estartu
>>> 
>>> Am 01.03.21 um 04:40 schrieb Michael Merickel:
>>>> Yay, Pyramid 2.0 is out. Get it while it's hot!
>>>> 
>>>> If you're able to run 1.10.8 without deprecation warnings then you're in a 
>>>> great spot to upgrade. Several warnings were added to 1.10.x releases to 
>>>> help you prepare.
>>>> 
>>>> There's shockingly few backward-incompatible changes in this release, so 
>>>> don't let it scare you. There IS a laundry list of new features. Here are 
>>>> a few highlights:

[pylons-discuss] Pyramid 2.0 released

[Michael Merickel]

Yay, Pyramid 2.0 is out. Get it while it's hot!

If you're able to run 1.10.8 without deprecation warnings then you're in a 
great spot to upgrade. Several warnings were added to 1.10.x releases to help 
you prepare.

There's shockingly few backward-incompatible changes in this release, so don't 
let it scare you. There IS a laundry list of new features. Here are a few 
highlights:

- First release to drop Python 2, as well as 3.4 and 3.5.

- New security system for authentication/authorization. The legacy API is 
deprecated but available and backward compatible. ACLs and principals are not 
going away, but the policies are merged and restructured slightly which should 
be more consumable and flexible for users.

- Several defaults have changed, such as serializing sessions using JSON 
instead of Pickle.

- Cookiecutter and tutorials updated with new pytest fixtures.

- The "pyramid.compat" module is gone.

Read the "What's New in Pyramid 2.0" document for a comprehensive list of 
changes and upgrading notes:

https://docs.pylonsproject.org/projects/pyramid/en/2.0-branch/whatsnew-2.0.html 


2.0 release documentation:

https://docs.pylonsproject.org/projects/pyramid/en/2.0-branch/ 


You can install it via PyPI:

  pip install Pyramid==2.0

As always report any issues to the issue tracker (or here on the mailing list).

https://github.com/Pylons/pyramid/issues 


Special thanks to a few people specifically for their excellent work on this 
release:

- Theron Luhn
- Bert JW Regeer
- Steve Piercy

Thanks for everyone's efforts as well as support in getting this work done!

- Pyramid core developers

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/14F179EB-F647-42B7-845A-6091E1AFF697%40gmail.com.

[pylons-discuss] Pyramid 1.10.8 released

[Michael Merickel]

Pyramid 1.10.8 has been released.

This release fixes a warning with using the deprecated "imp" module on newer 
versions of Python.

The full changelog is here:
https://docs.pylonsproject.org/projects/pyramid/en/1.10-branch/changes.html

What's New In Pyramid 1.10:
https://docs.pylonsproject.org/projects/pyramid/en/1.10-branch/whatsnew-1.10.html

1.10 release documentation (across all alphas and betas, as well as when it 
gets to final release):
https://docs.pylonsproject.org/projects/pyramid/en/1.10-branch/

You can install it via PyPI:

  pip install Pyramid==1.10.8

Enjoy, and please report any issues you find to the issue tracker at
https://github.com/Pylons/pyramid/issues

Thanks!

- Pyramid core developers

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/E82A12DA-FEC1-41FC-A5D5-F5EB44DECB79%40gmail.com.

[pylons-discuss] Pyramid 1.10.7 released

[Michael Merickel]

Pyramid 1.10.7 has been released. Whoops, had one remaining PR that hadn't been 
merged in 1.10.6.

- Fixed an issue where reified properties would show up as functions in certain 
tools instead of attributes.

The full changelog is here:
https://docs.pylonsproject.org/projects/pyramid/en/1.10-branch/changes.html 


What's New In Pyramid 1.10:
https://docs.pylonsproject.org/projects/pyramid/en/1.10-branch/whatsnew-1.10.html
 


1.10 release documentation (across all alphas and betas, as well as when it gets
to final release):
https://docs.pylonsproject.org/projects/pyramid/en/1.10-branch/ 


You can install it via PyPI:

 pip install Pyramid==1.10.7

Enjoy, and please report any issues you find to the issue tracker at
https://github.com/Pylons/pyramid/issues 


Thanks!

- Pyramid core developers

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/276D0A6B-3C17-4728-A61B-C39359B48244%40gmail.com.

Re: [pylons-discuss] Pyramid 2.0b1 released

[Michael Merickel]

It's in the cookiecutter's "get_tm_session" on the master branch. Which is not 
the default branch.

The purpose of the pattern is if your model objects needed access to some 
settings or some other request properties. It's up to you to decide if that's 
good or bad. Of course you don't need to use it.

- Michael

> On Feb 20, 2021, at 17:39, Mike Orr  wrote:
> 
> On Sat, Feb 20, 2021 at 12:57 PM Michael Merickel  wrote:
>> Check out the new pattern of storing the request in the SQLAlchemy session 
>> object for easier access in your model layer without threadlocals!
> 
> Where is that? I don't see it in 'pyramid-cookiecutter-starter' or the 2.0 
> docs.
> 
> I do essentially the opposite. I have a request subclass with reified
> methods for external resources like 'request.sa_session()'.
> Cookiecutter is still using this concept although using
> 'config.add_request_method()' instead of a subclass:
> 
> https://github.com/Pylons/pyramid-cookiecutter-starter/blob/latest/%7B%7Bcookiecutter.repo_name%7D%7D/%7B%7Bcookiecutter.repo_name%7D%7D/sqlalchemy_models/__init__.py
> (Lines 71-75.)
> 
> My model classes used to have querying methods but I moved away from
> that because they ended up putting too much into the models and being
> too limiting (my querying needs inevitably expanded beyond the method
> arguments). I now have just the columns and relations in the models,
> and separate lib modules for high-level queries. These take a session
> argument.  I don't usually need a request argument except occasionally
> for URL generation. For that I have a postprocessing function that
> adds the URL attributes to the query results. For instance:
> 
> # View
> results = myapp.lib.mysearch.do_search(sa_session, ...criteria..)
># Return list of ORM objects,
> postprocess(results, request)
># Iterate through objects, adding URL attributes for HTML links,
># formatting dates, calculating display values, etc.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/CAH9f%3Duo821nzLFSttTcvn5xBqbAL2hisuQWUE_%2BCS8q15zUgRA%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/AE3A3A43-AECE-49D5-BB83-AAC8A3DC0798%40gmail.com.

[pylons-discuss] Pyramid 2.0b1 released

[Michael Merickel]

Pyramid 2.0b1 has been released.

- Fixes a circular reference / memory leak between request and context in some 
apps.

- A bunch of documentation and cookiecutter improvements. Check out the new 
pattern of storing the request in the SQLAlchemy session object for easier 
access in your model layer without threadlocals!

This is the last release prior to 2.0 which will probably come next weekend.

The full changelog is here:
https://docs.pylonsproject.org/projects/pyramid/en/2.0-branch/changes.html

What's New In Pyramid 2.0:
https://docs.pylonsproject.org/projects/pyramid/en/2.0-branch/whatsnew-2.0.html

2.0 release documentation (across all alphas and betas, as well as when it gets 
to final release):
https://docs.pylonsproject.org/projects/pyramid/en/2.0-branch/

You can install it via PyPI:

  pip install Pyramid==2.0b1

Enjoy, and please report any issues you find to the issue tracker at
https://github.com/Pylons/pyramid/issues

Thanks!

- Pyramid core developers

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/43A02D06-00CC-4A86-982B-521726283814%40gmail.com.

[pylons-discuss] Pyramid 1.10.6 released

[Michael Merickel]

Pyramid 1.10.6 has been released.

- Deprecated pyramid.compat so that apps can prepare more easily for their 
upgrade to 2.0.

- Fixed a potential memory leak in which a circular reference between context 
and request may rely on the garbage collector to clean up.

The full changelog is here:
https://docs.pylonsproject.org/projects/pyramid/en/1.10-branch/changes.html

What's New In Pyramid 1.10:
https://docs.pylonsproject.org/projects/pyramid/en/1.10-branch/whatsnew-1.10.html

1.10 release documentation (across all alphas and betas, as well as when it gets
to final release):
https://docs.pylonsproject.org/projects/pyramid/en/1.10-branch/

You can install it via PyPI:

  pip install Pyramid==1.10.6

Enjoy, and please report any issues you find to the issue tracker at
https://github.com/Pylons/pyramid/issues

Thanks!

- Pyramid core developers

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/D79416FA-1BEF-4B85-A0B0-770DCBD47D4A%40gmail.com.

Re: [pylons-discuss] Cornice / Colander design for file upload

[Michael Merickel]

To me it just kind of depends on what level of atomicity you need in your API 
endpoint. If you can accept the binary data without any other parameters then 
that's great, just do that. If you need it alongside other input then multipart 
is great. Some people also marshal that stuff in x-foo headers but I think 
there's very few cases where that's compelling. BASE64'd binary data into JSON 
is the least desirable but it really depends on the situation - I'm definitely 
guilty of using it in certain scenarios (if you just really really really want 
a JSON body cuz that's what's easiest on the client-side). 

It's worth noting that Deform has its own mechanism for storing the file upload 
on the server such that you can connect it up to the form data from a separate 
request. This is handy to avoid re-transferring the data when parts of the form 
fail to validate.

The CGI FieldStorage object is basically a wrapper around the content, 
providing a file-like interface for messing with it as webob may have done you 
a solid and stored the data to disk instead of blowing out your memory. You can 
always just read the content in from there. You'll also want to pull the 
content-type from that object. After that validation is up to you.

Finally, if the request got all the way to your WSGI app then it's almost 
guaranteed that it was already fully parsed/received from the client by your 
WSGI server or any middleware/API calls so it's (probably) too late at that 
point to really validate and reject based on the size. You'd want to do that 
upstream prior to the body being read if it's really large data you're worried 
about receiving.

- Michael

> On Feb 7, 2021, at 09:06, Thierry Florac  wrote:
> 
> Hi,
> 
> I'm starting to use Cornice, Colander and Swagger and I'm trying to create a 
> REST service which should allow users to upload file(s)...
> 
> Until now I'm using "multipart/form-data" encoding and it seems OK, but :
>  - is it actually a good practice to handle file uploads in a REST API using 
> this encoding? Should I prefer something like JSON with base64 string 
> encoding (including for use cases with large data files)??
>  - how should I handle schema verification with Colander for file upload 
> fields (what I actually receive using form-data encoding is a CGI 
> FieldStorage object) ?
> 
> Best regards for any advise,
> Thierry
> -- 
>   https://www.ulthar.net  -- 
> http://pyams.readthedocs.io 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWBv8Ufg-ezeHPqeRXjwuF2f-8iSE8VRy7k3_n9iRp-LqA%40mail.gmail.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/314E86A9-A36F-439F-AD01-3F581ED87B92%40gmail.com.

Re: [pylons-discuss] testing users with webtest

[Michael Merickel]

On Feb 8, 2021, at 18:38, zsol...@gmail.com  wrote:
> 
> There are a few things which are confusing me here:
> 1. I can remove get_cookie and get_csrf_token and just hard-code 
> 'dummy_csrf_token' into login / post(), and it still works. Am I missing 
> something here?

The CookieCSRFStoragePolicy does not attempt to verify the cookie's value, it 
just needs the value in the submitted form request (header or body) to match 
the value in the cookie.

> 2. I don't get all the CSRF behaviour here. Isn't it generated on the server 
> side and webtest is client side only? How can it make Pyramid accept a 
> "dummy_csrf_token" as a CSRF token? Is there some magic in 
> CookieCSRFStoragePolicy or WebTest?

The assumption is that no one but your domain can set cookies on your domain, 
so it doesn't need to be overly protected from tampering - it just needs to be 
a value that malicious code running on another domain can't access or set.

> 3. Do I understand right that using CookieCSRFStoragePolicy is required for 
> webtest CSRF to work?

No it's not required, but it is a really simple approach. There's plenty of 
alternatives to grab the CSRF token in your code so that you can re-submit it 
in subsequent requests but they aren't as easy. The trick to any CSRF request 
is that it's trying to enforce a GET followed by a POST, to avoid people 
triggering a cross-site POST from their own domain with malicious content. So 
if you do a GET first, you can parse the token out of the returned form, or 
webtest also has APIs that can be used to submit the form without any parsing, 
assuming your app supports a submitted web form submitted in 
x-www-form-urlencoded format.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/D4ABEBDF-68FB-41AF-AC9D-33CDCAE66AAA%40gmail.com.

[pylons-discuss] Pyramid 2.0b0 released

[Michael Merickel]

This is likely the last release prior to 2.0, pending any bug reports from 
users. We've updated the cookiecutter, improved the fixtures and security 
examples, and tutorials. As such, I'm happy to announce 2.0b0.

Read the "What's New in Pyramid 2.0" document for a comprehensive list of 
changes and handy notes on upgrading:

https://docs.pylonsproject.org/projects/pyramid/en/2.0-branch/whatsnew-2.0.html 


2.0 release documentation (across all alphas and betas, as well as when it gets 
to final release):

https://docs.pylonsproject.org/projects/pyramid/en/2.0-branch/ 


You can install it via PyPI:

  pip install Pyramid==2.0b0

As always report any issues to the issue tracker (or here on the mailing list).

https://github.com/Pylons/pyramid/issues 


- Pyramid core developers

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/EF536DA3-1EF0-45DF-B946-542424C015C2%40gmail.com.

Re: [pylons-discuss] Poetry and Pyramid

[Michael Merickel]

I cooked up a pyproject.toml that almost works like the setuptools version for 
the Pyramid scaffold. There is one minor difference in which files are included 
in the sdist versus wheel, but it's an open bug in poetry to fix.

You simply use this file in your project, then do things like `poetry install` 
and `poetry run pserve development.ini`.

https://gist.github.com/mmerickel/33bc8edc633da132a8f92dbcb03ec1da 


- Michael

> On Dec 11, 2020, at 19:13, MJF  wrote:
> 
> I have been away from using Pyramid (well, from starting a new project in 
> Pyramid) for a handful of years. Since then, I've grown to like using 
> Poetry[1] in my Python development work.
> 
> I see that there was some discussion[2] a couple years ago about this; I 
> don't see a resolution, though it seems that mmerickel also likes poetry :-) 
> [3]
> 
> I'd like to continue to use Poetry in my work, and I'd like to use Pyramid 
> too; it seems that they use wholly different ways of dealing with 
> scripts/entrypoints, etc.
> 
> There are no hits for "poetry" on this mailing list; my online searches for 
> pyramid and poetry are coming up dry too. Does anyone have any resources for 
> how to make these tools work together?
> 
> 1: https://python-poetry.org/
> 2: https://github.com/Pylons/pyramid/issues/3270
> 3: https://github.com/Pylons/pyramid/issues/3270#issuecomment-385475580
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/fd66e280-036e-434e-a777-78d9b8b37782n%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/ED50998D-5172-4F12-9674-B7E8028D76BE%40gmail.com.

Re: [pylons-discuss] Pyramid authentication callback not being called

[Michael Merickel]

The groupfinder is only invoked on requests that check authentication 
information. For example a view with a "permission=..." constraint or when 
calling request.authenticated_userid, or request.has_permission(...). It is not 
invoked for every request.

- Michael

> On Dec 12, 2020, at 11:40, PremAnand Lakshmanan  wrote:
> 
> I'm trying to implement authentication for my web application built using 
> Pyramid.
> 
> 
> Here is the code - 
> 
> https://github.com/PREM1980/spa_authentication 
> 
> 
> I have a issue when the registered callback using AuthTktAuthenticationPolicy 
> is not being called.
> 
> 
> Here is the workflow of the application.
> 1. The user goes to the login URL.
> 2. He his redirected to a oauth callback(simulated here) and the remember 
> function is called to save the request.
> 3. Expect any future calls will call the registerd callback for 
> authorisation(implemented for resource_1 as a test)
> 
> Problem:-
> 
> The registered callback is not being called.
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/CAMpk0qPuOeMTtM8PGRm%3DhAxPwemLv55gz3tWNpw7n6jRtiwCgg%40mail.gmail.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/C741FFEF-DEC7-4AED-AC6E-C0CC58E6F891%40gmail.com.

Re: [pylons-discuss] Testing advise

[Michael Merickel]

The "recommended" approach for doing this would be to create a DummyRequest and 
then use its route_url, etc methods. This will allow you to use Pyramid's 
actual url generation apis instead of re-implementing them yourself, as well as 
avoiding needing to use the private route mapper.

req = DummyRequest()
req.route_url(...)

It's natural to use the DummyRequest as well because Pyramid's url generation 
facilities are always relative to the properties of a WSGI request like scheme, 
host, script_name, etc.

That being said your fixtures look great.

- Michael

> On Dec 6, 2020, at 04:41, Lele Gaifax  wrote:
> 
> Hi all,
> 
> I'm used to have something like the following in my tests setup:
> 
>  
> https://gitlab.com/metapensiero/SoL/-/blob/master/tests/server/conftest.py#L45
> 
> that allows me to use the route name in my tests instead of the view URL,
> like:
> 
>  
> https://gitlab.com/metapensiero/SoL/-/blob/master/tests/server/test_data.py#L15
> 
> I wonder if I'm missing some existing similar facility, of if instead that
> approach could be suggested in the documentation, say here
> 
>  
> https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/testing.html#creating-functional-tests
> 
> What do you think?
> 
> Thanks&bye, lele.
> -- 
> nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
> real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
> l...@metapensiero.it  | -- Fortunato Depero, 1929.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/87h7ozqkqx.fsf%40metapensiero.it.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/857B2A48-5263-4423-B664-03ED42B8311E%40gmail.com.

[pylons-discuss] Pyramid 2.0a0 released

[Michael Merickel]

Alright folks, it's been a long time coming but we're here - the first alpha of 
Pyramid 2.0.

There's shockingly few backward-incompatible changes in this release, so don't 
let it scare you. There IS a laundry list of new features. Here are a few 
highlights:

- First release to drop Python 2, as well as 3.4 and 3.5.

- New security system for authentication/authorization. The legacy API is 
deprecated but available and backward compatible. ACLs and principals are not 
going away, but the policies are merged and restructured slightly which should 
be more consumable and flexible for users.

- Several defaults have changed, such as serializing sessions using JSON 
instead of Pickle.

- Cookiecutter and tutorials updated with new pytest fixtures.

Read the "What's New in Pyramid 2.0" document for a comprehensive list of 
changes and handy notes on upgrading:

https://docs.pylonsproject.org/projects/pyramid/en/2.0-branch/whatsnew-2.0.html 


2.0 release documentation (across all alphas and betas, as well as when it gets 
to final release):

https://docs.pylonsproject.org/projects/pyramid/en/2.0-branch/ 


You can install it via PyPI:

  pip install Pyramid==2.0a0

As always report any issues to the issue tracker (or here on the mailing list).

https://github.com/Pylons/pyramid/issues 


There are a few documentation / cookiecutter changes coming in subsequent 
releases but no major work remaining prior to a full release so don't hesitate 
to give it a spin and provide feedback.

I'd like to single out a few people specifically for their excellent work on 
this release:

- Theron Luhn
- Bert JW Regeer
- Steve Piercy

Thanks for everyone's efforts as well as support in getting this work done!

- Pyramid core developers

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/8C403B1B-7EBE-42CD-85A7-0BD88A7BDAA6%40gmail.com.

Re: [pylons-discuss] Request attributes vs request environment

[Michael Merickel]

Webob "request" objects are semi-ephemeral in the context of WSGI. Pyramid 
creates one while processing, and if you're using pyramid_retry then it'll make 
a new one for each attempt. The "environ", however, is one-per WSGI request and 
state you put in there will survive for the entire WSGI lifecycle and can be 
shared with middleware etc that is not Pyramid.

The recommendation is to almost always store data on the request unless you 
have a really good reason not to.

- Michael

> On Nov 12, 2020, at 14:10, 'Jonathan Vanasco' via pylons-discuss 
>  wrote:
> 
> I don't know about "correct", but I use `add_request_method` to attach a 
> custom object(s) with `reify=True,` and then store all the information in 
> those objects.  IIRC, the narrative documentation and tutorials use it for 
> similar purposes.
> 
> https://docs.pylonsproject.org/projects/pyramid/en/latest/api/config.html#pyramid.config.Configurator.add_request_method
> 
> 
> On Thursday, November 12, 2020 at 8:57:58 AM UTC-5 tfl...@gmail.com wrote:
> Hi,
> When we have to store custom information about current request, we can use 
> it's properties, attributes or environment (or even annotations).
> What is the correct usage of each of them, and is there any benefit of using 
> one of them against the others?
> 
> Best regards,
> Thierry
> -- 
>   https://www.ulthar.net  -- 
> http://pyams.readthedocs.io 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/344da45e-9eef-436a-9eb9-66e40dda0e56n%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/47DF611C-D487-4AED-80BA-47F75A1888A4%40gmail.com.

[pylons-discuss] Pyramid 1.10.5 released

[Michael Merickel]

Hey folks,

It's been a while! Pyramid 1.10.5 has been released. It includes super minor 
changes, but helps by adding a warning related to a backward-incompatible 
change in 2.0 related to the AuthTkt authentication cookies.

As a point of order, an alpha of 2.0 is coming probably in this next week as 
well, so looking forward to that!

The full changelog is here:
https://docs.pylonsproject.org/projects/pyramid/en/1.10-branch/changes.html

What's New In Pyramid 1.10:
https://docs.pylonsproject.org/projects/pyramid/en/1.10-branch/whatsnew-1.10.html

1.10 release documentation:
https://docs.pylonsproject.org/projects/pyramid/en/1.10-branch/

You can install it via PyPI:

  pip install Pyramid==1.10.5

Enjoy, and please report any issues you find to the issue tracker at
https://github.com/Pylons/pyramid/issues

Thanks!

- Pyramid core developers

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/47A7F3C6-0936-466B-AF9E-4A9C38DF4210%40gmail.com.

Re: [pylons-discuss] Will `render_to_response` ever return something that is not `pyramid.response.Response`?

[Michael Merickel]

There is a null_renderer in Pyramid, but it's not available via 
render_to_response. I do not see a code path that would return anything other 
than an object implementing pyramid.interfaces.IResponse (the expected return 
value from the app's configured IResponseFactory). This is, of course, slightly 
more general than pyramid.response.Response.

- Michael

> On Oct 12, 2020, at 13:48, 'Jonathan Vanasco' via pylons-discuss 
>  wrote:
> 
> I've looked at the code and am pretty sure the answer is no, but a bunch of 
> subclasses are used here, so I think it's best to ask --  Will 
> render_to_response ever return something that is not 
> `pyramid.response.Response`?
> 
> I have a small library that will act on the output of render_to_response; I 
> am trying to catch edge cases where it is improperly invoked and a developer 
> passes in a normal dict (which would be rendered via view_config template or 
> json renderer) instead of the expected output of render_to_response.
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/32fddda9-dba9-4c84-a26a-c0a28a6953dbn%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/FF9784FC-2B2D-48DA-9516-8DED98ED9D1E%40gmail.com.

Re: [pylons-discuss] Waitress connection cleanup

[Michael Merickel]

If a client hangs up then there's no problems, the issue that is being 
referenced is about unused, idle, connections that are taking up space that 
counts toward the limit (because while they are idle, a client could send data 
over it at any moment). The cleanup interval, etc is around how Waitress 
decides to kill those connections based on how long they have been idle.

> On Oct 8, 2020, at 15:42, Cooper Baird  wrote:
> 
> The documentation says "When the connection related to a channel is closed, 
> the channel is destroyed and garbage collected.". So is that process 
> different from the channel timeout connection cleanup process in that clients 
> hanging up the connection get their corresponding channels cleaned up 
> immediately (making room for new connections), whereas in a channel timeout, 
> those connections are only cleaned up periodically on the configured "cleanup 
> interval" and they take up a connection slot (out of the configured 
> connection limit) until that point?
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/b9fe8d57-8b92-4fb9-8073-963c2aca37e6n%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/78F0FD93-BA44-4C0F-A4EB-53448C7C11DC%40gmail.com.

Re: [pylons-discuss] Newbie Questions About Waitress Channels

[Michael Merickel]

The only default I've really changed on waitress in most apps I've written has 
been the number of threads. On Heroku I also configure waitress to understand 
the forwarding headers (see trusted_proxy docs) so that client data shows up 
properly in the WSGI app.

I would not worry about these issues unless you feel your site is susceptible 
to specific types of abuse (DDOS, slowloris, etc) - at which point I would 
recommend you look at tuning a proxy like nginx in front of basically any WSGI 
server to buffer / filter clients before they hit your backend.

> On Oct 5, 2020, at 20:34, Cooper Baird  wrote:
> 
> Awesome. That clarifies my questions. Thanks! I'm trying to get a sense as to 
> what I should set connection limit on a heroku 1x dyno. I ran ulimit -a 
> within the dyno using heroku run and saw a value of 1 for the max # of 
> file descriptors. 100 does seem very conservative, as mentioned in the 
> documentation. I don't want to set this value to something unsafe, but I 
> would like to maximize the number of open connections per dyno. Do you have 
> any advice in what I should set that value? I don't anticipate needing to 
> support > 100 connections at once super soon, but would like to plan ahead.
> 
> On Monday, October 5, 2020 at 9:23:25 PM UTC-4 mmer...@gmail.com wrote:
> The connection limit dictates how many individual tcp connections waitress 
> will handle at a time, and while those are alive (until client hangs up or 
> idle channel timeout) no other connections will be made. The backlog is a 
> signal to the OS to not outright reject connections even if waitress is not 
> willing to handle them yet.
> 
> From the list of connections, waitress will handle requests based on the 
> number of threads.
> 
> 
>> On Oct 5, 2020, at 20:06, Cooper Baird > > wrote:
>> 
> 
>> I am starting to use Waitress, and I am trying to understand how channels 
>> and the backlog work, so forgive me for my ignorance if I'm not 
>> understanding this correctly. Let's say, hypothetically, that I am using all 
>> of the default settings (so 100 connection limit, 1024 backlog capacity, 4 
>> threads, etc.). Let's say 100 users, all using HTTP/1.1 clients, go to the 
>> site at once and begin browsing. Does this mean that any additional users 
>> (past the 100) that try to browse the site will hit an error or have a 
>> connection timeout since the 100 users fill up the channel capacity of 100 
>> (and being HTTP/1.1 clients, all their requests will be served over the same 
>> channel, keeping it open)? If this is the case, then does that mean anyone 
>> past those initial 100 users will have to wait some time between 30s 
>> (cleanup interval) and 120s (channel timeout) to be able to browse? Or is 
>> this where the backlog comes in and channels can be reused somehow between 
>> users/clients? I apologize if that didn't all make sense. I can clarify 
>> anything that was unclear in my thought process/questioning.
>> 
> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "pylons-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to pylons-discus...@googlegroups.com 
>> .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/pylons-discuss/b9870007-07ea-4e25-bbd0-266e6d05bac2n%40googlegroups.com
>>  
>> .
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/2b78006d-50f1-4acb-a7e7-4d722e372244n%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/DC31D835-6859-47E7-8661-E107D210CE31%40gmail.com.

Re: [pylons-discuss] Newbie Questions About Waitress Channels

[Michael Merickel]

The connection limit dictates how many individual tcp connections waitress will 
handle at a time, and while those are alive (until client hangs up or idle 
channel timeout) no other connections will be made. The backlog is a signal to 
the OS to not outright reject connections even if waitress is not willing to 
handle them yet.

>From the list of connections, waitress will handle requests based on the 
>number of threads.

> On Oct 5, 2020, at 20:06, Cooper Baird  wrote:
> 
> I am starting to use Waitress, and I am trying to understand how channels and 
> the backlog work, so forgive me for my ignorance if I'm not understanding 
> this correctly. Let's say, hypothetically, that I am using all of the default 
> settings (so 100 connection limit, 1024 backlog capacity, 4 threads, etc.). 
> Let's say 100 users, all using HTTP/1.1 clients, go to the site at once and 
> begin browsing. Does this mean that any additional users (past the 100) that 
> try to browse the site will hit an error or have a connection timeout since 
> the 100 users fill up the channel capacity of 100 (and being HTTP/1.1 
> clients, all their requests will be served over the same channel, keeping it 
> open)? If this is the case, then does that mean anyone past those initial 100 
> users will have to wait some time between 30s (cleanup interval) and 120s 
> (channel timeout) to be able to browse? Or is this where the backlog comes in 
> and channels can be reused somehow between users/clients? I apologize if that 
> didn't all make sense. I can clarify anything that was unclear in my thought 
> process/questioning.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/b9870007-07ea-4e25-bbd0-266e6d05bac2n%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/AD7DAE51-F0DE-4B2A-AA20-D8F2772DCCF2%40gmail.com.

Re: [pylons-discuss] changing session cookie max_age?

[Michael Merickel]

`remember` is an authentication api and not directly tied to sessions. It does 
support kwargs that the authentication policy can utilize as it chooses.

Your question is about sessions, and the session cookie. It is up to 
pyramid_session_redis how it chooses to set the cookie, Pyramid does not handle 
it. It could support changing the max_age when you invoke 
`adjust_timeout_for_session` but it apparently is not.

My suggestion is to not worry about how long the cookie lives, set it for a 
long time or not expiring, and focus on just invalidating it server-side which 
is the only real place you can control it.

- Michael

> On Sep 19, 2020, at 07:34, zsol...@gmail.com  wrote:
> 
> Hi,
> 
> I'd like to implement the following session cookie behaviour:
> - non-logged-in users get a short-lived one, like 1800 seconds, enough for 
> all CSRF validation
> - when logging in, they extend their cookie to 1 year
> 
> I'm using pyramid_session_redis, and I can achieve the redis side changing 
> using 
> 
> headers = remember(request, user.id)
> redis_timeout = 3600 * 24 * 365 # one year in Redis 
> request.session.adjust_timeout_for_session(redis_timeout)
> return HTTPFound(location=..., headers=headers)
> 
> This changes the redis side just fine, however, I see no way to change the 
> max_age on the already set cookie and I see that remember() supports max_age, 
> but it doesn't work.
> 
> I've asked the developer of pyramid_session_redis and he said that there is 
> no remember() in that package, so it's unrelated to that. Still, in Pyramid 
> docs, I see that remember() supports max_age, so how is it?
> 
> https://docs.pylonsproject.org/projects/pyramid/en/latest/api/authentication.html#pyramid.authentication.AuthTktAuthenticationPolicy.remember
> and
> https://docs.pylonsproject.org/projects/pyramid/en/latest/api/security.html#pyramid.security.remember
> 
> So my solution right now is to set the session cookie max_age to something 
> very big then just limit things in Redis.
> 
> Is this the right solution? Ideally, I'd like to achieve never logging out 
> logged-in users, as it's bad for user experience, but at the same time limit 
> bots and non-logged-in users to 1800 seconds.
> 
> Zsolt
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/62b25a2d-4bec-4ed9-b3db-8c0e310384een%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/43911B70-EC5B-4EEC-8B6F-77EB6F380995%40gmail.com.

Re: [pylons-discuss] Migration from Flask - options?

[Michael Merickel]

The 2.0 security system is pretty backward compatible so I'd recommend just 
starting with 1.x but I might suggest using the master version of 
pyramid-cookiecutter-starter as it has significant improvements to the default 
test fixtures for new projects.

With respect to 2.0, sorry everyone but I've been dragging my feet on an issue 
in the new security api so it isn't out yet! Mailing list activity is a great 
motivator though!

- Michael

> On Aug 31, 2020, at 12:12, 'Jonathan Vanasco' via pylons-discuss 
>  wrote:
> 
> I've done a handful of side-by-side migrations or deployments.
> 
> I think you already identified the generally best approach IMHO:
> 
> >  Conceptually the simplest would be to have a auth cookie that is valid 
> in both, it could be set to only be created in one and honoured in the other.
> 
> However I recommend it with one caveat -- I would approach this functionality 
> as if it were it's own authentication micro-service.  It can live in one of 
> the two processes, it doesn't need a third, but I would structure the UX, 
> tests and development as if it were totally independent -- so the cookie 
> value would just contain login state the core account info needed for auth 
> and permissions.  If either process needs to store other data in a cookie, 
> use another cookie.
> 
> Reading Pyramid cookies in Flask is pretty simple, Pyramid sessions just 
> automate loading webob cookies with defaults
> 
> * https://github.com/Pylons/pyramid/blob/master/src/pyramid/session.py 
> <https://github.com/Pylons/pyramid/blob/master/src/pyramid/session.py>
> * https://github.com/Pylons/webob/blob/master/src/webob/cookies.py 
> <https://github.com/Pylons/webob/blob/master/src/webob/cookies.py>
> 
> 
> In terms of Pyramid versions, if you need Python2 support - 1.10 is your only 
> option.  Otherwise, just pay attention to the deprecations on Pyramid2 and 
> you should be able to transition from 1.10 to 2 very easily if you don't want 
> to run the pre-release.  Michael Merickel is the expert on this, but I think 
> he and his team have done a great job in planning the 2.0 changes by offering 
> a lot of "forward compatibility' options in 1.10 to prepare us for switching 
> to 2.0.  I could be wrong, but I think almost every one of the 2.0 changes 
> can be elected in 1.10.  The only difference I know of is the drastic change 
> to authentication/authorization (which may be worth using the pre-release). 
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com 
> <mailto:pylons-discuss+unsubscr...@googlegroups.com>.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/b0d9e559-d070-4522-82ef-217af2c63de2o%40googlegroups.com
>  
> <https://groups.google.com/d/msgid/pylons-discuss/b0d9e559-d070-4522-82ef-217af2c63de2o%40googlegroups.com?utm_medium=email&utm_source=footer>.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/7146D524-E61C-4B70-A78A-505F42059524%40gmail.com.

Re: [pylons-discuss] Migration from Flask - options?

[Michael Merickel]

Are you trying to host the apps in the same process? Where do you want certain 
shared parts to live as you migrate? In Pyramid code? In Flask code? Agnostic?

Option 1: If you want Pyramid to be able to use any Flask code then you'll have 
to likely setup Flask's threadlocal variables / request object inside the 
Pyramid request so that you can invoke Flask APIs.
Option 2: If you want Flask to be able to use Pyramid code as you move it into 
Pyramid then you'll have to setup Pyramid's threadlocal variables / request 
object so that you can invoke Pyramid APIs.
Option 3: If you want it agnostic then both Flask and Pyramid code need to pass 
their relevant data (path, headers, etc) into a function that doesn't care 
where it came from.

You can do any of the things I mentioned above - I'm not sure what's the best 
for your situation. If you move your auth into Pyramid then you're basically 
talking about giving Flask a way to invoke 
`pyramid_request.has_permission(...)` which falls into Option 2 and would 
likely be best going forward if you're trying to move code into Pyramid.

If you just want the cookie to be parseable in both frameworks, and write most 
of the security code separately in each, that should be quite doable as well 
because in Pyramid you can just define your own AuthenticationPolicy which 
could probably reuse a lot of code to parse/validate the cookie. Worst case 
scenario you can always write an AuthenticationPolicy that just hits a trusted 
REST API or something in the Flask code to parse/validate it, and then you just 
have Flask do the work there.

If you're running the apps in the same process then they can potentially call 
each other pretty easily. For example with WebOb it has 
`request.get_response(app)` in which you construct a WebOb request object, and 
it can talk directly to another WSGI app (Flask, etc) and get back a WebOb 
response. This would automatically handle all the threadlocal stuff I mentioned 
above at least between Flask/Pyramid - no promises for whatever you're doing 
with other globals / threadlocals from other frameworks you're using for ZODB 
etc.

Anyway without more details I can only focus on these high level options but I 
hope it helps. There are certain advantages of them both being Python WSGI apps 
in that you can actually run them together in the same process, and invoke code 
from each other without hitting a network. You can setup something like rutter 
(path-based dispatch between multiple WSGI apps) or pyramid.wsgi.wsgiapp2 
(Pyramid can wrap other WSGI apps) to handle what you want in Pyramid and 
dispatch the rest to Flask.

- Michael

> On Aug 28, 2020, at 21:12, p...@thirdfloor.com.au  
> wrote:
> 
> Hi. 
> 
> We have a non-trivial Flask app that I'm in the process of moving chunks of 
> to Pyramid and would like to get some input on options for how we run two 
> systems until the whole lots is migrated.
> 
> The data layer is agnostic with regards web framework (using ZODB) so that 
> there are no problems having two systems concurrently access it. However I'm 
> struggling with the best way to go with auth across both. Conceptually the 
> simplest would be to have a auth cookie that is valid in both, it could be 
> set to only be created in one and honoured in the other. A call back from one 
> system to the other to verify a cookie is another way but that just increases 
> moving parts.
> 
> Has anyone done this before or have any other ideas?
> 
> Thanks for any input.
> 
> Peter W.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/124cda96-4016-4049-b591-bb1fa25da663n%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/2A72C370-0A31-4086-B70D-A36441D83E4F%40gmail.com.

Re: [pylons-discuss] Waitress how setup channel_timeout and max size limits

[Michael Merickel]

Yeah it's possible for the channel / socket to close (client hangup, channel 
timeout, etc) while the WSGI app is still processing the request. The WSGI 
execution thread will have no idea until it tries to actually write the 
response at which point an exception may be raised but this is usually after 
the WSGI app is done processing the request unless you go out of your way to 
stream a response.

- Michael

> On May 20, 2020, at 15:58, Michal Seidl  wrote:
> 
> Thanks both Michael and Bert for explanation. process x thread
> Yes the documentation confuse me a little bit specially this sentence in 
> channel_timeout docs.
> 
> "Inactive" is defined as "has received no data from a client and has sent no 
> data to a client".
> 
> Best regards Michal
> 
> On 5/20/20 10:06 PM, Bert JW Regeer wrote:
>> Hey Michal,
>> Unfortunately there is nothing that will cause Waitress to kill a thread, 
>> because killing a thread in Python is not possible. The channel_timeout 
>> refers to how long a keep-alive connection stays open, a request that is 
>> processing in the WSGI thread will still be considered open, and active. The 
>> documentation around this probably needs a bit more work.
>> What you want is not possible with waitress.
>> Bert
>>> On May 20, 2020, at 06:53, Michal Seidl >> > wrote:
>>> 
>>> Hello,
>>> I can not figure out how to setup timeout (request processing takes to much 
>>> time) and size limit for output data.
>>> 
>>> I am starting server like this:
>>> 
>>> 
>>> |serve(application, host='0.0.0.0', port=port, cleanup_interval=2, 
>>> channel_timeout=4 )
>>> |
>>> 
>>> I am using Werkzeug |Request| and |Response| objects and |jsonrpcserver 
>>> dispatch| method to delegate request to my processing method. In this 
>>> method I have
>>> 
>>> |time.sleep(timeToSleep)
>>> |
>>> 
>>> But even I setup |timeToSleep| much bigger (30s) than |channel_timeout 
>>> (4s)| I can seen that the request is always processed sucefully?
>>> 
>>> 
>>> For output size limit I even can not find any setting argument.
>>> 
>>> 
>>> In best case, if one of these limit would be fulfill the server 
>>> stop/cancel/kill according process/thread and returns to client some 
>>> meaningful response.
>>> 
>>> 
>>> Thanks for any help.
>>> 
>>> 
>>> -- 
>>> You received this message because you are subscribed to the Google Groups 
>>> "pylons-discuss" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to pylons-discuss+unsubscr...@googlegroups.com 
>>> .
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/pylons-discuss/e2f650bb-fdc7-4a72-a769-129280354c15%40googlegroups.com
>>>  
>>> .
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "pylons-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to pylons-discuss+unsubscr...@googlegroups.com 
>> .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/pylons-discuss/ED813905-230F-4C87-9D64-AA5BCBEEDC2E%400x58.com
>>  
>> .
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/67899690-b617-02ba-cbbb-3dc182d2021d%40gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/4FECF8F5-342A-4746-B57E-5E8B7C8D17EC%40gmail.com.

Re: [pylons-discuss] Waitress how setup channel_timeout and max size limits

[Michael Merickel]

Waitress is a threaded WSGI server and there isn't a safe way to kill threads 
inside of a process while they are blocked executing synchronous code. Even if 
the underlying channel is closed, the thread continues until the request is 
completed processing and then the response is simply discarded because the 
channel is gone. This is definitely a limitation of the thread-based model but 
in practice isn't generally a big deal. If you do need fine-grained control 
per-request, then a request-per-process server like gunicorn or uwsgi can help 
here because they can kill the process if it's taking too long to handle a 
request. There are pros and cons to the different approaches in terms of 
control, memory usage, etc.

- Michael

> On May 20, 2020, at 06:29, Michal Seidl  wrote:
> 
> Hello,
> I can not figure out how to setup timeout (request processing takes to much 
> time) and limit for output data.
> 
> I am starting server like this.
> 
> 
>  serve(application,
> host='0.0.0.0',
> port=port,
> cleanup_interval=2,
> channel_timeout=4
>   )
> 
> 
> I am using Werkzeug Request and Response objects and jsonrpcserver dispatch 
> method to delegate request to my processing method. In this method I have
> 
> time.sleep(timeToSleep)
>  
> But even I setup timeToSleep much bigger (120s) than channel_timeout (4s) I 
> can seen that the request is always processed and return to client?
> 
> 
> 
> For response size limit I can not even find suitable setting parameter.
> 
> 
> 
> It would be nice if in both cases (timeout, maxsize reponse) server will be 
> able to stop/cancel/kill the corresponding process/thread/call and return 
> some meaningful response.
> 
> 
> 
> Thanks for any help.
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/88812140-6abf-4403-af8c-d2639418a543%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/A37B2F04-FB0E-4713-AF3B-BC5FBB85E708%40gmail.com.

Re: [pylons-discuss] WebSocket Integration

[Michael Merickel]

This looks awesome!

Things that aren’t clear to me are how the config and potentially other 
services / settings are shared as well as authentication and data store 
connections. And what the pitfalls may be there. 

It looks like it’s using asgiref so I assume you are supposed to use its 
mechanisms for jumping between sync and async apis?

Also what can you say about Pyramid’s apis being used at all in the async 
context? 

- Michael

> On May 13, 2020, at 17:51, Landreville  wrote:
> 
> 
> Hi All,
> 
> I've started experimenting with integrating WebSockets into Pyramid 
> applications and came up with this framework/library for implementing them: 
> https://gitlab.com/landreville/rbow
> 
> I haven't used it in a production application yet. Right now I'm looking for 
> some feedback if this seems useful and if there are any ideas for improving 
> this.
> 
> In general it runs an ASGI application that wraps the Pyramid app and uses 
> Venusian to register websocket "views". These are classes that manage the 
> websocket functionality. They also provide options for working with 
> websockets asynchronously, synchronously, or with a Pyramid request (kinda).
> 
> Lastly it also provides "channels" that can send messages to all websockets 
> subscribed to a channel and can call a function to send data periodically 
> (such as grabbing stats and sending them to the clients every 5 seconds).
> 
> 
> Thanks for any feedback,
> 
> Landreville
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/4db62ffd-09cb-4ccc-a277-73c5104a47e9%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/87BCC579-8317-4596-9ACB-C080FE1792D3%40gmail.com.

Re: [pylons-discuss] Python 3.8 and __init__.py in views directory

[Michael Merickel]

Sorry can you give more context? Did you delete the __init__.py in your views 
folder? If so, I would expect some issues.

> On Mar 24, 2020, at 22:12, Sydo Luciani  wrote:
> 
> No problem with python 3.5 environment, but missing views/__init__.py, throws 
> below errors in Python 3.8.
> 
> Even including relative path to view module in 
> config.scan('.module_name/views/view_module') has no effect.
> 
> Error starts with:
> pyramid.httpexceptions.HTTPNotFound: The resource could not be found.
> During handling of the above exception, another exception occurred:
> 
> And trace ends with:
>   File "/usr/lib/python3.8/posixpath.py", line 152, in dirname
> p = os.fspath(p)
> 
> 
> Is it by design ?
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/7edc8353-96b9-4bcc-85eb-372f049e6934%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/1E9C3387-2E70-4C4A-8AD7-F47905B31D6C%40gmail.com.

Re: [pylons-discuss] Saving a lot of data to Postgresql database in Pyramid using SQLAlchemy

[Michael Merickel]

dbsession.add on each row is pretty much worse case scenario. Start with 
https://docs.sqlalchemy.org/en/13/_modules/examples/performance/bulk_inserts.html
 

 which shows you how to use dbsession.bulk_insert_mappings and several other 
approaches. The only caveat that's Pyramid-related is that since the dbsession 
is controlled by the tm, if you drop down to some of the core apis you'll need 
to use zope.sqlalchemy.mark_changed(dbsession) at the end to have your data 
committed.

Of course there's always the ability to use the raw psycopg2 connection as well 
which can get you closer to COPY-like performance - kinda depends on your 
needs, but I've had success just using bulk_insert_mappings on the ORM.

- Michael

> On Mar 20, 2020, at 14:41, Emerson Barea  wrote:
> 
> Hi there.
> 
> Some times my app needs to create and save almost a million records in a 
> Postgres database. In that case, I'm looking for the best way to do this, 
> because the procedures I've used so far are very slow.
> 
> I will present some ways that I tried that were very slow, as well the 
> workarounds that I tried to improve them and the errors that occurred:
> 
> 1 - a for loop to generate the records and add them to the transaction manager
> 
> import argparse
> import getopt
> import sys
> 
> from pyramid.paster import bootstrap, setup_logging
> from sqlalchemy.exc import OperationalError
> 
> 
> class Topology(object):
> def __init__(self, dbsession):
> self.dbsession = dbsession
> 
> def autonomous_system(self):
> 
> 
> # SOME PROCESS THAT RETURNS array1 AND array2
> 
> 
> for i in array1:
> record = Table(field1=array1[i],
>#other fields here
>   )
> self.dbsession.add(record)
> 
> 
> def parse_args(config_file):
> parser = argparse.ArgumentParser()
> parser.add_argument(
> 'config_uri',
> help='Configuration file, e.g., pyramid.ini',
> )
> return parser.parse_args(config_file.split())
> 
> 
> def main(argv=sys.argv[1:]):
> try:
> opts, args = getopt.getopt(argv, 'h:', ["config-file="])
> except getopt.GetoptError:
> print('* Usage: topology --config-file=pyramid.ini')
> sys.exit(2)
> for opt, arg in opts:
> if opt == '-h':
> print('* Usage: topology --config-file=pyramid.ini')
> sys.exit()
> elif opt == '--config-file':
> config_file = arg
> 
> args = parse_args(config_file)
> setup_logging(args.config_uri)
> env = bootstrap(args.config_uri)
> try:
> t = Topology(dbsession)
> with env['request'].tm:
> dbsession = env['request'].dbsession
> t.autonomous_system()
> except OperationalError:
> print('Database error')
> 
> In this case, I noticed that the "dbsession.add (prefix)" inside the loop 
> made the process very slow. I tried to create an array with the records in 
> the loop and add everything to the transaction manager just once outside the 
> loop using "bulk_save_objects" 
> (https://docs.sqlalchemy.org/en/13/orm/persistence_techniques.html#bulk-operations
>  
> ),
>  but I don't understand what "s = Session ()" is.
> 
> s = Session()
> objects = [
> models.AutonomousSystem(autonomous_system=2, id_topology=68),
> models.AutonomousSystem(autonomous_system=3, id_topology=68),
> models.AutonomousSystem(autonomous_system=4, id_topology=68),
> models.AutonomousSystem(autonomous_system=5, id_topology=68)
> ]
> s.bulk_save_objects(objects)
> 
> 2 - insert a pandas dataframe into the database
> 
> Another way that I tried, and that I think it has the best performance for 
> working with almost a million records, is to use panda dataframes. It is very 
> fast to generate all records using pandas dataframes, but I'm not getting 
> success to insert all data from the dataframe into Postgres database.
> 
> I tried "to_sql" procedure 
> (https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_sql.html
>  
> ),
>  but I don't know what is the "engine" in the "df.to_sql('users', 
> con=engine)" command.
> 
> Please, can someone help me with these questions?
> 
> Emerson
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/CAO1UhdXJSP_mvJm3Br-_oiGx3FZBR9fcKagQtfFnRXscKWQErg%40ma

Re: [pylons-discuss] Serving a Single File from the Root

[Michael Merickel]

The unicode issue is just due to opening the file in unicode mode (the default 
in python 3) and then setting the result to body (which expects bytes). Modify 
the open calls to use open(..., 'rb') and it should work.

> On Mar 14, 2020, at 11:16, Sydo Luciani  wrote:
> 
> Procedure to serve favicon.ico from root:
> https://docs.pylonsproject.org/projects/pyramid-cookbook/en/latest/static_assets/serving-files.html#serving-a-single-file-from-the-root
>  
> 
> 
> Throws error:
> UnicodeDecodeError: 'utf-8' codec can't decode byte 0x9e in position 34: 
> invalid start byte
> 
> Apparently open().read() can't decode favicon.ico.
> 
> Has any one had success with this procedure ?
> 
> I had to change the favicon_view to return "FileResponse" object to make it 
> work:
> https://docs.pylonsproject.org/projects/pyramid-cookbook/en/latest/static_assets/serving-files.html#serving-file-content-dynamically
>  
> 
> 
> somehow the .rst file on github not showing the code that shows in .html or 
> in .raw file:
> https://github.com/Pylons/pyramid_cookbook/blob/master/docs/static_assets/serving-files.rst#serving-a-single-file-from-the-root
>  
> 
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/6aab8e65-007a-48e4-b400-41af3fa26623%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/1F9C7B1F-32EC-4FDB-B700-13F2700E302C%40gmail.com.

Re: [pylons-discuss] Trying to execute console_scripts in web browser (with nginx and uwsgi)

[Michael Merickel]

Sure I'd at least see if things work fine using waitress to try and confirm
that it's a uwsgi issue. I'm sure uwsgi can be fixed to deal with the
subprocesses better but I do not know how myself.

On Tue, Mar 3, 2020 at 1:39 PM Emerson Barea 
wrote:

> So I'll look for another python server than uwsgi to run my app.
>
> Thank you Michael.
>
>
> Em terça-feira, 3 de março de 2020 15:40:42 UTC-3, Michael Merickel
> escreveu:
>>
>> I suspect it's something to do with UWsgi doing funky things with fork
>> and subprocesses. For example
>> https://stackoverflow.com/questions/17592692/running-a-subprocess-in-uwsgi-application
>> .
>>
>> On Tue, Mar 3, 2020 at 12:37 PM Emerson Barea 
>> wrote:
>>
>>> Hi Michael, thanks for your reply. I'm sorry I didn't confirm it
>>> earlier, but I've been quite sick this past week.
>>>
>>> Well, I tried to follow the suggestion you made, making the following
>>> changes:
>>>
>>> scripts/tests.py
>>>
>>> def main(argv):
>>> 
>>>
>>>
>>> if __name__ == '__main__':
>>> main(argv=sys.argv[1:])
>>>
>>>
>>>
>>> my view code:
>>>
>>> arguments = ['--config_file=minisecbgp.ini',
>>>  '--execution_type=create_node',
>>>  '--hostname=%s' % form.node.data,
>>>  '--username=%s' % form.username.data,
>>>  '--password=%s' % form.password.data]
>>> #subprocess.Popen(['tests'] + arguments)
>>> subprocess.Popen([sys.executable, '-m', 'minisecbgp.scripts.tests'] + 
>>> arguments)
>>>
>>>
>>> and now I receive this erro when I try to execute the view in browser:
>>>
>>> Mar  3 15:24:57 lpttch uwsgi[7284]: /home/tocha/Documentos/projetos/
>>> MiniSecBGP/venv/bin/uwsgi: unrecognized option
>>> '--config_file=minisecbgp.ini'
>>> Mar  3 15:24:57 lpttch uwsgi[7284]: getopt_long() error
>>>
>>> I tried to modify the setyp.py console_scripts to
>>>
>>> 'tests = minisecbgp.scripts.tests'
>>>
>>> or removing this entry_point, since the call to the script is being made
>>> directly now, but I always got the same error.
>>>
>>> Please, do you known what I'm doing wrong?
>>>
>>> Thank you.
>>>
>>> Emerson
>>>
>>>
>>> Em quinta-feira, 27 de fevereiro de 2020 21:28:04 UTC-3, Michael
>>> Merickel escreveu:
>>>
>>>> Your environment isn't modifying the env PATH, which is what Popen is
>>>> relying on to find the script.
>>>>
>>>> It'd be better not rely on the PATH and instead just run the code using
>>>> `python -m foo`, but that doesn't actually work with console scripts. You
>>>> would instead do `subprocess.Popen([sys.executable, '-m',
>>>> 'minisecbgp.scripts.config'] + arguments)`. You'd then need to define an
>>>> `if __name__ == '__main__': main()` in your script instead of relying on
>>>> the console script to invoke your main function.
>>>>
>>>> Alternatively fix your PATH, but I find that less ideal because it can
>>>> change per-environment where the console scripts are actually installed.
>>>>
>>>> - Michael
>>>>
>>>> On Thu, Feb 27, 2020 at 5:41 PM Emerson Barea 
>>>> wrote:
>>>>
>>>>> Hi there.
>>>>>
>>>>> My application has some scripts on .app.scripts (.app.scripts.tests
>>>>> and .app.scripts.config).
>>>>>
>>>>> I configured this scripts in setup.py file like this:
>>>>>
>>>>> entry_points={
>>>>> 'paste.app_factory': [
>>>>> 'main = minisecbgp:main'
>>>>> ],
>>>>> 'console_scripts': [
>>>>> 'initialize_minisecbgp_db = 
>>>>> minisecbgp.scripts.initialize_db:main',
>>>>> 'tests = minisecbgp.scripts.tests:main',
>>>>> 'validate_hostname = minisecbgp.scripts.validate_hostname:main',
>>>>> 'config = minisecbgp.scripts.config:main',
>>>>> ],
>>>>>
>>>>>
>>>>

Re: [pylons-discuss] Trying to execute console_scripts in web browser (with nginx and uwsgi)

[Michael Merickel]

I suspect it's something to do with UWsgi doing funky things with fork and
subprocesses. For example
https://stackoverflow.com/questions/17592692/running-a-subprocess-in-uwsgi-application
.

On Tue, Mar 3, 2020 at 12:37 PM Emerson Barea 
wrote:

> Hi Michael, thanks for your reply. I'm sorry I didn't confirm it earlier,
> but I've been quite sick this past week.
>
> Well, I tried to follow the suggestion you made, making the following
> changes:
>
> scripts/tests.py
>
> def main(argv):
> 
>
>
> if __name__ == '__main__':
> main(argv=sys.argv[1:])
>
>
>
> my view code:
>
> arguments = ['--config_file=minisecbgp.ini',
>  '--execution_type=create_node',
>  '--hostname=%s' % form.node.data,
>  '--username=%s' % form.username.data,
>  '--password=%s' % form.password.data]
> #subprocess.Popen(['tests'] + arguments)
> subprocess.Popen([sys.executable, '-m', 'minisecbgp.scripts.tests'] + 
> arguments)
>
>
> and now I receive this erro when I try to execute the view in browser:
>
> Mar  3 15:24:57 lpttch uwsgi[7284]: /home/tocha/Documentos/projetos/
> MiniSecBGP/venv/bin/uwsgi: unrecognized option
> '--config_file=minisecbgp.ini'
> Mar  3 15:24:57 lpttch uwsgi[7284]: getopt_long() error
>
> I tried to modify the setyp.py console_scripts to
>
> 'tests = minisecbgp.scripts.tests'
>
> or removing this entry_point, since the call to the script is being made
> directly now, but I always got the same error.
>
> Please, do you known what I'm doing wrong?
>
> Thank you.
>
> Emerson
>
>
> Em quinta-feira, 27 de fevereiro de 2020 21:28:04 UTC-3, Michael Merickel
> escreveu:
>
>> Your environment isn't modifying the env PATH, which is what Popen is
>> relying on to find the script.
>>
>> It'd be better not rely on the PATH and instead just run the code using
>> `python -m foo`, but that doesn't actually work with console scripts. You
>> would instead do `subprocess.Popen([sys.executable, '-m',
>> 'minisecbgp.scripts.config'] + arguments)`. You'd then need to define an
>> `if __name__ == '__main__': main()` in your script instead of relying on
>> the console script to invoke your main function.
>>
>> Alternatively fix your PATH, but I find that less ideal because it can
>> change per-environment where the console scripts are actually installed.
>>
>> - Michael
>>
>> On Thu, Feb 27, 2020 at 5:41 PM Emerson Barea 
>> wrote:
>>
>>> Hi there.
>>>
>>> My application has some scripts on .app.scripts (.app.scripts.tests and
>>> .app.scripts.config).
>>>
>>> I configured this scripts in setup.py file like this:
>>>
>>> entry_points={
>>> 'paste.app_factory': [
>>> 'main = minisecbgp:main'
>>> ],
>>> 'console_scripts': [
>>> 'initialize_minisecbgp_db = minisecbgp.scripts.initialize_db:main',
>>> 'tests = minisecbgp.scripts.tests:main',
>>> 'validate_hostname = minisecbgp.scripts.validate_hostname:main',
>>> 'config = minisecbgp.scripts.config:main',
>>> ],
>>>
>>>
>>> I call these two scripts in my view with the code below:
>>>
>>> arguments = ['--config_file=minisecbgp.ini',
>>>  '--hostname=%s' % form.node.data,
>>>  '--username=%s' % form.username.data,
>>>  '--password=%s' % form.password.data]
>>> subprocess.Popen(['config'] + arguments)
>>>
>>>
>>> and, when I run my application with the commands  below, everything
>>> works well. The page works fine and the scripts tests and config works well.
>>>
>>> pip install -e ".[testing]"
>>> pserve minisecbgp.ini --reload
>>>
>>>
>>> So, I want to put my app in production, and I'm trying to use uwsgi and
>>> nginx to do it. When I configured uwsgi and nginx and open the app in
>>> browser, the app works well, but when I call the view that executes the
>>> scripts tests and config, I receave a 502 bad gateway error. Looking at
>>> syslog file, I receaved this error:
>>>
>>> Feb 27 20:12:56 lpttch uwsgi[14110]: subprocess.Popen(['tests'] +
>>> arguments)
>>>

Re: [pylons-discuss] Trying to execute console_scripts in web browser (with nginx and uwsgi)

[Michael Merickel]

Your environment isn't modifying the env PATH, which is what Popen is
relying on to find the script.

It'd be better not rely on the PATH and instead just run the code using
`python -m foo`, but that doesn't actually work with console scripts. You
would instead do `subprocess.Popen([sys.executable, '-m',
'minisecbgp.scripts.config'] + arguments)`. You'd then need to define an
`if __name__ == '__main__': main()` in your script instead of relying on
the console script to invoke your main function.

Alternatively fix your PATH, but I find that less ideal because it can
change per-environment where the console scripts are actually installed.

- Michael

On Thu, Feb 27, 2020 at 5:41 PM Emerson Barea 
wrote:

> Hi there.
>
> My application has some scripts on .app.scripts (.app.scripts.tests and
> .app.scripts.config).
>
> I configured this scripts in setup.py file like this:
>
> entry_points={
> 'paste.app_factory': [
> 'main = minisecbgp:main'
> ],
> 'console_scripts': [
> 'initialize_minisecbgp_db = minisecbgp.scripts.initialize_db:main',
> 'tests = minisecbgp.scripts.tests:main',
> 'validate_hostname = minisecbgp.scripts.validate_hostname:main',
> 'config = minisecbgp.scripts.config:main',
> ],
>
>
> I call these two scripts in my view with the code below:
>
> arguments = ['--config_file=minisecbgp.ini',
>  '--hostname=%s' % form.node.data,
>  '--username=%s' % form.username.data,
>  '--password=%s' % form.password.data]
> subprocess.Popen(['config'] + arguments)
>
>
> and, when I run my application with the commands  below, everything works
> well. The page works fine and the scripts tests and config works well.
>
> pip install -e ".[testing]"
> pserve minisecbgp.ini --reload
>
>
> So, I want to put my app in production, and I'm trying to use uwsgi and
> nginx to do it. When I configured uwsgi and nginx and open the app in
> browser, the app works well, but when I call the view that executes the
> scripts tests and config, I receave a 502 bad gateway error. Looking at
> syslog file, I receaved this error:
>
> Feb 27 20:12:56 lpttch uwsgi[14110]: subprocess.Popen(['tests'] +
> arguments)
> Feb 27 20:12:56 lpttch uwsgi[14110]:   File
> "/usr/lib/python3.6/subprocess.py", line 729, in __init__
> Feb 27 20:12:56 lpttch uwsgi[14110]: restore_signals,
> start_new_session)
> Feb 27 20:12:56 lpttch uwsgi[14110]:   File
> "/usr/lib/python3.6/subprocess.py", line 1364, in _execute_child
> Feb 27 20:12:56 lpttch uwsgi[14110]: raise child_exception_type(
> errno_num, err_msg, err_filename)
> Feb 27 20:12:56 lpttch uwsgi[14110]: FileNotFoundError: [Errno 2] No such
> file or directory: 'tests': 'tests'
>
> Please, can somebody help me?
>
> Emerson
>
> --
> You received this message because you are subscribed to the Google Groups
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pylons-discuss/9a78ebba-f50d-4143-9fea-550d3bce9e1e%40googlegroups.com
> 
> .
>


-- 

Michael

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwFreu9DZgsBA6iwUaPQrHTJvKW_rpNhei9C%3DTApF7ohkg%40mail.gmail.com.

Re: [pylons-discuss] pyramid_tm - is there an acceptable way to unjoin the transaction and use explicit commits?

[Michael Merickel]

It’s not recommended. But it’s there for you to use if you want. When you
do a commit you’re crossing transaction boundaries to continue the request
and all of the checks you did early on are not guaranteed any longer to be
valid.

Using a separate connection is the best and most generally sound way to go
for recording data not specific to the resource-level goals of processing
the request.

On Sun, Feb 9, 2020 at 12:37 Mike Orr  wrote:

> I sometimes use explicit commits to log usage stats, which I want to
> succeed even if the request later aborts. I use a separate database
> connection not attached to the transaction. I tried to use pyramid_tm
> commit multiple times but it seemed it could only be used once. So
> you're supposed to use transaction.commint/abort followed immediately
> by transaction.begin? That wasn't clear in the docs.
>
> I've also used a custom transaction adapter to get a Redis session
> attached to the transaction. Since Redis is non-transactinal, t saves
> its pending work in the object, and on commit it writes it to the
> database.
>
> On Fri, Feb 7, 2020 at 9:04 AM Michael Merickel 
> wrote:
> >
> > pyramid_tm does have a configurable activate_hook that you can use to
> turn off the TM for certain endpoints. You would need to coordinate that
> with something else that managed the transaction. If I were doing it myself
> I'd probably configure my activate hook to return true/false based on an
> custom attribute on the request like `request.activate_tm`. I'd then set
> this to False in a tween mounted over pyramid_tm. The reason I'd do it this
> way is because if you're turning off pyramid_tm then you probably still
> need to do something to manage the connection - and a tween would be a good
> spot to do this.
> >
> > That being said, I'd ask why you would want to turn it off instead of
> just committing multiple times? You just need to use `request.tm.commit()`,
> `request.tm.abort()` and `request.tm.begin()` instead of
> `request.dbsession.commit()`. Make sure to always call begin after
> commit/abort and things should work fine - whether it's a good idea or not
> in general (see Mikko's comments) is a separate question.
> >
> > - Michael
> >
> > On Thu, Feb 6, 2020 at 7:03 PM Jonathan Vanasco 
> wrote:
> >>
> >> one of my apps uses pyramid_tm, but has reached a point where one
> (possibly two) views (out of dozens) need to make explicit commits with the
> SqlAlchemy session.
> >>
> >> does anyone know if it's currently possible to handle this?  i have a
> few apps that only integrate pyramid_tm on a small subset of views, but i
> haven't tried to bypass it on a single view in many years.  I'm trying to
> avoid a secondary database connection, because this app can support a
> sqlite database and I've had problems with too many connections and sqlite
> in the past.
> >>
> >> --
> >> You received this message because you are subscribed to the Google
> Groups "pylons-discuss" group.
> >> To unsubscribe from this group and stop receiving emails from it, send
> an email to pylons-discuss+unsubscr...@googlegroups.com.
> >> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pylons-discuss/c3d3f561-8bcc-4c20-b1e8-f2aa147483f0%40googlegroups.com
> .
> >
> >
> >
> > --
> >
> > Michael
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups "pylons-discuss" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to pylons-discuss+unsubscr...@googlegroups.com.
> > To view this discussion on the web visit
> https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwENeR4239BK2cNYTudNn28-uiDTkWQX%3D4GhkqgwrwNG_g%40mail.gmail.com
> .
>
>
>
> --
> Mike Orr 
>
> --
> You received this message because you are subscribed to the Google Groups
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pylons-discuss/CAH9f%3Duq890OGO-xVWqx0Dcu_kBeoNqBb%3DfbiwnWZ6xZAUZRz1g%40mail.gmail.com
> .
>
-- 

Michael

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwEJqHxFqCeOBa7%3D2fv1s%2BrybbY8npvWPLfpprqGKCcaLg%40mail.gmail.com.

Re: [pylons-discuss] pyramid_tm - is there an acceptable way to unjoin the transaction and use explicit commits?

[Michael Merickel]

pyramid_tm does have a configurable activate_hook that you can use to turn
off the TM for certain endpoints. You would need to coordinate that with
something else that managed the transaction. If I were doing it myself I'd
probably configure my activate hook to return true/false based on an custom
attribute on the request like `request.activate_tm`. I'd then set this to
False in a tween mounted over pyramid_tm. The reason I'd do it this way is
because if you're turning off pyramid_tm then you probably still need to do
something to manage the connection - and a tween would be a good spot to do
this.

That being said, I'd ask why you would want to turn it off instead of just
committing multiple times? You just need to use `request.tm.commit()`,
`request.tm.abort()` and `request.tm.begin()` instead of
`request.dbsession.commit()`. Make sure to always call begin after
commit/abort and things should work fine - whether it's a good idea or not
in general (see Mikko's comments) is a separate question.

- Michael

On Thu, Feb 6, 2020 at 7:03 PM Jonathan Vanasco 
wrote:

> one of my apps uses pyramid_tm, but has reached a point where one
> (possibly two) views (out of dozens) need to make explicit commits with the
> SqlAlchemy session.
>
> does anyone know if it's currently possible to handle this?  i have a few
> apps that only integrate pyramid_tm on a small subset of views, but i
> haven't tried to bypass it on a single view in many years.  I'm trying to
> avoid a secondary database connection, because this app can support a
> sqlite database and I've had problems with too many connections and sqlite
> in the past.
>
> --
> You received this message because you are subscribed to the Google Groups
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pylons-discuss/c3d3f561-8bcc-4c20-b1e8-f2aa147483f0%40googlegroups.com
> 
> .
>


-- 

Michael

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwENeR4239BK2cNYTudNn28-uiDTkWQX%3D4GhkqgwrwNG_g%40mail.gmail.com.

Re: [pylons-discuss] Re: tracking down a test issue - ResourceWarning: unclosed file <_io.BufferedRandom

[Michael Merickel]

If you're opening files you should use the context manager - for example:

with request.POST['file'] as field:
data = field.file.read()

On Tue, Feb 4, 2020 at 3:53 PM Jonathan Vanasco 
wrote:

> After even more testing (half my day!)...
>
> My app needed an `add_finished_callback` to close any
> `webob.compat.cgi_FieldStorage` objects in forms.
>
> --
> You received this message because you are subscribed to the Google Groups
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pylons-discuss/b87e949c-62f7-43b9-a609-d78bd8ea6015%40googlegroups.com
> 
> .
>


-- 

Michael

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwH6aQnp%2Br-cugEpx3N4J0D8nekKdqhgavTYyTGQQ_YAtQ%40mail.gmail.com.

Re: [pylons-discuss] upgraded to 1.4.2 and requests got stuck (load 1.0)

[Michael Merickel]

I've been using 1.4.2 in production on heroku with 5 Standard-1X dynos and
have not observed this issue across a decent amount of traffic. That being
said we'll probably need more info to go on than your observations so far
to reproduce or turn it into any sort of bugfix. I'm not really sure what
questions to ask to diagnose what's going on other than finding a raw
request payload that triggers an error in the parser. The diff on 1.4.2 is
quite tiny.

- Michael

On Thu, Jan 9, 2020 at 10:22 AM 'Peter Lada' via pylons-discuss <
pylons-discuss@googlegroups.com> wrote:

> Github dependabot has opened a PR for me to upgrade to 1.4.2 (thanks for
> the release), and I merged it yesterday around 1730.
>
> At around 1930 one of the 6 dynos (heroku, 1X instance, single CPU, 0.5GB
> RAM) has come to get pegged at 1.0 load and timed out every subsequent
> request (heroku router cuts connection after 30s).
>
> At 1945 it happened to another dyno.
>
> At around 2000 I restarted the dynos and the problem got rectified,
> probably temporarily.
>
> I've reverted to 1.4.1 and the issue has not surfaced since (12 plus
> hours).
>
> Has anyone else used 1.4.2 in production? Any issues?
>
> Sadly I cannot provide more info, beyond the 1,5,15-minute load avg graphs
> as the logs just show timed out requests and no other info.
>
> --peter
> Formsort.com
>
> --
> You received this message because you are subscribed to the Google Groups
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pylons-discuss/54aa81bf-b935-4afc-b71c-f52d1fb15516%40googlegroups.com
> 
> .
>


-- 

Michael

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwGZN1qYBHn%3D_OCr-uBWo-aOdT9Ei0DKH1TqT7RAbz5ssw%40mail.gmail.com.

Re: [pylons-discuss] Confused accessing db sessions

[Michael Merickel]

This is a super common complaint from people and it doesn't have a single
definitive solution. There are a couple handy tricks:

1. If your classmethod receives any managed objects, you can get a
reference to the session from the object itself. For example
``sqlalchemy.orm.object_session(user)``.

2. The easy answer when the above doesn't work is to pass the dbsession
into the class method.

After this - if still not satisfied - are some hacky things you can do:

1. Stop using classmethod approaches and instead put in a real service
layer. This has tons of benefits, see something like pyramid_services for
more information about this. This approach will allow you to bind the whole
service to a dbsession, and no longer need to pass it around at all.

2. Define a helper to grab the session from the request threadlocal.
Something like ``get_current_dbsession = lambda:
get_current_request().dbsession``. Then import that and use it via
``dbsession = get_current_dbsession()``.

The threadlocal approach above has some downsides but is still better than
using a scoped session. Allowing the request to seep into and infect your
data model is a little wonky and makes your model code less reusable
outside of Pyramid. Even if you are okay with that, something like
``get_current_dbsession().query()`` is going to be better than
``scoped_session.query()`` because it's more explicitly a value that
changes over time versus the magic scoped session object proxy.

In my own apps I use all the approaches mentioned above except the
threadlocal one.

- Michael

On Mon, Jan 6, 2020 at 2:37 PM Kate Boelhauf  wrote:

> A little background - I'm updating my entire site and used a cookiecutter
> as referenced in Pyramids docs to do so. I am using sqlalchemy.
>
> It has a db session as a request method. This works perfectly for me in
> all of my views. However, I have class methods in my models that need to
> access a session.  I can't figure out a way to access the session that the
> cookiecutter set up in models.py so I reverted to creating a separate
> session using
> scoped_session(sessionmaker(extension=ZopeTransactionExtension())) and
> importing it into my model and using the request session in my views.
>
> This just seems wrong - am I approaching this wrong?
>
> Below is the model.py file that the cookiecutter created:
>
> from sqlalchemy import engine_from_config
> from sqlalchemy.orm import sessionmaker
> from sqlalchemy.orm import configure_mappers
> import zope.sqlalchemy
>
> # import or define all models here to ensure they are attached to the
> # Base.metadata prior to any initialization routines
> from .mymodel import MyModel  # flake8: noqa
>
> # run configure_mappers after defining all of the models to ensure
> # all relationships can be setup
> configure_mappers()
>
>
> def get_engine(settings, prefix='sqlalchemy.'):
> return engine_from_config(settings, prefix)
>
>
> def get_session_factory(engine):
> factory = sessionmaker()
> factory.configure(bind=engine)
> return factory
>
>
> def get_tm_session(session_factory, transaction_manager):
> """
> Get a ``sqlalchemy.orm.Session`` instance backed by a transaction.
>
> This function will hook the session to the transaction manager which
> will take care of committing any changes.
>
> - When using pyramid_tm it will automatically be committed or aborted
>   depending on whether an exception is raised.
>
> - When using scripts you should wrap the session in a manager yourself.
>   For example::
>
>   import transaction
>
>   engine = get_engine(settings)
>   session_factory = get_session_factory(engine)
>   with transaction.manager:
>   dbsession = get_tm_session(session_factory, transaction.manager)
>
> """
> dbsession = session_factory()
> zope.sqlalchemy.register(
> dbsession, transaction_manager=transaction_manager)
> return dbsession
>
>
> def includeme(config):
> """
> Initialize the model for a Pyramid app.
>
> Activate this setup using ``config.include('tutorial.models')``.
>
> """
> settings = config.get_settings()
> settings['tm.manager_hook'] = 'pyramid_tm.explicit_manager'
>
> # use pyramid_tm to hook the transaction lifecycle to the request
> config.include('pyramid_tm')
>
> # use pyramid_retry to retry a request when transient exceptions occur
> config.include('pyramid_retry')
>
> session_factory = get_session_factory(get_engine(settings))
> config.registry['dbsession_factory'] = session_factory
>
> # make request.dbsession available for use in Pyramid
> config.add_request_method(
> # r.tm is the transaction manager used by pyramid_tm
> lambda r: get_tm_session(session_factory, r.tm),
> 'dbsession',
> reify=True
> )
>
>
>
>
>
>
>
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "pylons-discuss" group.
> To unsubscr

[pylons-discuss] pyramid_tm 2.4 has been released

[Michael Merickel]

pyramid_tm 2.4 has been released.

This is a very minor release in preparation for Pyramid 2.0 but contains a
handy hook for testing and some info on how to override pyramid_tm in a
functional test suite.

PyP/Changelog: https://pypi.org/project/pyramid_tm/2.4/

Documentation:
https://docs.pylonsproject.org/projects/pyramid_tm/en/latest/

You can install it via PyPI:

  pip install pyramid_tm==2.4

Enjoy, and please report any issues you find to the issue tracker at
https://github.com/Pylons/pyramid_tm/issues

Thanks!

- Michael

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwF7uTcBLAMDoRyRhHN_w2G9m2JX_GPmRsWSg624%3DeEXoA%40mail.gmail.com.

Re: [pylons-discuss] question regarding url dispatch with very similar routes

[Michael Merickel]

You didn't make it completely clear what you're doing with your view_config
but I assume it's something to do with the match_param predicate after your
factory updates the matchdict. That's fine, and you could use it to
annotate the request with another attribute and then write a custom
predicate for your views to use instead of the match_param. Think something
like request.user_focus = 'a_username',
and @view_config(route_name='user_focus', user_type='a_username') which
will work well if they are mutually exclusive.

On Wed, Dec 18, 2019 at 7:39 PM Jonathan Vanasco 
wrote:

> I hope someone here can suggest a good way of handling this situation.
>
> We have an application that supports a given user profile view via two or
> more possible URL patterns:
>
>"/users/{username_regex}"
>"/users/{userid_regex}"
>
> To handle this within one route declaration, a `factory` is used:
>
> config.add_route("user_focus", "/users/{user_identifier}",
> factory=factory__useraccount)
>
> The factory does regex on the `user_identifier` placeholder, and updates
> the request matchdict with an extracted `user_name` or `user_id`. this
> works perfect and allows us to avoid this:
>
> config.add_route("user_focus_a", "/users/{username_identifier}")
> config.add_route("user_focus_b", "/users/{userid_identifier}")
>
> we also avoid using view_config to register 2 routes onto each callable
>
>
> We've added some new functionality, and I'm having a problem figuring out
> the best way to implement it.
>
> users matching a particular new format of the `user_identifier`
> placeholder need to respond to another route
>
> there are essentially 4 potential patterns
>
> config.add_route("usertype_a_focus_username",
> "/users/{username_A_regex}")
> config.add_route("usertype_a_focus_userid",
> "/users/{userid_A_regex}")
> config.add_route("usertype_b_focus_username",
> "/users/{username_B_regex}")
> config.add_route("usertype_b_focus_userid",
> "/users/{userid_B_regex}")
>
> ideally i'd love to simplify this into 2 routes, as they map to 2 views.
> something like:
>
> config.add_route("usertype_a_focus",
> "/users/{usertype_a_identifier}", ...)
> config.add_route("usertype_b_focus",
> "/users/{usertype_b_identifier}", ...)
>
> the problem with the approach is that `usertype_a_focus` grabs the route,
> and the `factory` (to my knowledge) can only raise a not-found - not fail
> the lookup and continue on to future patterns - so `usertype_b_focus` never
> runs.
>
> it looks like I could do this by running a regex in the placeholder, but
> then I'm running it again in the factory to alter the matchdict. that will
> also require duplicating an already complied regex into the pattern syntax
> for the route.  the app is large, so I'd like to avoid having to keep two
> different regex patterns in sync with one another.
>
> does anyone know if it is possible to better use factory, or implement
> some other amazing Pyramid utility like predicates or something else in
> this situation?
>
> thank you all in advance.
>
> --
> You received this message because you are subscribed to the Google Groups
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pylons-discuss/d603ef95-b391-43e2-8c2f-82132fc1f952%40googlegroups.com
> 
> .
>


-- 

Michael

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwEV1E2y15OoEbhR6-QSXAbJT8x--N4H0STB4qjWyhiJjA%40mail.gmail.com.

Re: [pylons-discuss] GET, HEAD, the Guillotine middleware extension

[Michael Merickel]

It's worth pointing out that while WSGI apps in general have that issue,
webob (and thus pyramid apps) automatically handle this.

1) webob automatically truncates the body on a head request.
2) pyramid automatically configures any view that takes
`request_method='GET'` to also take HEAD and documents that it does that.

- Michael

On Sun, Oct 20, 2019 at 4:46 PM Bert JW Regeer  wrote:

> If your WSGI application for instance does not natively support HEAD
> requests, because it only handles GET requests, you can use this middleware
> to do the rewrite, have your WSGI app return the whole response, and
> Guillotine will then return a body less response to the remote client.
>
> Bert
>
> On Oct 18, 2019, at 15:47, Jens Troeger  wrote:
>
> Hello,
>
> Today I stumbled upon a somewhat dated Pylons project called Guillotine
> , a WSGI middleware
> 
> extension. It seems to have only one purpose: rewrite the request method of
> HEAD requests into GET.
>
> Now I am curious about the *why*. A HEAD request is, as per documentation
> , pretty
> much a GET without body. So what’s the purpose then of this particular
> extension and the rewrite?
>
> Much thanks,
> Jens
>
> --
> You received this message because you are subscribed to the Google Groups
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pylons-discuss/cd9466d9-47ce-410f-9b36-5a47417cf8f2%40googlegroups.com
> 
> .
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pylons-discuss/F107895E-EF6E-4FEA-8971-AC414FADC28A%400x58.com
> 
> .
>


-- 

Michael

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwHaxnMyu%3D%2BcS6rzGWiGcb0L83i-pdCrCFvD-94WYHz0fg%40mail.gmail.com.

Re: [pylons-discuss] waitress - customer headers

[Michael Merickel]

If you can see the header in a flask app served with gunicorn but not in
the same app using waitress then the answer is likely related to the actual
content of the request and header so you probably need to provide a
reproducible example. I think the current dummy example you're showing is
unlikely to be different. Obviously it's not the case that every header is
ignored on waitress and works on gunicorn, it's related to the request
content.

On Tue, Oct 15, 2019 at 4:37 PM 'Roman S' via pylons-discuss <
pylons-discuss@googlegroups.com> wrote:

> hi all,
>
> how do I allow a custom header in the request (retrieving it in a simple
> flask app)?
>
>
> example:
>
>
> curl -X GET --header 'Accept: application/json' --header 'SESSION_DATA: 
> ...c2Vy...' http://localhost:8080/endpoint
>
>
> waitress just ignores the header SESSION_DATA, whereby gunicorn e.g.
> allows it.
>
>
> *waitress==1.3.1*
>
> --
> You received this message because you are subscribed to the Google Groups
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pylons-discuss/c45a4c53-4d8d-41db-952e-f61914a01c76%40googlegroups.com
> 
> .
>


-- 

Michael

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwGruj-EWfV0FE5M-3bFgM5_S-8oK%3DdC3b%3Dthd9cTMqQsQ%40mail.gmail.com.

Re: [pylons-discuss] Adding functionality to the core

[Michael Merickel]

My personal feeling is most things that can be done out of core should be
done out of core, but if new hooks or functionality is needed in core to
accomplish that then we should consider adding it. The only exceptions are
things that seem clear to affect a large percentage of the community or
things that are implementing standards.

CORS is a great example of something that I *think* view derivers and a
custom module should be able to do completely outside of core, but in the
vein of CSRF are something that might benefit from being shipped with
Pyramid if the implementation was deemed useful for a large percentage of
use cases.

- Michael


On Thu, Sep 19, 2019 at 12:38 PM Theron Luhn  wrote:

> Engaging with https://github.com/Pylons/pyramid/issues/3452 got me
> thinking about functionality I’ve wished was shipped with Pyramid.  CORS
> for example, or a base traversal resource.
>
> So I was wondering what’s the bar to add additional stuff to the project?
> Not an official policy or anything, just the general feeling of the
> community.
>
> — Theron
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pylons-discuss/AD1DB163-D691-4EF6-B8CA-A19F3D76E245%40luhn.com
> 
> .
>


-- 

Michael

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwExSZwM-kpVQUfYkFWDgeV%2B%3DZv%3DV%3DOFXMeLp-_T%2BrbOuA%40mail.gmail.com.

Re: [pylons-discuss] Is there a catch-all “predicate mismatch” view?

[Michael Merickel]

pyramid.exceptions.PredicateMismatch is a subclass of HTTPNotFound and thus
a generic 404 handler (notfound_view_config) will catch it. You can define
an exception_view_config specifically for PredicateMismatch if you wish to
be more specific but most people would let it fall through to the 404
handler which you apparently are not defining.

On Thu, Aug 8, 2019 at 1:16 AM Jens Troeger  wrote:

> Hello,
>
> During testing we just noticed that existing views whose predicate
> mismatched when making a request return an error like so:
>
> 404 Not Found
> The resource could not be found.
>
> predicate mismatch for view ... (request_method = POST)
>
> Is there a way to define a view that matches any and all of such
> mismatches to avoid returning too much information to the client? At least
> for production…
>
> Cheers!
> Jens
>
> --
> You received this message because you are subscribed to the Google Groups
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pylons-discuss/78b7b977-8c47-481b-8488-21c223888a26%40googlegroups.com
> 
> .
>


-- 

Michael

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwGKuU5nF%3D8edm6vEzUsztQDktVZ5v3Gq2EsnSFvRQq9zg%40mail.gmail.com.

Re: [pylons-discuss] Docs page down?

[Michael Merickel]

Thanks Brian, it seems to be related to RTD being down right now.

https://twitter.com/readthedocs/status/1156337277640908801

On Tue, Jul 30, 2019 at 7:07 PM Brian Connor  wrote:

> Hi all.  Thanks for the great docs and tutorials.  I've been enjoying
> learning the framework, though I've been getting 504's on the docs page for
> the last hour or so.  Here for example: 
> https://docs.pylonsproject.org/projects/pyramid/en/latest/quick_tutorial/templating.html
>
>
> I'm happy to post someplace else if it's more appropriate but I thought
> I'd start here and hopefully get some visibility.
>
> Thanks!
>
> --
> You received this message because you are subscribed to the Google Groups
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pylons-discuss/e5d6d47c-bda5-45b2-b7c2-d449ffb221f1%40googlegroups.com
> 
> .
>


-- 

Michael

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwGATHngMcZYACCQ%2BiGJMW4gSxjga1nQHi7NQK7%3DrMF9uQ%40mail.gmail.com.

Re: [pylons-discuss] Protecting a single view with permission based on context

[Michael Merickel]

Does the permission actually need to change or can the context object just
return an appropriate ACL based on its state? If you can have the context
object be smarter then problem solved with a single permission. Otherwise
yes you can certainly just handle it imperatively in the view code.

On Thu, Jul 25, 2019 at 8:37 AM Thierry Florac  wrote:

> Hi,
> In an application using ZODB traversal, I need to create a view whose
> required permission depends on the state of the "context" object to which
> the view is applied.
> Can I just create an "un-protected" view (without static permission) and
> check the permission in the view initialization code (and raise an
> HTTPForbidden exception if the required permission is missing), or is there
> a more elegant way to handle such a use case?
> Best regards,
> Thierry
> --
>   https://www.ulthar.net -- http://pyams.readthedocs.io
>
> --
> You received this message because you are subscribed to the Google Groups
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWAChxFOKJrLUs8L%3DRUz6r6u9xQn2fwCFSqZTBA1ehK%2BAQ%40mail.gmail.com
> 
> .
>


-- 

Michael

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwHTU4H2nKSm7b3j4SNA2T1bG8AT4Yhnx%2BjcQACA_HHb3g%40mail.gmail.com.

Re: [pylons-discuss] waitress: can not see the 4 default threads

[Michael Merickel]

processes != threads, it looks like you're thinking that waitress is using
processes based on your suggestion that you think os.getpid() should return
something different, or that you can't see them being started.

- Michael

On Thu, Jul 18, 2019 at 2:54 PM Philip May  wrote:

> In the waitress documentation it is said that waitress starts 4 threads to
> handle the requests. When I start a waitress server to serve a flask rest
> API I can not see 4 threads being started (MacOS). Also when I output the
> ID of the threads (os.getpid()) I always get the same ID. What can be the
> reason here?
> Thanks
> Philip
>
> --
> You received this message because you are subscribed to the Google Groups
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To post to this group, send email to pylons-discuss@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pylons-discuss/a98dff6f-a3bb-48e3-a19f-7723ce729fb7%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>


-- 

- Michael

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To post to this group, send email to pylons-discuss@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwF%2BBpgTcJeUEAqsv4S1txuocys%3DL%2Bj8qarWXNg9VXXrmw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [pylons-discuss] Velruse… or what else?

[Michael Merickel]

On Fri, Jun 7, 2019 at 2:38 AM Jens Troeger  wrote:

> The recent OAuth thread
> 
> recommends requests-oauthlib
> … Can you recommend any
> particular package, is the that one good?
>

I've used requests-oauthlib successfully in the past. I'd recommend it. I
never found much reason to use it during the process of acquiring an access
token, but you can. Once you have an access token and possibly a refresh
token, it serves as a nice wrapper around a requests.Session to handle
adding the appropriate Authorization header as well as assisting in token
expiration and using the refresh token to get new access tokens.

- Michael

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To post to this group, send email to pylons-discuss@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwFnYSgNLW%3Dfac606p86cQLo_QAF8iZ5OgQ%2Bot-uth5MCw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [pylons-discuss] Velruse… or what else?

[Michael Merickel]

On Mon, Jun 3, 2019 at 8:59 PM Jens Troeger  wrote:

> I’ve been using Velruse  quite
> happily for a while now, although it’s been stale for a few years.
> MichaelM  was somewhat active on that
> project too, and he still seems to be around working on Pylons in general.
>
> However, a few things could need updating with Velruse (see its issues
> ) and now that Apple has
> released a sign-in API, it would be nice to add that to Velruse as well.
>
> So I was wondering: are there plans to continue working on Velruse, maybe
> integrating it closer into Pyramid? Are there more recent alternatives? Are
> there people here who are using Velruse?
>

Yep I'm active on other Pylons projects but I haven't worked on velruse in
years.

I put a significant amount of time into reworking velruse so that it
integrated very well into Pyramid and I think that work was a success and
shows a good pattern for how to do it. The problem is that the velruse test
suite (and anyone else who tries to consume arbitrary third party
providers) is a damn nightmare to run. It required setting up accounts on
all of the services, including some that required google translate to even
get through, and then constantly tweaking the slow selenium tests. It's
unavoidable work but it burned me out. On top of that, it's actually really
easy (imo) to consume OAuth 2.x + OpenID Connect apis directly. Most
bindings for velruse end up being just a very thin wrapper around a small
amount of code you can easily write yourself. It basically just involves
implementing a redirection endpoint that the provider will send the browser
to once the user has granted access, and from there taking the token and
querying whatever you want from the provider's api. OpenID Connect has
helped a lot in standardizing the type of data you'd want to receive -
something velruse was trying to standardize before that standard existed.

I don't have any plans to do anything with velruse in the future and
ultimately it's Ben's project and it'd be up to him what he wants to do
with it.

- Michael

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To post to this group, send email to pylons-discuss@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwEH7vgXs9TjA1Za-hDKpBBiJcrzd69hrTJU-Wk1okj1rg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [pylons-discuss] Questions about DB engine setup

[Michael Merickel]

The session and sessionmaker are only necessary if you want to hook into
pyramid_tm, because zope.sqlalchemy requires a session object and won't
work with a simple connection. That being said, I would want to such that
pyramid_tm handles the commits for you. That being said, you might abstract
that a bit.

- You'll probably want to adjust z.sqla.register() to pass in
initial_state='changed' unless you want to always be calling
z.sqla.mark_changed(session). This is a very minor optimization z.sqla does
with the ORM where it knows which operations are mutations versus selects
and won't work with core.

- You might want to abstract your api so request.dbconn or something is the
connection, and just leave request.dbsession in the background hooked to
z.sqla to deal with commit/rollback.

- pool_pre_ping is done at the connection pool / engine level and will work
with core or the orm.

- pool_reset_on_return is just a last resort when you're done with a
connection, normal pyramid_tm usage will already close the txn such that
when it's returned to the pool no cleanup needs to be done. "rollback" (the
default) should be fine.

- The ids are probably the same because you're checking the id of the
engine (which is actually the connection pool usually). There is usually
only one engine instance, created via create_engine() or
engine_from_config() and from there separate connections are created.

Hope that helps,

- Michael

On Sat, Apr 27, 2019 at 9:34 AM Zsolt Ero  wrote:

> I'm trying to set up an API which would use SQLAlchemy Core (not ORM) +
> PostgreSQL. The server is a Google managed PostgreSQL instance, on external
> IP.
>
> I have a couple of questions. Since I needed to manually add SSL
> certificates as connect_args to create_engine + some additional arguments,
> I'm using create_engine(). My questions are related to this
>
> 1. Does this look ok?
>
> import zope.sqlalchemy
> from populus_lib.config import in_worker, pg_certs, pg_url
> from sqlalchemy import create_engine
> from sqlalchemy.orm import sessionmaker
>
> def get_session_factory(engine):
> factory = sessionmaker()
> factory.configure(bind=engine)
> return factory
>
> def get_tm_session(session_factory, transaction_manager):
> dbsession = session_factory()
> zope.sqlalchemy.register(dbsession, transaction_manager=
> transaction_manager)
> return dbsession
>
> def includeme(config):
> settings = config.get_settings()
> settings['tm.manager_hook'] = 'pyramid_tm.explicit_manager'
>
> config.include('pyramid_tm')
>
> engine = create_engine(
> pg_url,
> connect_args=pg_certs,
> pool_pre_ping=True,
> pool_reset_on_return='rollback' if in_worker else None,  #
> in_worker means production
> )
>
> session_factory = get_session_factory(engine)
> config.registry['dbsession_factory'] = session_factory
>
> config.add_request_method(
> lambda r: get_tm_session(session_factory, r.tm),
> 'dbsession',
> reify=True,
> )
>
>
> 2. Since I'm not using ORM, but core only, do I need from sqlalchemy.orm
> import sessionmaker?
>
> 3. Is pool_pre_ping supported with Pyramid's way of session/transaction
> handling? I want to be sure that external server disconnects/reconnects are
> handled automatically and I think using pool_pre_ping is the best for
> this.
>
> 4. Isn't pool_reset_on_return conflicting pyramid_tm / session handling?
> I only need to use this in development, since the SQL server is in US and
> I'm in Europe and without this settings SQLAlchemy has a huge overhead on
> each query, like 300 ms.
>
> 5, Finally, what's puzzling me is that if I create a view like this:
> def ping(request):
> print(id(request.dbsession.connection().engine))
> sleep(60)
>
> And I run this via curl from two concurrent terminal windows, I get equal
> ids in pserve / Waitress, while I get different ids with gunicorn defaults
> (which I believe is multiprocessing).
>
> As I understand each worker needs it's own engine instance, don't they? I
> think the gunicorn behaviour is good, but I'm puzzled by the
> pserve/Waitress behaviour. Is this by design?
>
> --
> You received this message because you are subscribed to the Google Groups
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To post to this group, send email to pylons-discuss@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pylons-discuss/e6dc1571-5b50-4b77-be1e-62c9c5964f4d%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiv

[pylons-discuss] Pyramid 1.10.4 released

[Michael Merickel]

Hey folks, I just released 1.10.4. This simply fixes a performance
regression introduced in 1.10.3 related to the view_config decorator.

Check the "What's New in Pyramid 1.10" for more information about 1.10:
https://docs.pylonsproject.org/projects/pyramid/en/1.10-branch/whatsnew-1.10.html

As always, please submit any issues on our issue tracker:
https://github.com/Pylons/pyramid/issues

Thanks!

- Pyramid core developers

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To post to this group, send email to pylons-discuss@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwHWuJ1KrPknNg%2BcAX9eiSi3WccN4J%3DJYSz20QvCWd%2BoBw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[pylons-discuss] Pyramid 1.10.3 released

[Michael Merickel]

Hey folks, I just released 1.10.3. The changes are small but it does add
the ignore_files setting to [pserve] which some people may appreciate.

Check the "What's New in Pyramid 1.10" for more information about 1.10.2:
https://docs.pylonsproject.org/projects/pyramid/en/1.10-branch/whatsnew-1.10.html

As always, please submit any issues on our issue tracker:
https://github.com/Pylons/pyramid/issues

Thanks!

- Pyramid core developers

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To post to this group, send email to pylons-discuss@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwF2_ZnbF%3DeajBuZaPJcAdA302zXB%3DcBxRN333U5_M0vHw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [pylons-discuss] a library for building config stores

[Michael Merickel]

Pyramid uses plaster to load files and hasn't depended on ini directly
since 2017. See
https://docs.pylonsproject.org/projects/pyramid/en/1.9-branch/whatsnew-1.9.html#major-feature-additions.
Granted all tutorials etc still use plaster_pastedeploy as the primary
connector, but anyone is free to replace it with their own binding to any
format they wish.

On Thu, Apr 4, 2019 at 1:12 PM Chris Withers  wrote:

> Hey Guys,
>
> I wanted to let Pyramid-y people know about a library I finally got
> documented and released:
>
> https://configurator.readthedocs.io/en/latest/
>
> Partly because you're some of the best software engineers I know, and
> I'd really value your feedback.
>
> Partly because I wonder if Pyramid could make use of configurator (I
> know, the name class is unfortunate...) for building the config object,
> rather than being locked to .ini-ness?
>
> Anyway, any feedback very gratefully received.
>
> cheers,
>
> Chris
>
> --
> You received this message because you are subscribed to the Google Groups
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To post to this group, send email to pylons-discuss@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pylons-discuss/d443543f-ef3e-c810-992e-d4bd113a32ea%40withers.org
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To post to this group, send email to pylons-discuss@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwHzg-oLLDM80XiTzXa4a7xduDfe1%2BJg8yp9Rxmo5Pmc9g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.