Re: [pylons-discuss] Relationship between requests and Zope transactions

[Michael Merickel]

>
> Added yet another one or two parameters (sigh) to functions, but it keeps
> things functional and without outside (global, thread-local) state.


This is where something like pyramid_services [1] really excels. It'll
drastically cut down on concerns like that when you adopt some form of
inversion-of-control pattern in your codebase.

[1] https://github.com/mmerickel/pyramid_services

On Fri, Sep 7, 2018 at 3:44 AM  wrote:

> Thank you Mike for the information, that all makes sense. I ended up with
> the same approach that SQLA takes: a “Session” object owned by a
> DataManager, joined to the Request’s TransactionManager. And then just pass
> a `dbsession` and `fnsession` and `jobsession` around to functions as
> needed.
>
> Added yet another one or two parameters (sigh) to functions, but it keeps
> things functional and without outside (global, thread-local) state.
>
> --
> You received this message because you are subscribed to the Google Groups
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To post to this group, send email to pylons-discuss@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pylons-discuss/148c7bec-2d75-4525-9aa4-3902a4a2f2cb%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To post to this group, send email to pylons-discuss@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwHgc8TOpYPcJdjyPWEtPgFhqbPrWPFPUFuEYT%3DhNjWZ0A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [pylons-discuss] Relationship between requests and Zope transactions

[Mike Orr]

functools.partal might help in some cases.
On Fri, Sep 7, 2018 at 1:44 AM  wrote:
>
> Thank you Mike for the information, that all makes sense. I ended up with the 
> same approach that SQLA takes: a “Session” object owned by a DataManager, 
> joined to the Request’s TransactionManager. And then just pass a `dbsession` 
> and `fnsession` and `jobsession` around to functions as needed.
>
> Added yet another one or two parameters (sigh) to functions, but it keeps 
> things functional and without outside (global, thread-local) state.
>
> --
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To post to this group, send email to pylons-discuss@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/148c7bec-2d75-4525-9aa4-3902a4a2f2cb%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



-- 
Mike Orr 

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To post to this group, send email to pylons-discuss@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/CAH9f%3Durcme78micgQE8hT5B%2B0QdVYHUAaLW5J6t2Jy32HOqsbQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [pylons-discuss] Relationship between requests and Zope transactions

[jens . troeger]

Thank you Mike for the information, that all makes sense. I ended up with 
the same approach that SQLA takes: a “Session” object owned by a 
DataManager, joined to the Request’s TransactionManager. And then just pass 
a `dbsession` and `fnsession` and `jobsession` around to functions as 
needed.

Added yet another one or two parameters (sigh) to functions, but it keeps 
things functional and without outside (global, thread-local) state.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To post to this group, send email to pylons-discuss@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/148c7bec-2d75-4525-9aa4-3902a4a2f2cb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [pylons-discuss] Relationship between requests and Zope transactions

[Jonathan Vanasco]

+1 on being explicit by passing around the request (or something similar). 
there are way too many edge cases when you try to compute the current 
request/transaction, and it's incredibly hard (a nightmare) to write tests 
that work correctly when you rely on the derived/computed methods.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To post to this group, send email to pylons-discuss@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/e90800f3-9b1b-48fa-91f5-2c4b9668104b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [pylons-discuss] Relationship between requests and Zope transactions

[Mike Orr]

On Sun, Sep 2, 2018 at 7:31 AM  wrote:
> If requests during their life cycle would derive from threading.local and 
> initialize such a storage area upon construction (e.g. after tween ingress) 
> and tear it down (e.g. after tween egress) then deeply nested function could 
> still access a request’s context without the need to pass that context around 
> in parameters.

This is thinking about it the wrong way around. The best solution is
to add a 'request' argument to those functions, or the specific
request subobject they need. Sometimes you can't because you have to
pass through a function that wasn't written for Pyramid, but that's
what the threadlocal functions are for.

Pylons had a magic request object and a magic "template variables"
object that the subfunctions could import; they were threadlocal proxy
stacks behind the scenes. That was to replicate the popular behavior
in Myghty (definitely) and Rails (I assume). But they caused a lot of
problems and were more trouble than they were worth. So it was
considered an improvement when we migrated to Pyramid which passes an
explicit request object instead.

Why would you want to go back to threadlocals? Even if it sometimes
uses a threadlocal function to find the Zope registry; I consider that
a flaw rather than a feature.Semantically the subobjects are
REQUEST-local, not THREAD-local, so why not treat them that way?

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To post to this group, send email to pylons-discuss@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/CAH9f%3DuqbPgFDfT5Tjod%3D73xNe3tdcHzFMwgmkjAnTw-6MyUqOQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [pylons-discuss] Relationship between requests and Zope transactions

[Michael Merickel]

>
> You are talking about these two lines of code
> 
>  in
> the cookiecutter’s model/__init__.py, correct?


Right.

Using that manager hook I can then derive from an explicit manager and
> extend with custom functionality—and that would be the recommended way of
> doing so? That may help me solve a question I just asked at the transaction
> Github repo (see issue #62
>  there).


Yes, if you don't have a consistent location in your code where you can
push/pop threadlocals then you're going to have a bad time. For something
like the `tm` you'd still be wise to use your own threadlocal manager that
is set to explicit mode versus using the transaction.manager shipped with
the transaction project. You'd probably make a manager and register an
ISynchronizer that pushed/popped the threadlocal so that it's only
available when a transaction is open.

Would it make sense to use a custom request factory to implement such a
> thread local approach? Are there other recommendations, considering the
> warnings in pyramid.threadlocal
> 
> ?


So I thought it would be useful if much/all of a request’s execution
> context would be available through the current thread local
>  memory.
> However, reading Why You Shouldn’t Abuse Thread Local
> 
>  (see
> the pyramid.threadlocal
> 
>  module)
> recommends against such an approach. Are there more recent and different
> recommendations?

Sure you could do this with a custom request factory. The recommendations
are not "out of date", they are about general programming paradigms that
you should avoid if you want to write testable/reusable code. Pyramid tries
to avoid *forcing* you to use threadlocals anywhere - but it's not going to
try to stop you if you want to do it.

On Sun, Sep 2, 2018 at 9:42 AM  wrote:

> Maybe some background… the reason why I asked the initial question is
> because a few (and a growing number of) request handlers are complex; that
> is, they call multiple function levels deep.
>
> Now request.dbsession, for example, is SQLA’s current DB session tied to
> the request’s transaction. To get access to it, functions keep handing that
> `dbsession` reference around. As we now begin to tie more data managers to
> the request’s transaction, handing their context around convolutes the
> parameters handed to functions.
>
> So I thought it would be useful if much/all of a request’s execution
> context would be available through the current thread local
>  memory.
> However, reading Why You Shouldn’t Abuse Thread Local
> 
>  (see
> the pyramid.threadlocal
> 
> module) recommends against such an approach. Are there more recent and
> different recommendations?
>
> --
> You received this message because you are subscribed to the Google Groups
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To post to this group, send email to pylons-discuss@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pylons-discuss/5b86bb49-8fb5-4ebe-ad2b-c3a4da52bad3%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To post to this group, send email to pylons-discuss@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwFh5QQ%2Biy94Vxmx1US1CG%3DFFhxCT7Hum-MAwhZ00UUwUw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [pylons-discuss] Relationship between requests and Zope transactions

[jens . troeger]

Maybe some background… the reason why I asked the initial question is 
because a few (and a growing number of) request handlers are complex; that 
is, they call multiple function levels deep.

Now request.dbsession, for example, is SQLA’s current DB session tied to 
the request’s transaction. To get access to it, functions keep handing that 
`dbsession` reference around. As we now begin to tie more data managers to 
the request’s transaction, handing their context around convolutes the 
parameters handed to functions.

So I thought it would be useful if much/all of a request’s execution 
context would be available through the current thread local 
 memory. 
However, reading Why You Shouldn’t Abuse Thread Local 

 (see 
the pyramid.threadlocal 

 
module) recommends against such an approach. Are there more recent and 
different recommendations?

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To post to this group, send email to pylons-discuss@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/5b86bb49-8fb5-4ebe-ad2b-c3a4da52bad3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [pylons-discuss] Relationship between requests and Zope transactions

[jens . troeger]

Maybe some background… the reason why I asked the initial question is 
because a few (and a growing number of) request handlers are complex; that 
is, they call multiple function levels deep.

Now request.dbsession, for example, is SQLA’s current DB session tied to 
the request’s transaction. To get access to it, functions keep handing that 
`dbsession` reference around. As we now begin to tie more data managers to 
the request’s transaction, handing their context around convolutes the 
parameters handed to functions.

So I thought it would be useful if much/all of a request’s execution 
context would be available through the current thread local 
 
memory. If requests during their life cycle 
 
would derive from threading.local and initialize such a storage area upon 
construction (e.g. after tween ingress) and tear it down (e.g. after tween 
egress) then deeply nested function could still access a request’s context 
*without* the need to pass that context around in parameters.

Would it make sense to use a custom request factory to implement such a 
thread local approach? Are there other recommendations, considering the 
warnings in pyramid.threadlocal 

?

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To post to this group, send email to pylons-discuss@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/080f5943-b2df-4d6c-8b49-ab5cfa7bbe6c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [pylons-discuss] Relationship between requests and Zope transactions

[jens . troeger]

Thanks Michael!

You are talking about these two lines of code 

 in 
the cookiecutter’s model/__init__.py, correct?

settings = config.get_settings()
settings['tm.manager_hook'] = 'pyramid_tm.explicit_manager'

That manager hook allocates its own explicit transaction manager (see code 
)
 
instead of using the transaction.manager default, as you’ve mentioned.

Using that manager hook I can then derive from an explicit manager and 
extend with custom functionality—and that would be the recommended way of 
doing so? That may help me solve a question I just asked at the transaction 
Github repo (see issue #62 
 there).


On Sunday, September 2, 2018 at 9:04:58 AM UTC+10, Michael Merickel wrote:
>
> Jens, by default your example is true, but it is not true in the 
> cookiecutter configuration. The value of request.tm is defined by the 
> tm.manager_hook setting and by default it is the threadlocal 
> transaction.manager. The cookiecutter overrides the hook (and I suggest you 
> do as well) to define a non-threadlocal manager configured in explicit=True 
> mode which will help weed out bugs accessing transactions after they have 
> been committed. I strongly suggest any code you write that needs the tm 
> should use request.tm, not transaction.manager - as the former is 
> configurable.
>
> - Michael
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To post to this group, send email to pylons-discuss@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/0f690504-4a20-42c7-a10e-0da915e602b3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [pylons-discuss] Relationship between requests and Zope transactions

[Michael Merickel]

Jens, by default your example is true, but it is not true in the
cookiecutter configuration. The value of request.tm is defined by the
tm.manager_hook setting and by default it is the threadlocal
transaction.manager. The cookiecutter overrides the hook (and I suggest you
do as well) to define a non-threadlocal manager configured in explicit=True
mode which will help weed out bugs accessing transactions after they have
been committed. I strongly suggest any code you write that needs the tm
should use request.tm, not transaction.manager - as the former is
configurable.

- Michael

On Sat, Sep 1, 2018 at 5:57 AM  wrote:

> Hi,
>
> According to the documentation here
> , “Pyramid
> 
> requests [to] join the active transaction
> 
> as provided by the Python transaction
>  package”. Looking at the
> transactions code, the default transaction manager used is a
> ThreadTransactionManager
> ,
> i.e. one transaction per Python execution thread.
>
> Now I did follow the Pyramid-SQLAlchemy-Cookiecutter
>  recipe where DB
> sessions join the request’s transaction. Considering that a single incoming
> request is handled in Pyramid by a single thread (correct?) is it safe to
> say that the following is true?
>
> import transaction
>
> @view_config(…)
> def some_view(request):
># Request's transaction manager is the thread's transaction manager.
>request.tm == transaction.manager
># Request's and thread's and tm's transaction is the same object.
>request.tm.get() == transaction.get()  # == transaction.manager.get()
>
> Thanks!
> Jens
>
> --
> You received this message because you are subscribed to the Google Groups
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pylons-discuss+unsubscr...@googlegroups.com.
> To post to this group, send email to pylons-discuss@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pylons-discuss/79da9d5a-91e9-483e-b96f-b5e54cc3f1a5%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To post to this group, send email to pylons-discuss@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwEpmYEvEjFx74MOBHA-iSM%2BC1KNtBynkxBTPM06ufvuyQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.