Re: Pyramid transactions and other request lifecycle events
On Wed, 2011-08-03 at 22:44 -0700, Mike Orr wrote: On Wed, Aug 3, 2011 at 12:34 PM, Matt Feifarek matt.feifa...@gmail.com wrote: On Wed, Aug 3, 2011 at 1:12 PM, Chris McDonough chr...@plope.com wrote: In fact, an argument could be made that if a transaction fails, the entire contents of the view callable should not execute... We don't know whether the transaction failed until the view callable is called. The definition of a transaction failing is either an exception is raised by view code or the view response code is 4XX/5XX. And by the time it fails, our view callable is finished (or has thrown an exception), so we can't branch code based on failure. But, back down to earth, doing something like setting a value in session when there is a transaction failure could lead to inconsistent state. Say a view callable is supposed to log someone in... but a database transaction fails. Meanwhile, our view callable has turned on authenticated in the session, but they didn't actually login successfully... It might be far-fetched, but I found it already ;-) Thankfully, not a security breach, just a bad UI message, but still... The transaction should fail when the update query is run, not when it commits. So you shouldn't set the 'authenticated' flag until the query finishes without error. If there's an unrelated write later in the transaction and it fails... then you have to consider whether you should have committed the login immediately rather than waiting for the transaction manager to commit it later. I think the syntax is 'transaction.commit()'. You might also question whether two unrelated writes should be in the same transaction anyway, because of this very problem. Sometimes you have to commit each part separately rather than waiting for the transaction manager to commit it all. Other times you have to use a separate connection not managed by the transaction manager (for instance, to log the request to a logging database, which should commit even if the main transaction aborts). FWIW, I think the only 100% sensible solution is to use a transactional storage for session data too. Partially committing by hand (or even completely committing by hand) at various likely points to work around the fact that a storage isn't transactional is always a guessing game that you'll eventually lose. But when all of the storages are transactional, and they're all managed by the transaction manager, you don't need to think very hard, you just write the code as if it will succeed, without paying attention to ordering or committing, and if there's any error, the storages will all just be rolled back then. - C -- You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com. To unsubscribe from this group, send email to pylons-discuss+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en.
Re: Pyramid transactions and other request lifecycle events
On Wed, Aug 3, 2011 at 11:06 PM, Chris McDonough chr...@plope.com wrote: On Wed, 2011-08-03 at 22:44 -0700, Mike Orr wrote: On Wed, Aug 3, 2011 at 12:34 PM, Matt Feifarek matt.feifa...@gmail.com wrote: On Wed, Aug 3, 2011 at 1:12 PM, Chris McDonough chr...@plope.com wrote: In fact, an argument could be made that if a transaction fails, the entire contents of the view callable should not execute... We don't know whether the transaction failed until the view callable is called. The definition of a transaction failing is either an exception is raised by view code or the view response code is 4XX/5XX. And by the time it fails, our view callable is finished (or has thrown an exception), so we can't branch code based on failure. But, back down to earth, doing something like setting a value in session when there is a transaction failure could lead to inconsistent state. Say a view callable is supposed to log someone in... but a database transaction fails. Meanwhile, our view callable has turned on authenticated in the session, but they didn't actually login successfully... It might be far-fetched, but I found it already ;-) Thankfully, not a security breach, just a bad UI message, but still... The transaction should fail when the update query is run, not when it commits. So you shouldn't set the 'authenticated' flag until the query finishes without error. If there's an unrelated write later in the transaction and it fails... then you have to consider whether you should have committed the login immediately rather than waiting for the transaction manager to commit it later. I think the syntax is 'transaction.commit()'. You might also question whether two unrelated writes should be in the same transaction anyway, because of this very problem. Sometimes you have to commit each part separately rather than waiting for the transaction manager to commit it all. Other times you have to use a separate connection not managed by the transaction manager (for instance, to log the request to a logging database, which should commit even if the main transaction aborts). FWIW, I think the only 100% sensible solution is to use a transactional storage for session data too. Partially committing by hand (or even completely committing by hand) at various likely points to work around the fact that a storage isn't transactional is always a guessing game that you'll eventually lose. But when all of the storages are transactional, and they're all managed by the transaction manager, you don't need to think very hard, you just write the code as if it will succeed, without paying attention to ordering or committing, and if there's any error, the storages will all just be rolled back then. I wasn't talking about sessions; I was talking about some random external resource. For sessions, well, Pylons doesn't have a transaction manager, and the issue of the database and session being inconsistent never came up. I think that's because Beaker has `session.save()` which acts like commit, and I probably tended to put my session writes at the very end of the view after the db transaction had succeeded. - C -- You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com. To unsubscribe from this group, send email to pylons-discuss+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en. -- Mike Orr sluggos...@gmail.com -- You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com. To unsubscribe from this group, send email to pylons-discuss+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en.
Switching to declarative configuration breaks paster
Hi, I am switching my application from imperative to declarative configuration using view_config. However, when I add config.scan() to __init.py__, my application refuses to start. If I do: 'paster serve config.ini ', I get a message saying 'no such option --reload'. If I just do 'paster serve config.ini' the error is: ConifgParser.NoSectionError. Removing config.scan() once again allows the application to run normally. Am I missing other steps or options? Thanks for any thoughts, Ben -- You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com. To unsubscribe from this group, send email to pylons-discuss+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en.
Pyramid: choose renderer at runtime
Is there a *suggested* way to set a view renderer at runtime in
Pyramid?
What I need is to render, in the same view, a basic form then, if some
more info are needed (this is known by first form data), render
another form with already entered info as hidden field and more fields
to be filled.
In a Pylons controller I could select which renderer use like this:
def my_view(self):
c.form = Form(request, BasicSchema)
if c.form.validate():
obj = form.bind(MYModel())
if **need more info**:
c.form = Form(AdvancedSchema, obj=obj)
if form.validate():
obj = form.bind(obj)
else:
return render('advanced_form.mako')
**persist obj someway**
redirect(...)
return render('basic_form.mako')
In my Pyramid app I have:
config.add_route('new', '/new')
config.add_view(my_view, route_name='new',
renderer='basic_form.mako')
and in `basic_form.mako` according to a passed value I render the form
one way or another:
%if step == 1:
#render basic form
%else:
#render advanced form
%endif
but it seems a bit ugly and cumbersome to me.
Any advice?
Thanks for your support
--
You received this message because you are subscribed to the Google Groups
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com.
To unsubscribe from this group, send email to
pylons-discuss+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/pylons-discuss?hl=en.
Re: Pyramid: choose renderer at runtime
This may help you:
http://stackoverflow.com/questions/6553569/in-pyramid-how-can-i-use-a-different-renderer-based-on-contents-of-context/6557728#6557728
On top of that, pyramid also supports:
request.override_renderer = ...
via
http://docs.pylonsproject.org/projects/pyramid/1.1/narr/renderers.html#overriding-a-renderer-at-runtime
On Thu, Aug 4, 2011 at 8:57 AM, neurino neur...@gmail.com wrote:
Is there a *suggested* way to set a view renderer at runtime in
Pyramid?
What I need is to render, in the same view, a basic form then, if some
more info are needed (this is known by first form data), render
another form with already entered info as hidden field and more fields
to be filled.
In a Pylons controller I could select which renderer use like this:
def my_view(self):
c.form = Form(request, BasicSchema)
if c.form.validate():
obj = form.bind(MYModel())
if **need more info**:
c.form = Form(AdvancedSchema, obj=obj)
if form.validate():
obj = form.bind(obj)
else:
return render('advanced_form.mako')
**persist obj someway**
redirect(...)
return render('basic_form.mako')
In my Pyramid app I have:
config.add_route('new', '/new')
config.add_view(my_view, route_name='new',
renderer='basic_form.mako')
and in `basic_form.mako` according to a passed value I render the form
one way or another:
%if step == 1:
#render basic form
%else:
#render advanced form
%endif
but it seems a bit ugly and cumbersome to me.
Any advice?
Thanks for your support
--
You received this message because you are subscribed to the Google Groups
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com.
To unsubscribe from this group, send email to
pylons-discuss+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/pylons-discuss?hl=en.
--
Michael
--
You received this message because you are subscribed to the Google Groups
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com.
To unsubscribe from this group, send email to
pylons-discuss+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/pylons-discuss?hl=en.
Re: Pyramid transactions and other request lifecycle events
On Thu, Aug 4, 2011 at 12:44 AM, Mike Orr sluggos...@gmail.com wrote: The transaction should fail when the update query is run, not when it commits. So you shouldn't set the 'authenticated' flag until the query finishes without error. Good advice, but not possible in this case; I'm using ZODB... the query is just foo.bar = 7 and isn't *really* done 'till commit, at which case the execution has left my code and is happening deep in the guts of WSGI middleware. I remember the .save() in the old Pylons session, and thought that made sense to revive, too. I kinda feel like there is still a larger question here, though... using transaction manager, code that is executed in a view callable doesn't *really* finish till after the users' function is over... so there's no chance of cleaning up if the transaction fails. I guess what is likely to happen is an exception, and in production, that would lead to a blank 500 error, which is the right thing. But because it's kinda unpredictable what has actually happened in the view callable, it's hard to write safe code... which means that the safe way is to manage ones own transactions when there is sensitive code, which means that maybe transaction manager itself is not a good fit? I hope not, because I like it. We can fix session storage to be transaction aware; maybe some kind soul will (maybe me)... but I can think of other dangerous code inconsistencies... say a database-dependent security check is required before a user is allowed to delete a file on the filesystem... we can't make file io transaction aware, so it would be possible to actually delete the file and have the database part fail. This might be unlikely because why would you WRITE to the database for a security check, but I'm working in the land of theory here. The very nature of transactions says that when you do something, it doesn't really happen 'till later. Surely we are not the first folks to discuss the ramifications of this in computer science... and surely there must be a pattern to make it work. I would assume it would be all or nothing, kinda like twisted. If you're in on transactions, you have to be ALL in. Pyramid likes callback functions, maybe we need to have some here? What if every view callable had three components, main, success_callback, fail_callback. Perhaps a handler class could do this? It would completely axe the nice view-callable metaphor that pyramid has going for it... just a nice function attached to HTTP. And even so, as far as I know, when repoze.tm2 executes, pyramid is no longer in charge, it's wsgi pipeline, so we couldn't get the execution back anyway. Ugh. -- You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com. To unsubscribe from this group, send email to pylons-discuss+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en.
Re: Pyramid transactions and other request lifecycle events
On Thu, 2011-08-04 at 11:15 -0500, Matt Feifarek wrote: On Thu, Aug 4, 2011 at 12:44 AM, Mike Orr sluggos...@gmail.com wrote: The transaction should fail when the update query is run, not when it commits. So you shouldn't set the 'authenticated' flag until the query finishes without error. Good advice, but not possible in this case; I'm using ZODB... the query is just foo.bar = 7 and isn't *really* done 'till commit, at which case the execution has left my code and is happening deep in the guts of WSGI middleware. I remember the .save() in the old Pylons session, and thought that made sense to revive, too. I kinda feel like there is still a larger question here, though... using transaction manager, code that is executed in a view callable doesn't *really* finish till after the users' function is over... so there's no chance of cleaning up if the transaction fails. Actually, there is: http://www.zodb.org/zodbbook/transactions.html#after-commit-hooks I guess what is likely to happen is an exception, and in production, that would lead to a blank 500 error, which is the right thing. But because it's kinda unpredictable what has actually happened in the view callable, it's hard to write safe code... which means that the safe way is to manage ones own transactions when there is sensitive code, which means that maybe transaction manager itself is not a good fit? I hope not, because I like it. We can fix session storage to be transaction aware; maybe some kind soul will (maybe me)... but I can think of other dangerous code inconsistencies... say a database-dependent security check is required before a user is allowed to delete a file on the filesystem... we can't make file io transaction aware, so it would be possible to actually delete the file and have the database part fail. This might be unlikely because why would you WRITE to the database for a security check, but I'm working in the land of theory here. We actually can and do make file io transactional: http://docs.repoze.org/filesafe/ The very nature of transactions says that when you do something, it doesn't really happen 'till later. Surely we are not the first folks to discuss the ramifications of this in computer science... and surely there must be a pattern to make it work. I would assume it would be all or nothing, kinda like twisted. If you're in on transactions, you have to be ALL in. Pyramid likes callback functions, maybe we need to have some here? What if every view callable had three components, main, success_callback, fail_callback. Perhaps a handler class could do this? It would completely axe the nice view-callable metaphor that pyramid has going for it... just a nice function attached to HTTP. And even so, as far as I know, when repoze.tm2 executes, pyramid is no longer in charge, it's wsgi pipeline, so we couldn't get the execution back anyway. Ugh. Is there an actual problem here that can't be solved by either making sessions transactional or by after-commit? - C -- You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com. To unsubscribe from this group, send email to pylons-discuss+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en.
Re: Switching to declarative configuration breaks paster
On Thu, Aug 4, 2011 at 6:27 AM, Benjamin Sims benjamins...@gmail.com wrote: Hi, I am switching my application from imperative to declarative configuration using view_config. However, when I add config.scan() to __init.py__, my application refuses to start. If I do: 'paster serve config.ini ', I get a message saying 'no such option --reload'. If I just do 'paster serve config.ini' the error is: ConifgParser.NoSectionError. Removing config.scan() once again allows the application to run normally. Am I missing other steps or options? That sounds really strange. What is the traceback? It must be happening after Paste has read the config file and is loading the application and running the main function, because that's the point where scan would be called. Do you have the scan call inside the main function and not at module level? Having it at module level could well do something weird; if something it expects to have happened hasn't yet. The second question is, is something in the application trying to parse the config file again? (That is, something called by main, or in any of your application modules at module level. You might do this to read a custom section for an application-specific purpose.) It sounds like it's reading the wrong file or is trying to read a nonexistent file. ConfigParser does have a misfeature where if you try to read a file that doesn't exist, it succeeds as if it were an empty file. That would then lead to a missing section error or missing option error. There's a third-party version of ConfigParser that fixes that, but I'm not sure if Paste's version (paste.deploy.something.NicerConfigParser) does. -- Mike Orr sluggos...@gmail.com -- You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com. To unsubscribe from this group, send email to pylons-discuss+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en.
Re: Pyramid transactions and other request lifecycle events
On Thu, Aug 4, 2011 at 9:15 AM, Matt Feifarek matt.feifa...@gmail.com wrote: On Thu, Aug 4, 2011 at 12:44 AM, Mike Orr sluggos...@gmail.com wrote: The transaction should fail when the update query is run, not when it commits. So you shouldn't set the 'authenticated' flag until the query finishes without error. Good advice, but not possible in this case; I'm using ZODB... [Slaps head.] I'm so used to SQLAlchemy I forgot that pyramid_tm originated in Zope/ZODB. I kinda feel like there is still a larger question here, though... using transaction manager, code that is executed in a view callable doesn't *really* finish till after the users' function is over... so there's no chance of cleaning up if the transaction fails. I guess what is likely to happen is an exception, and in production, that would lead to a blank 500 error, which is the right thing. But because it's kinda unpredictable what has actually happened in the view callable, it's hard to write safe code... which means that the safe way is to manage ones own transactions when there is sensitive code, which means that maybe transaction manager itself is not a good fit? I hope not, because I like it. We can fix session storage to be transaction aware; maybe some kind soul will (maybe me)... but I can think of other dangerous code inconsistencies... say a database-dependent security check is required before a user is allowed to delete a file on the filesystem... Well, again, what's a concrete case where this has been a problem? In sites where users upload files attached to database records, there's a possibility that the file is modified and the database record isn't, but I have an application that's been doing that for six years and this has never come up. Commit failures themselves are pretty rare , except for cases where the autoincrement-sequence has gotten out of sync, the data is too long for the field, or there's a Unicode error or unexpected null. And those are bugs in themselves that have to be fixed, so it's not surprising if there's collateral damage when they fail. -- Mike Orr sluggos...@gmail.com -- You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com. To unsubscribe from this group, send email to pylons-discuss+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en.
Re: Pyramid transactions and other request lifecycle events
On Thu, Aug 4, 2011 at 12:06 PM, Chris McDonough chr...@plope.com wrote: I kinda feel like there is still a larger question here, though... using transaction manager, code that is executed in a view callable doesn't *really* finish till after the users' function is over... so there's no chance of cleaning up if the transaction fails. Actually, there is: http://www.zodb.org/zodbbook/transactions.html#after-commit-hooks Neat, that's helpful, thanks. In critical cases, one can use this... but it's not the usual Pyramid pattern, so the larger philosophical question still stands, IMO: when using transactions, how all-in does one have to go? We actually can and do make file io transactional: http://docs.repoze.org/filesafe/ Also cool, but it wouldn't work for deletes, renames, etc. Is there an actual problem here that can't be solved by either making sessions transactional or by after-commit? I am not having an actual problem... I had one, but it was pilot error, and it's gone. It's more like a theoretical problem. Does that annoy you ;-)? As I grok pyramid, I'm seeing the theoretical chance to write unsafe code when using transactions; I can internalize the possibility as I code that a partially successful login event needs to be treated very carefully. I suppose it's a side effect of repoze.tm being in the pipeline and magically saving my database changes when I don't have to. Of course I don't have to use this feature, I know... This discussion is teasing out some things, I think: 1. Sessions should be transactional if transaction manager is on the stack. (as you've said like 3 times now) 2. perhaps we need a cookbook entry coupled with some fancy view-callable handler code that supports after-commit. (Forgive me if these things are all boring done-deal items in the Zope land, I'm not from that world.) -- You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com. To unsubscribe from this group, send email to pylons-discuss+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en.
Re: deform: Support for readonly on individual form fields?
You can do something like:
Assuming you have two schema nodes in your schema:
foo and bar, you can have a readonly foo like this:
schema = Schema()
form = Form(schema)
form.set_widgets({'foo':widget.TextInputWidget(template='readonly/
textinput')})
--
You received this message because you are subscribed to the Google Groups
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com.
To unsubscribe from this group, send email to
pylons-discuss+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/pylons-discuss?hl=en.
Re: Pyramid: choose renderer at runtime
--On Donnerstag, August 04, 2011 06:57:15 -0700
neurino neur...@gmail.com wrote:
[...]
In a Pylons controller I could select which renderer use like this:
def my_view(self):
c.form = Form(request, BasicSchema)
if c.form.validate():
obj = form.bind(MYModel())
if **need more info**:
c.form = Form(AdvancedSchema, obj=obj)
if form.validate():
obj = form.bind(obj)
else:
return render('advanced_form.mako')
**persist obj someway**
redirect(...)
return render('basic_form.mako')
In my Pyramid app I have:
config.add_route('new', '/new')
config.add_view(my_view, route_name='new',
renderer='basic_form.mako')
and in `basic_form.mako` according to a passed value I render the form
one way or another:
%if step == 1:
#render basic form
%else:
#render advanced form
%endif
but it seems a bit ugly and cumbersome to me.
Any advice?
You can implement the same pattern if you return a response object [1] and
don't have to specify the renderer in add_view(). Even if you specify
one it will be skipped if you return a response object, e.g. with
render_to_response().
..Carsten
[1]
http://docs.pylonsproject.org/projects/pyramid/1.1/narr/templates.html#using-templates-directly
--
You received this message because you are subscribed to the Google Groups
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com.
To unsubscribe from this group, send email to
pylons-discuss+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/pylons-discuss?hl=en.
Re: Pyramid transactions and other request lifecycle events
On Thu, Aug 4, 2011 at 1:37 PM, Chris McDonough chr...@plope.com wrote: when using transactions, how all-in does one have to go? I guess in a sense the answer is none and a lot at the same time. The transaction machinery provides a hook to do something based on the commit status. If the transaction has aborted, do X; if it has committed, do Y. There's probably second-order details that would need to be addressed in any particular integration that uses the after commit hook but without seeing code it's hard to make any sort of general statement about it. Agreed; time for code. I might be able to take a crack at this. If you're using ZODB, you have Blob objects, which, although they're not documented very well, make file IO 100% transactional. Yeah, I'm playing with them for the first time, thanks. No so far. ;-) But it probably will if it goes on much longer without any code being shown. A codeless conversation seems more suited to be had over a beer or in a consulting engagement than to be had over the maillist, I think. Right. I would love to buy you a beer; I see from the twitter that you like strong ales and so-on; my brother runs the local micro-brew supply shop and I'm in Madison, WI, which is good beer country. I don't think you're in Madison, WI. But yeah, time to code. Maybe. I really dont know. I've internalized the transaction pattern so much and I've written so much code that relies on synchronization of multiple storages via a single commit point that I tend to think of code that uses multiple storages and which doesn't use a transaction manager as broken by design. I think this may be my point, just I didn't know how to make it. Only part of Pyramid is transactional, and it's the good parts, but there seems to be a small bit of leakage, at least on the branch* that I'm following. (* Pyramid seems to have some branches of common patterns since it's so flexible: zodb/sqla, traversal/routes, etc.. I'm on the traversal, zodb, pyramid_mailer, pyramid_deform, pyramid_beaker path ) Those things above sound reasonable. (FTR, that's not me volunteering to do them right now, just an agreement they sound reasonable and deserve an entry in some TODO). So noted. For The Record. -- You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com. To unsubscribe from this group, send email to pylons-discuss+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en.
