[sqlalchemy] Hyper-eager autoflush for debugging
Hi, a common pattern for me is to encounter a ProgrammingError or other exception on autoflush. Then the point in my code at which I made a booboo has already passed and so is not indicated by the stack trace. If the cause isn't obvious, I will go rerunning the failing test, inserting session.flush() at strategic places before the offending code path, to narrow down where the bad DB operation really happens. Is it feasible to add a mode like autoflush='eager' on the session? Meaning that any operation that will result in an autoflush does so immediately. Terrible in general, but a useful mode to enable for debugging cases like this. Downsides I imagine: (a) check constraints may be violated on intermediate states being flushed (e.g. two properties need to be updated together but in this mode wouldn't be), and (b) a check for this state might need to be in a critical path, nibbling at runtime performance. But Michael/others will know better than me whether these are real hindrances. Regards, - Gulli --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Elixir 0.7.1 released
I am very pleased to announce that version 0.7.1 of Elixir (http://elixir.ematia.de) is now available. As always, feedback is very welcome, preferably on Elixir mailing list. This release packs a few interesting new features (abstract classes and a new collection type providing explicit relative resolution of target entities) and some bug fixes. It is a safe upgrade from the 0.7.0 release. The full list of changes can be seen at: http://elixir.ematia.de/trac/browser/elixir/tags/0.7.1/CHANGES What is Elixir? - Elixir is a declarative layer on top of the SQLAlchemy library. It is a fairly thin wrapper, which provides the ability to create simple Python classes that map directly to relational database tables (this pattern is often referred to as the Active Record design pattern), providing many of the benefits of traditional databases without losing the convenience of Python objects. Elixir does not intend to replace SQLAlchemy's core features, and instead focuses on providing a simpler syntax for defining model objects when you do not need the full expressiveness of SQLAlchemy's manual mapper definitions. Mailing list http://groups.google.com/group/sqlelixir/about -- Gaëtan de Menten http://openhex.org --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Primary key is must in every table.
Dear All, In my application, i have a master table and a details table. In the details table containes foreign key only no primary key. I will be getting the following error while running. ArgumentError: Mapper Mapper|LpoIssueDetails|tbl_lpo_issue_details could not assemble any primary key columns for mapped table 'tbl_lpo_issue_details' Is it must a Primary key in the details table.? Thanks in advance. Regards, Niyas --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] UnboundExecutionError with object on session
Hi all! I've experiencing this problem for some time now and even after debugging, i could not find why it happens. I have a medium sized multi-thread application that manipulates SQLAlchemy objects - the objects are passed from thread to thread, so, when i load an instance i close the session. After working on the object, i need to update the status of the it, so i do something like this: job.status = FINISHED session.add(job) log.info(Finished job %s % job.filename) session.flush() This code works pretty well, but from time to time, i get this on the logs: UnboundExecutionError: Instance Job at 0x413ee50 is not bound to a Session; attribute refresh operation cannot proceed The complete stacktrace is here: http://pastebin.org/54196 Could this be happening because of my pool_recycle setting (300 seconds) ? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: UnboundExecutionError with object on session
Fernando Takai wrote: Hi all! I've experiencing this problem for some time now and even after debugging, i could not find why it happens. I have a medium sized multi-thread application that manipulates SQLAlchemy objects - the objects are passed from thread to thread, so, when i load an instance i close the session. After working on the object, i need to update the status of the it, so i do something like this: job.status = FINISHED session.add(job) log.info(Finished job %s % job.filename) session.flush() This code works pretty well, but from time to time, i get this on the logs: UnboundExecutionError: Instance Job at 0x413ee50 is not bound to a Session; attribute refresh operation cannot proceed The complete stacktrace is here: http://pastebin.org/54196 Could this be happening because of my pool_recycle setting (300 seconds) ? its not related to the pool. Your objects have expired or unloaded attributes present on them which will attempt to load when accessed. The object must be attached to a Session for this to proceed. the easiest way to deal with this is to merge() the objects into a new thread-local Session before using. Alternatively, ensure all required attributes are loaded. This often requires touching the attributes explicitly in the case of joined table inheritance or lazily-loaded relations(). Also note that calling session.commit() or session.rollback() expires all attributes, so avoid these in the case of objects becoming detached, or expunge() the objects before an expiration occurs. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Select as
Hi. I am sort of a newbie on SQLAlchemy. Is there a way to do a query with the ORM, and doing an as on the select. For example: class Example(Base): name = Column(Unicode(512) ) query = session.query(Example.name AS foo) query.all() Thanks in advance.. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Modify the start number of a sequence after it's been created
Hello everyone!
I've been searching information regarding this topic but without help
so far. I have a dbloader.py script that takes information from a
number of CSV files and dumps it into the database. After that we
execute a number of tests to check all services are working fine
(returning the right information and calculated data such as
availability and so on).
In these CSV files I have manually set up the id fields of every
table, but when I add the rows to the database, I am passing the id
value as part of the dictionary, therefore the sequence of the primary
key is not updated. So, after all data has been loaded, I'd like to
update the start value of the sequence.
It's PostgreSQL, therefore I could do something like:
maxid = engine.execute('select max(id) from table').fetchone()[0]
engine.execute(select setval('table_id_seq', %s) % (maxid + 1))
But I'd like to do this properly, at SA level. Just in case database
changes in the future or just to satisfy myself. But I have not been
able to find the way to modify the attributes of the sequence of each
table I find in the metadata. I have found ways of retrieving the
nextid in the documentation, but not modifying it.
Is there a way to grab a sequence and set it's next value using the
abstractions SA facilitates?
Thanks in advance.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: Select as
Tomas Zulberti wrote: Hi. I am sort of a newbie on SQLAlchemy. Is there a way to do a query with the ORM, and doing an as on the select. For example: class Example(Base): name = Column(Unicode(512) ) query = session.query(Example.name AS foo) query.all() Every column object has a label method, e.g. Example.name.label(foo). However, the label is lost when you do query.all(), so I'm not sure what you're trying to do here. -Conor --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Primary key is must in every table.
Niyas wrote: Dear All, In my application, i have a master table and a details table. In the details table containes foreign key only no primary key. I will be getting the following error while running. ArgumentError: Mapper Mapper|LpoIssueDetails|tbl_lpo_issue_details could not assemble any primary key columns for mapped table 'tbl_lpo_issue_details' Is it must a Primary key in the details table.? Thanks in advance. Regards, Niyas SQLAlchemy mappers require some sort of key to map rows to instances. The key that it uses does not have to be the primary key of the table you are using: any set of columns usable for a unique constraint would work. See the allow_null_pks and primary_key mapper args at http://www.sqlalchemy.org/docs/05/reference/orm/mapping.html#defining-mappings for more details. BTW, is there a reason you cannot make your foreign key column also be the primary key? -Conor --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Hyper-eager autoflush for debugging
On Nov 16, 3:38 pm, Michael Bayer mike...@zzzcomputing.com wrote: id like to know what errors specifically you get which aren't fairly obvious even if they happen later on. I'm guessing some of the dependency oriented messages like dependency rule tried to blank out primary key. Nah, the particular one I stubbed my toe on today was: TypeError: can't compare offset-naive and offset-aware datetimes ... caused by accidentally slipping a non-timezone-aware datetime into a query/operation somewhere (with timezone-aware datetime columns, on PostgreSQL). But I have also found myself wanting this when debugging general ProgrammingErrors --- I get all kinds, because I am mixing ORM and sqlexpression calls with literal SQL through session.execute() --- unavoidable because my model is fairly dynamic (a sort of generalized warehousing application, thousands of tables, bunch of bulk operations and DDL at runtime). It's always an easy option to add more contextual information to our exception messages if that helps. The exception messages are generally fine --- rather, I'm after more accurate contextual exception stack traces (not generally, just when I particularly ask for it). This is the 2nd use case in a couple of weeks where such a proxy system is useful, leading to the notion of having a common extension that allows generic wrapping of attribute operations ...but like that other use case, both are cases that I'm not entirely convinced are even necessary. My case is absolutely not necessary. Just an idea for debugging convenience. well as far as performance, that would be entirely out the window. and yeah constraints like composite primary keys and such wouldn't work. But also, this level of meddling is definitely not something I'd want to see in the core, adding conditionals to everything. Implementation wise it would be a SessionExtension that intercepts attach, plus the abovementioned proxies for InstrumentedAttribute and CollectionAdapter. Yeah, performance would be abysmal, but one would only do this when debugging anyway. Thanks for the pointers --- I'll give the SessionExtension (Co.) approach a try, if I get another troubleshooting case that itches enough. If I come up with something useful, I'll post it here or put it on UsageRecipes. - Gulli --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] filtering with an association_proxy
Is there a way to filter a query involving an association_proxy?
For example, say I have a one to many relation between Pages and Tags,
and an association_proxy like this to let me represent tags as a list
of strings.
tag_objects = orm.relation('Tag')
tags = association_proxy('tag_objects', 'name')
Now I want to find all the pages tagged with 'foo'. As far as I know I
have to break the abstraction barrier provided by the
association_proxy and do something like:
sess.query(Page.tag_objects.any(name='foo'))
Is there any mechanism for doing something like this instead?
sess.query(Page.tags.any('foo'))
If there's nothing similar already existing, is this functionality
desirable? It seems like it could be really useful for clarifying
complex filtering, especially involving many to many relations with
association objects. I wrote an example patch implementing this
for .any() and .has(), with tests. I'm happy to post a ticket and
flesh it out more if it seems reasonable.
http://web.mit.edu/storborg/Public/better-associationproxy-filtering.patch
Thanks,
Scott
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
