[sqlalchemy] Re: app memory leak - execute() with SA 0.5.2, PG 8.3, Psycopg2
On Feb 22, 6:08 am, Allen Bierbaum abierb...@gmail.com wrote: The python process. The number of objects seems to remain fairly controlled. But the amount of resident memory used by the python process does not decrease. I had expected that by calling gc.collect(2) python would reclaim any objects that could be freed and free all memory associated with them, thus decreasing the consumed memory. Maybe this is an invalid assumption. Do you know any way to ask python to shrink it's process size (ie. clear unused memory that has been freed but evidently not given back to the OS)? Python 2.5 and later will free up garbage collected memory, handing it back to the system. Previous versions of Python would never free up memory (hence never shrink in size). Are you using Python 2.4? Cheers, Chris Miles --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: app memory leak - execute() with SA 0.5.2, PG 8.3, Psycopg2
Allen Bierbaum schrieb: The python process. The number of objects seems to remain fairly controlled. But the amount of resident memory used by the python process does not decrease. I had expected that by calling gc.collect(2) python would reclaim any objects that could be freed and free all memory associated with them, thus decreasing the consumed memory. Maybe this is an invalid assumption. Do you know any way to ask python to shrink it's process size (ie. clear unused memory that has been freed but evidently not given back to the OS)? Thats an invalid assumption for most systems with pooling allocators. Its usually called a high-watermark allocator which assumes your process has a steady state of memory usage. Its usually not a problem, unless your running short of virtual address space/swap space because any decent OS will just page out the unused memory blocks. But you may be out of luck still if the layout of the pool prevents such e.g. if one object in every memory page is active the system cannot swap out anything and you need a harder working GC to readjust all references to really get completly free blocks. In addition you need the cooperation of the OS free() call, not all systems really give memory back, even if you free() it, some just keep it in a pool for the process. One typical pattern to get around this is to just fork a worker for the memory intensive stuff and let it quit when done so the memory gets returned to the system. Michael -- Michael Schlenker Software Engineer CONTACT Software GmbH Tel.: +49 (421) 20153-80 Wiener Straße 1-3 Fax:+49 (421) 20153-41 28359 Bremen http://www.contact.de/ E-Mail: m...@contact.de Sitz der Gesellschaft: Bremen Geschäftsführer: Karl Heinz Zachries, Ralf Holtgrefe Eingetragen im Handelsregister des Amtsgerichts Bremen unter HRB 13215 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] boolean value for column_property
Hi guys,
I just discovered column_property and I am trying to make a boolean
property out of a table column.
Let's say I have this didactic table:
tblUsers = Table('users', metadata,
Column('user', Integer),
Column('type', Integer))
mapper(Users, tblUsers, properties =
dict(isSubcontractor = column_property(select([True], tblUsers.c.type ==
2)))
I want to have a property on the object that would say if the user is a
subcontractor or not:
Users.isSubcontractor
I am not sure if this is column_property's fault or rather the way I am
using select there.
Also, if there's a simpler way to do this I am thankful.
Thanks.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] with_statement support in session objects
Hi, trying to be careful to close all sessions when I'm done with them, I find myself doing this all the time: session = Session() try: do_stuff_with(session) finally: session.close() This would be neater: with Session() as session: do_stuff_with(session) but the sessionmaker-produced class does not implement the context manager protocol (the __enter__ and __exit__ methods) used by the with statement. Now, I can add on the context manager protocol using contextlib: from contextlib import closing with closing(Session()) as session: do_stuff_with(session) but is there any reason for the session itself not to support the context manager protocol and save me the extra closing()? Regards, - Gulli --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: with_statement support in session objects
I am using a decorator that handles all this dirty work:
@session_manager('get_session')
def do_session_stuff(self):
pass
Gunnlaugur Briem wrote:
Hi,
trying to be careful to close all sessions when I'm done with them, I
find myself doing this all the time:
session = Session()
try:
do_stuff_with(session)
finally:
session.close()
This would be neater:
with Session() as session:
do_stuff_with(session)
but the sessionmaker-produced class does not implement the context
manager protocol (the __enter__ and __exit__ methods) used by the with
statement. Now, I can add on the context manager protocol using
contextlib:
from contextlib import closing
with closing(Session()) as session:
do_stuff_with(session)
but is there any reason for the session itself not to support the
context manager protocol and save me the extra closing()?
Regards,
- Gulli
Earn more money. Click here to be trained in human resources and launch your
career.
http://ads.lavabit.com/fc/BLSrjwrsFB3b2AgV4nQOOBOi9zEHy45QcpNGLOxaH5Pv5FyYj3gBD3XRd2w/
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Use with multiple Processes
Hi, I am using SQLAlchemy 0.4.8 with Python 2.4. I have a process which is selecting some rows, doing something with the data of these rows and then writes a result into the database. How can I achieve a solution where multiple of these processes are running without having them selecting the same rows and doing the same stuff? Normally I would suggest s.th. like Select for update or an trigger which sets a timestamp after a select... Do you have any suggestions for having multiple processes working on the same table but not on the same rows? Thanks --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Events Undo/Redo Functionality?
Hi everyone, I am developing an offline music synthesizing and sequencing application in Python, using the SQLite backend to store song data. At the moment, I am using my own object relational mapper between SQL and Python, which is fairly minimal. I feel compelled to use SQLAlchemy, but there are two areas where I would like to assess my options before I go forward and embed SQLAlchemy into my application. 1) Is it possible/planned to connect signal handlers to database changes? In my app, changing object properties will call connected event handlers, which in turn refresh UI views. How could I do this with SQLA? Can you think of another way to update the UI from DB changes? Another way could be to have a hook listen to transactions and generate journal records from which events can be signalled - this would even enable multiple processes to handle events. Does that sound more plausible? Is it possible? 2) I see that SQLA supports rollback for pending transactions, which is a great feature. But as far as I understand, it is not possible to rollback transactions that have already been committed (undo). How would I go about and implement such a feature into my application best, using SQLA as a backend? I understand that SQL is traditionally used in web services, where event handling is virtually impossible and extensive undo/redo functionality is rarely needed. But I believe it makes perfect sense to use databases for document formats as well. Looking forward to your answer. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Use with multiple Processes
I would select out the total set of rows and then hand off groups of those rows, converted into serializable objects first, using the imap function of a multiprocessing.Pool object. It would be best if the rows are returned via a ResultProxy so that work can begin on results before all results are fetched. See http://docs.python.org/library/multiprocessing.html#module-multiprocessing.pool . Its critical that you call create_engine() within each subprocess if additional queries are to be issued in workers, and don't pass any other database resources like connections or cursors across. Pool allows initialization activities via the initializer argument. On Feb 24, 7:05 am, dagooglaa danielso...@gmx.de wrote: Hi, I am using SQLAlchemy 0.4.8 with Python 2.4. I have a process which is selecting some rows, doing something with the data of these rows and then writes a result into the database. How can I achieve a solution where multiple of these processes are running without having them selecting the same rows and doing the same stuff? Normally I would suggest s.th. like Select for update or an trigger which sets a timestamp after a select... Do you have any suggestions for having multiple processes working on the same table but not on the same rows? Thanks --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] using relation in non-declarative table and class definitions (newbie problem)
Hello everyone,
I've been following tutorial
http://www.sqlalchemy.org/docs/05/ormtutorial.html which explains how to
use 'relation' function using declarative extension.
However, in my app I define classes and tables for classes separately,
and tutorial does not cover how to use relation in such context
(googling for it does not return much info either) . All it says is this:
When declarative is not in use, you typically define your mapper() well
after the target classes and Table objects have been defined, so string
expressions are not needed.
Does that mean that I need to define relation in mapper(UserClass,
usertable) call somehow?
I have:
hosts_table = Table('hosts',md,
Column('id',Integer,primary_key=True),
Column('IP',String),
Column('HostName',String),
Column('Location',String),
Column('Architecture_id',Integer,ForeignKey('architecture.id')),
Column('OS_Kind_id',Integer,ForeignKey('os_kind.id')),
Column('OS_version_id',Integer,ForeignKey('os_version.id')),
Column('Additional_info__visible_for_all_users',String),
Column('End_Date',SLDate),
...
mapper(Host, hosts_table)
architecture_table = Table('architecture',md,
Column('id',Integer,primary_key=True),
Column('Architecture',String))
mapper(Architecture, architecture_table)
os_kind_table = Table('os_kind',md,
Column('id',Integer,primary_key=True),
Column('OS_Kind',String))
mapper(OS_Kind, os_kind_table)
os_version_table = Table('os_version',md,
Column('id',Integer,primary_key=True),
Column('OS_version',String))
mapper(OS_version, os_version_table)
email_table = Table('email',md,
Column('id',Integer,primary_key=True),
Column('Email',String))
mapper(Email,email_table)
Now, md.create_all() does create proper db structure (I know bc I
examined sqlite db after session.commit()).
However, if I do not use the relation() somewhere, does that mean that
foreign keys may not work properly or there will be some other adverse
effects?
How do I use relation() with above?
I'd love to see such info added to the tutorial..
Regards,
mk
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: Events Undo/Redo Functionality?
these are two general patterns, observer/listener and undo/redo (command-pattern), which have nothing to do with DB. i dont know about 2) DB-observers which would react on somebody else changing the DB, AFAIK that is possible on certain servers only, but the rest should be done independly on higher level than DB-stuff. read about Model-View-Controller. On Tuesday 24 February 2009 15:00:28 paniq303 wrote: Hi everyone, I am developing an offline music synthesizing and sequencing application in Python, using the SQLite backend to store song data. At the moment, I am using my own object relational mapper between SQL and Python, which is fairly minimal. I feel compelled to use SQLAlchemy, but there are two areas where I would like to assess my options before I go forward and embed SQLAlchemy into my application. 1) Is it possible/planned to connect signal handlers to database changes? In my app, changing object properties will call connected event handlers, which in turn refresh UI views. How could I do this with SQLA? Can you think of another way to update the UI from DB changes? Another way could be to have a hook listen to transactions and generate journal records from which events can be signalled - this would even enable multiple processes to handle events. Does that sound more plausible? Is it possible? 2) I see that SQLA supports rollback for pending transactions, which is a great feature. But as far as I understand, it is not possible to rollback transactions that have already been committed (undo). How would I go about and implement such a feature into my application best, using SQLA as a backend? I understand that SQL is traditionally used in web services, where event handling is virtually impossible and extensive undo/redo functionality is rarely needed. But I believe it makes perfect sense to use databases for document formats as well. Looking forward to your answer. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] sequence with predicate, locks and select for update
Hi All!
I use postgresql and I have to emulate partial sequence (sequence with
predicate).
I have transactions table with serial_id field - this field have to
be autoincrement field and unique for commit_date and
user_office_id.
When I perform an insert to this table I have to do something like
that in SQL:
BEGIN;
LOCK transactions IN ROW SHARE MODE;
SELECT serial_id FROM transactions WHERE FOR UPDATE;
MAX_SERIAL_ID = SELECT MAX(serial_id) FROM transactions WHERE
commit_date=CURRENT_TIMESTAMP AND user_office_id=93;
INSERT INTO transactions VALUES (MAX_SERIAL_ID+1, 93, 'other data1'),
(MAX_SERIAL_ID+1, 93, 'other data2'), (MAX_SERIAL_ID+1, 93, 'other
data3');
COMMIT;
--OR--
last_serial = execute(select([func.max(transactions.c.serial)],
transactions.c.user_office_id==93,
transactions.c.commit_date==datetime.date.today())).fetchone()[0]
AND then execute(transactions_table.insert(), [{93, last_serial+1,
'other', 'data'}, ...])
How to do that to not allow other insert's to be executed between my
select and insert statements from other requests and I don't want to
block select statements to transactions table!? Is there any magic
in SA to do that or may be I choose wrong way to do that kind of
stuff? Help me please I'm stucked with that :(
Thanks a lot!
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: sequence with predicate, locks and select for update
In SQL I hate that second SELECT, I can't perform SELECT MAX (serial_id) FROM transactions WHERE bla,bla,bla FOR UPDATE; because I get ERROR: SELECT FOR UPDATE/SHARE is not allowed with aggregate functions... --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Problems with cursor, don't know what to do.
Hi all, I'm new to SQLAlchemy, and I've a problem using it.
[tu...@2kan-2:~/bigur/calculation]%
python
Python 2.5.2 (r252:60911, Feb 23 2009, 21:22:55)
[GCC 4.0.1 (Apple Inc. build 5488)] on darwin
Type help, copyright, credits or license for more information.
from sqlalchemy import *
db =
create_engine('mssql://some_name:some_passwpod@/?dsn=Libradriver=FreeTDS')
metadata = MetaData()
metadata.bind = db
metadata.bind.echo = True
users_table = Table('tax', metadata, autoload=True)
2009-02-24 20:54:55,261 INFO sqlalchemy.engine.base.Engine.0x...2850
SELECT user_name() as user_name;
2009-02-24 20:54:55,261 INFO sqlalchemy.engine.base.Engine.0x...2850
[]
2009-02-24 20:54:55,280 WARNING sqlalchemy.pool.QueuePool.0x...27b0
Error closing cursor: Attempt to use a closed cursor.
2009-02-24 20:54:55,280 INFO sqlalchemy.engine.base.Engine.0x...2850
ROLLBACK
Traceback (most recent call last):
File stdin, line 1, in module
File /opt/local/lib/python2.5/site-packages/
SQLAlchemy-0.5.3dev_r5814-py2.5.egg/sqlalchemy/schema.py, line 113,
in __call__
return type.__call__(self, name, metadata, *args, **kwargs)
File /opt/local/lib/python2.5/site-packages/
SQLAlchemy-0.5.3dev_r5814-py2.5.egg/sqlalchemy/schema.py, line 241,
in __init__
_bind_or_error(metadata).reflecttable(self,
include_columns=include_columns)
File /opt/local/lib/python2.5/site-packages/
SQLAlchemy-0.5.3dev_r5814-py2.5.egg/sqlalchemy/engine/base.py, line
1265, in reflecttable
self.dialect.reflecttable(conn, table, include_columns)
File /opt/local/lib/python2.5/site-packages/
SQLAlchemy-0.5.3dev_r5814-py2.5.egg/sqlalchemy/databases/mssql.py,
line 1123, in reflecttable
current_schema = self.get_default_schema_name(connection)
File string, line 1, in lambda
File /opt/local/lib/python2.5/site-packages/
SQLAlchemy-0.5.3dev_r5814-py2.5.egg/sqlalchemy/engine/base.py, line
1894, in decorated
connection.info[key] = val = fn(self, connection)
File /opt/local/lib/python2.5/site-packages/
SQLAlchemy-0.5.3dev_r5814-py2.5.egg/sqlalchemy/databases/mssql.py,
line 1070, in get_default_schema_name
user_name = connection.scalar(sql.text(query))
File /opt/local/lib/python2.5/site-packages/
SQLAlchemy-0.5.3dev_r5814-py2.5.egg/sqlalchemy/engine/base.py, line
814, in scalar
return self.execute(object, *multiparams, **params).scalar()
File /opt/local/lib/python2.5/site-packages/
SQLAlchemy-0.5.3dev_r5814-py2.5.egg/sqlalchemy/engine/base.py, line
1676, in scalar
self.connection._handle_dbapi_exception(e, None, None,
self.cursor, self.context)
File /opt/local/lib/python2.5/site-packages/
SQLAlchemy-0.5.3dev_r5814-py2.5.egg/sqlalchemy/engine/base.py, line
931, in _handle_dbapi_exception
raise exc.DBAPIError.instance(statement, parameters, e,
connection_invalidated=is_disconnect)
sqlalchemy.exc.ProgrammingError: (ProgrammingError) Attempt to use a
closed cursor. None None
What I'm doing wrong?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: app memory leak - execute() with SA 0.5.2, PG 8.3, Psycopg2
On Tue, Feb 24, 2009 at 4:44 AM, Chris Miles miles.ch...@gmail.com wrote: On Feb 22, 6:08 am, Allen Bierbaum abierb...@gmail.com wrote: The python process. The number of objects seems to remain fairly controlled. But the amount of resident memory used by the python process does not decrease. I had expected that by calling gc.collect(2) python would reclaim any objects that could be freed and free all memory associated with them, thus decreasing the consumed memory. Maybe this is an invalid assumption. Do you know any way to ask python to shrink it's process size (ie. clear unused memory that has been freed but evidently not given back to the OS)? Python 2.5 and later will free up garbage collected memory, handing it back to the system. Previous versions of Python would never free up memory (hence never shrink in size). Are you using Python 2.4? I am using Python 2.5. But now that I understand the issue better I have come up with a workaround. The biggest issue was that I didn't understand what I should be seeing as far as memory usage. Thanks to everyone for the help. -Allen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Separating session and db/models definitions
Hello. I have the following states of things: 1) I need to place some common database metadata description and models in one package 2) There are at least to applications, which will use this models: first app uses scoped session with their own scope_func, second app uses session from sessionmaker 3) I have some logic in models' methods that need session object to present (delete relations or etc.) I have no idea how to do it... Is there need for some kind of proxy to session, which will appear later, when app decide to instantiate it? Or I must define session in place with models? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Events Undo/Redo Functionality?
On Feb 24, 5:16 pm, a...@svilendobrev.com wrote: these are two general patterns, observer/listener and undo/redo (command-pattern), which have nothing to do with DB. i dont know about 2) DB-observers which would react on somebody else changing the DB, AFAIK that is possible on certain servers only, but the rest should be done independly on higher level than DB-stuff. read about Model-View-Controller. Your reply sounds a bit offended. I'm sorry if my request came over as kind of rude. I don't mean to upset anybody, and I don't want to take up your time, so please send me elsewhere if my inquiry does not fit the projects scope. I'm using a Model-View-Controller scheme at the moment. I also know of the command pattern and how it works. But I'm trying to write as little code on my own as I can. SQLAlchemy provides a high level model part for the application, and borders on the issues I presented above. I can see that the issues I presented are not neccessarily of importance in the scope of SQL. So let me rephrase my question: how could I solve above two issues in the most efficient manner, leveraging whatever SQL and SQLAlchemy have to offer? Is there any short cut I can take on the way to undo/redo functionality? If journaling DB changes would be a good way to track changes, how can I hook up on them best? Where would my plan fit in? I'm sorry if these questions swamp you. Maybe I am thinking too much. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: is _sa_instance_state.get_history stable to use?
It would be useful for me, but I am not sure how many others. I am not confident enough to write a patch for it yet though. I could not get the above to work, that may be my bug, as I use mappers a bit too ruthlessly over my tests they (or I) get a bit confused. It fails less though if I run compile_mappers() and even less if I run mapper.compile() before I run the above but I always hit an UnmappedClassError at some point, for a class which, to me, seems mapped (I run the above just after its been mapped and run compile). The class, table and mapper are wrapped in another class instance, which may be the issue, but the actual reason currently mystifies me. At the moment I am just using an unmodified AttributeExtension on each column I want to get active_history for. Which was not a pain for me and looking at the AttributeImpl code, incurs little overhead. On Feb 24, 2:46 am, Michael Bayer mike...@zzzcomputing.com wrote: ive thought about this and it might be the kind of thing we just add a flag on mapper() for. I had exactly this same issue with a project I did for someone. The solution I gave below works, but needs to be called only after compile_mappers() is called. On Feb 23, 2009, at 5:16 PM, Michael Bayer wrote: there is not, you'd have to do something along the lines of: for prop in local_mapper.iterate_properties: getattr(local_mapper.class_, prop.key).impl.active_history = True On Feb 23, 2009, at 4:18 PM, kindly wrote: This did not end up working as, I think, active_history is by default false. Is there a way in a mapper to set this for all attributes to be True? Its a shame as any AttributeExtension will fix this as they imply True. On Feb 18, 3:23 pm, Michael Bayer mike...@zzzcomputing.com wrote: get_history() is a public function within the attributes package at the top level, and I also added API documentation for it recently (not on the site yet). from sqlalchemy.orm.attributes import get_history, instance_state get_history(instance_state(myobject), someattribute) in the latest trunk you can pass myobject straight to get_history without using instance_state. session.is_modified() just calls get_history() on every attribute, so you might want to consider just looping through the attributes the way is_modfied() does and do your history operation inline. On Feb 18, 2009, at 9:43 AM, kindly wrote: Hello I intend to log any changes (not new rows) to a table. The simplist way I can see to do this is to check every object that is added for changes. I intend to, before every flush, look in dirty then use is_modified and if there is a change use get_history to find out the original and new attribute values. I am wary of doing it this way as _sa_instance_state looks private even though get_history is not. Should this be safe to use? Or is there a better way? Thanks David --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: is _sa_instance_state.get_history stable to use?
It would be useful for me, but I am not sure how many others. I am not confident enough to write a patch for it yet though. I could not get the above to work, that may be my bug, as I use mappers a bit too ruthlessly over my tests they (or I) get a bit confused. It fails less though if I run compile_mappers() and even less if I run mapper.compile() before I run the above but I always hit an UnmappedClassError at some point, for a class which, to me, seems mapped (I run the above just after its been mapped and run compile). The class, table and mapper are wrapped in another class instance, which may be the issue, but the actual reason currently mystifies me. At the moment I am just using an unmodified AttributeExtension on each column I want to get active_history for. Which was not a pain for me and looking at the AttributeImpl code, incurs little overhead. On Feb 24, 2:46 am, Michael Bayer mike...@zzzcomputing.com wrote: ive thought about this and it might be the kind of thing we just add a flag on mapper() for. I had exactly this same issue with a project I did for someone. The solution I gave below works, but needs to be called only after compile_mappers() is called. On Feb 23, 2009, at 5:16 PM, Michael Bayer wrote: there is not, you'd have to do something along the lines of: for prop in local_mapper.iterate_properties: getattr(local_mapper.class_, prop.key).impl.active_history = True On Feb 23, 2009, at 4:18 PM, kindly wrote: This did not end up working as, I think, active_history is by default false. Is there a way in a mapper to set this for all attributes to be True? Its a shame as any AttributeExtension will fix this as they imply True. On Feb 18, 3:23 pm, Michael Bayer mike...@zzzcomputing.com wrote: get_history() is a public function within the attributes package at the top level, and I also added API documentation for it recently (not on the site yet). from sqlalchemy.orm.attributes import get_history, instance_state get_history(instance_state(myobject), someattribute) in the latest trunk you can pass myobject straight to get_history without using instance_state. session.is_modified() just calls get_history() on every attribute, so you might want to consider just looping through the attributes the way is_modfied() does and do your history operation inline. On Feb 18, 2009, at 9:43 AM, kindly wrote: Hello I intend to log any changes (not new rows) to a table. The simplist way I can see to do this is to check every object that is added for changes. I intend to, before every flush, look in dirty then use is_modified and if there is a change use get_history to find out the original and new attribute values. I am wary of doing it this way as _sa_instance_state looks private even though get_history is not. Should this be safe to use? Or is there a better way? Thanks David --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Events Undo/Redo Functionality?
If you choose to go down the journaling route, some things would be helpful: 1. all of your functions (journalable actions) should have inverse functions to reverse them. If this is true, then from a particular state, you can simply record the actions, checkpointing all user objects periodically. A sequence might be something like: save binary state of file cut(1:30, 1:40) loop(a,13, sec=10) Since you described an music editing app, doing this in clean way (without losing precision and information) might require storing actual binary states. This sounds, to me, a little nightmarish. I don't have any insights on the SA front, alas. Gregg Lind On Tue, Feb 24, 2009 at 2:41 PM, paniq303 paniq...@googlemail.com wrote: On Feb 24, 5:16 pm, a...@svilendobrev.com wrote: these are two general patterns, observer/listener and undo/redo (command-pattern), which have nothing to do with DB. i dont know about 2) DB-observers which would react on somebody else changing the DB, AFAIK that is possible on certain servers only, but the rest should be done independly on higher level than DB-stuff. read about Model-View-Controller. Your reply sounds a bit offended. I'm sorry if my request came over as kind of rude. I don't mean to upset anybody, and I don't want to take up your time, so please send me elsewhere if my inquiry does not fit the projects scope. I'm using a Model-View-Controller scheme at the moment. I also know of the command pattern and how it works. But I'm trying to write as little code on my own as I can. SQLAlchemy provides a high level model part for the application, and borders on the issues I presented above. I can see that the issues I presented are not neccessarily of importance in the scope of SQL. So let me rephrase my question: how could I solve above two issues in the most efficient manner, leveraging whatever SQL and SQLAlchemy have to offer? Is there any short cut I can take on the way to undo/redo functionality? If journaling DB changes would be a good way to track changes, how can I hook up on them best? Where would my plan fit in? I'm sorry if these questions swamp you. Maybe I am thinking too much. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: is _sa_instance_state.get_history stable to use?
It would be useful for me, but I am not sure how many others. I am not confident enough to write a patch for it yet though. I could not get the above to work, that may be my bug, I always hit an UnmappedClassError at some point in my testing, for a class which seems mapped (I run the above just after its been mapped and then run compile_mappers). The compile_mappers throws this error too. I could not work out why as it only does it for certain classes. At the moment I am just using an unmodified AttributeExtension on each column I want to get active_history for. On Feb 24, 2:46 am, Michael Bayer mike...@zzzcomputing.com wrote: ive thought about this and it might be the kind of thing we just add a flag on mapper() for. I had exactly this same issue with a project I did for someone. The solution I gave below works, but needs to be called only after compile_mappers() is called. On Feb 23, 2009, at 5:16 PM, Michael Bayer wrote: there is not, you'd have to do something along the lines of: for prop in local_mapper.iterate_properties: getattr(local_mapper.class_, prop.key).impl.active_history = True On Feb 23, 2009, at 4:18 PM, kindly wrote: This did not end up working as, I think, active_history is by default false. Is there a way in a mapper to set this for all attributes to be True? Its a shame as any AttributeExtension will fix this as they imply True. On Feb 18, 3:23 pm, Michael Bayer mike...@zzzcomputing.com wrote: get_history() is a public function within the attributes package at the top level, and I also added API documentation for it recently (not on the site yet). from sqlalchemy.orm.attributes import get_history, instance_state get_history(instance_state(myobject), someattribute) in the latest trunk you can pass myobject straight to get_history without using instance_state. session.is_modified() just calls get_history() on every attribute, so you might want to consider just looping through the attributes the way is_modfied() does and do your history operation inline. On Feb 18, 2009, at 9:43 AM, kindly wrote: Hello I intend to log any changes (not new rows) to a table. The simplist way I can see to do this is to check every object that is added for changes. I intend to, before every flush, look in dirty then use is_modified and if there is a change use get_history to find out the original and new attribute values. I am wary of doing it this way as _sa_instance_state looks private even though get_history is not. Should this be safe to use? Or is there a better way? Thanks David --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Self Join via Join Table ?
Hello Everyone,
First of all, kudos on SQLAlchemy.. the speed is pretty amazing - I
am coming from the SQLObject world and there is a definite difference.
Excellent work. I am also getting to grips with it pretty quickly,
using object_session and all that good stuff. This said, I have hit
that 20% problem, and am hoping someone can shine a light on it.
I have a table, lets call it Foo and another table Bar. Foo should
be able to get a list of it's parents via Bar or it's children via
Bar. I am also using the declarative_base system rather than table/
mapper defined seperately.
class Foo(Base):
id = Column(Integer, primary_key=True)
class Bar(Base):
parent_id = Column(Integer, default=0)
child_id = Column(Integer, default=0)
So, I thought something like ; children = relation(Foo,
backref=backref('parents'), primaryjoin=and_(Foo.id==Bar.parent_id)
But that's where I hit the 'wall' as it were, is there a way to
setup a synonym for Foo in the primaryjoin clause ? Am I missing
something stupid ? (I am okay with that ;)
Regards
Stef
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Creating SQL Expression
Hello ,
I am trying to make query like
select (a+b) from xyz;
to do this
xyz = sqlalchemy.Table('xyz',metadata)
a = sqlalchemy.Column('a', sqlalchemy.Integer)
xyz.append_column(a)
b = sqlalchemy.Column('b', sqlalchemy.Integer)
xyz.append_column(b)
column = [(a + b)]
select = sqlalchemy.select(from_obj=xyz, columns=column,distinct=True)
This works fine for me.
Now when the columns a and b are dynamic (Enter by the user in form of
string) and the operator too comes from user
columns_list = ['a','b']
operator = ['+']
like this i get the input
so i make the loop and make
for both the columns something like this
columns = []
for x in column_list :
t = sqlalchemy.Column(x, sqlalchemy.Integer)
xyz.append_column(a)
columns.append(t)
so now
how to add + to make the quer run
Thanks in the advance.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
