[sqlalchemy] sqlamp - an implementation of Materialized Path for SQLAlchemy
I hope it will be interesting for somebody: http://sqlamp.angri.ru/ License: BSD. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Can't determine join between 'engines' and 'versions'. Please specify the 'onclause' of this join explicitly ?
I wrote in my first post, that I am expecting result in SQL - close to this one select e.id, e.type_id, tv.version as min_version, tv2.version as max_version from engines e join versions tv on(e.min_version_id=tv.id) join versions tv2 on(e.max_version_id=tv2.id) where '7.0.1.32' between tv.version and tv2.version;. I don't know how to call tables to check condition in between clause. On 22 Maj, 22:30, Michael Bayer mike...@zzzcomputing.com wrote: that means min_version and max_version aren't columns. I guess you're looking for Version.version, in which case you probably need to JOIN to that table twice on both the min_version and max_version. write (and test) the query you want in SQL first to get an idea for what you're doing. sniipe wrote: After change (delete 'version' attribute): ProgrammingError: (ProgrammingError) (1064, You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'AND' at line 3) u'SELECT engines.id AS engines_id, engines.min_version_id AS engines_min_version_id, engines.max_version_id AS engines_max_version_id, engines.type_id AS engines_type_id \nFROM engines LEFT OUTER JOIN versions ON engines.min_version_id = versions.id \nWHERE %s BETWEEN AND ' ['7.0.1.4'] On 22 Maj, 21:40, Michael Bayer mike...@zzzcomputing.com wrote: Engine.min_version and max_version are instrumented column attributes. they don't have an attribute called version. i think you want between(x, Engine.min_version, Engine.max_version). sniipe wrote: Ok It's working but I can't use 'between': engine = meta.Session.query(Engine).outerjoin((Version, Engine.min_version_id==Version.id)).filter(between(request.POST ['version'], Engine.min_version.version, Engine.max_version.version)).all() and I've got error: AttributeError: Neither 'InstrumentedAttribute' object nor 'Comparator' object has an attribute 'version' What am I doing wrong? On 22 Maj, 17:59, Michael Bayer mike...@zzzcomputing.com wrote: sniipe wrote: Hi :) I have three tables: 1) t_version = sa.Table(versions, meta.metadata, sa.Column(id, sa.types.Integer(), primary_key=True, autoincrement=True), sa.Column(version, mysql.MSChar(length=100, collation='utf8_polish_ci'), nullable=False, unique=True) ) class Version(object): pass orm.mapper(Version, t_version) 2) t_type = sa.Table(types, meta.metadata, sa.Column(id, sa.types.Integer(), primary_key=True, autoincrement=True), sa.Column(name, mysql.MSChar(length=100, collation='utf8_polish_ci'), nullable=False, unique=True), ) class Type(object): pass orm.mapper(Type, t_type, properties = { 'engine' : orm.relation(Engine, uselist=False, backref='Type_Engine') }) 3) t_engine = sa.Table(engines, meta.metadata, sa.Column(id, sa.types.Integer(), primary_key=True, autoincrement=True), sa.Column(min_version_id, sa.types.Integer(), sa.ForeignKey (versions.id), nullable=False), sa.Column(max_version_id, sa.types.Integer(), sa.ForeignKey (versions.id), nullable=False), sa.Column(type_id, sa.types.Integer(), sa.ForeignKey (types.id), nullable=False), ) class Engine(object): pass orm.mapper(Engine, t_engine, properties = { 'type' : orm.relation(Type, uselist=False, backref='Engine_Type'), 'min_version' : orm.relation(Version, primaryjoin=t_engine.c.min_version_id==t_version.c.id), 'max_version' : orm.relation(Version, primaryjoin=t_engine.c.max_version_id==t_version.c.id) }) My problem is how to make query equal this SQL instruction select e.id, e.type_id, tv.version as min_version, tv2.version as max_version from engines e join versions tv on(e.min_version_id=tv.id) join versions tv2 on(e.max_version_id=tv2.id) where '7.0.1.32' between tv.version and tv2.version; I've tried to do something like that: engine = meta.Session.query(Engine).outerjoin(Version).filter(between (request.POST['version'], Engine.min_version.version, Engine.max_version.version)).all() when you do the outerjoin, pass it the relation you're joining on or an onclause, such as query.outerjoin(Version.engine) or query.outerjoin((Version, Engine.some_col==Version.some_other_col)) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Q: fetch value of autoincrement column
On Friday 22 May 2009 23.00:05 Werner F. Bruhin wrote: What do you want to do with the autoincrement column? Often these are used for primary keys, which in turn get used as foreign keys. I want to use the id as filename; the table will cache some info that comes from the file. Using it as a filename directly has several advantages: * I don't have to care about a unique filename, since the db already takes care of that. * I don't have to create an additional column with the filename. Obviously, I can just first create the file with a temp filename, but I'd like to avoid this additional step if it's possible. Less code == fewer bugs. And I hoped that using the id from the beginning would be easily possible. Right now, I'm undecided if I should just drop portability and fetch the next value from the mytable_id_seq that postgres has generated for me with the autoincrement columns (from what I understand, even if I declare the column autoincrement, if I set the id manually it will Just Work(tm) like it should or if I should go the temporary filename route. I'm leaning towards the former because I probably will drop portability at some point in any case, since PostgreSQL just has too many cool features waiting to be used :-) (And no, I'm not going to use blobs and store the file as a large object in the db. Relational databases are not made for this, and besides it makes the db harder to maintain because db dumps get huge or the blobs have to be backed up separately, and debugging is harder because I can't use shell tools to get at the file data.) (So you see, I'm not trying to build foreign keys by hand or any such thing. I actually know relational databases quite well, it's just my knowledge of SQLAlchemy and Python which is lacking.) thanks -- vbi -- Alle Wirksamkeit ist stärker am Mittelpunkt als gegen die Peripherie zu. Raum zwischen Mars und Jupiter. -- Goethe, Maximen und Reflektionen, Nr. 1336 signature.asc Description: This is a digitally signed message part.
[sqlalchemy] Self Reference within Joined Inheritance
Dear Group, I have a problem with setting up self referential relations within a joined table inheritance scheme and declarative mapping. Let's say I have a base class B with a derived class S. S has a self-referential many-to-one relationship to B (and with that also to all of B's derived classes). The declarative definition seems to compile fine if a database system without native support for foreign keys is used (like sqlite), but breaks down with systems like InnoDB on mysql where foreign keys are supported natively (in the latter case I get a tables have more than one foreign key constraint relationship between them. Please specify the 'onclause' of this join explicitly error). So, the following testcase works on sqlite but fails if I move to SQL and InnoDB (after setting SQL server in the engine declaration and the InnoDB table in the __table_args__ of each class). Could anyone give me advice for setting up this kind of relation on InnoDB? Thanks a lot. -sven-eric import sqlalchemy from sqlalchemy import create_engine from sqlalchemy import String, Column, Integer, ForeignKey, ForeignKeyConstraint from sqlalchemy.orm import relation, backref from sqlalchemy.ext.declarative import declarative_base engine = create_engine('sqlite:///:memory:', echo=False) Base = declarative_base() # B is the superclas of S class B(Base): __tablename__ = 'b' id = Column(Integer, primary_key=True) type = Column(String(1)) __mapper_args__ = {'polymorphic_on': type, 'polymorphic_identity': 'b'} # S is a subclass of B and also references an object of the B hierarchy via a foreign key class S(B): __tablename__ = 's' __table_args__ = (ForeignKeyConstraint(['b_id'], ['b.id'])) __mapper_args__ = {'polymorphic_identity': 's'} id = Column(Integer, ForeignKey('b.id'), primary_key=True) b_id = Column(Integer) b = relation('B', backref='s_collection', primaryjoin='S.b_id==B.id') Base.metadata.create_all(engine) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] session.autocommit, session.begin double SessionExtension call
Hi, when from my pygtk application i commit, I really do:: if self.session.autocommit: self.session.begin() self.session.commit() I'm normally using session.autocommit = True as a mean to prevent all those 'idle in transaction' processes (that prevent me from changing the structure of the database - I use PostgreSQL). moreover I use autoflush=False to prevent flushing objects when I just need to ge more info from the database via normal 'select'. Everything seems to work nicely apart the fact that using this along with after_commit hook in SessionExtension, turns out in a double call of the hook. The first when I run session.begin() and the second when I run session.commit(). Is there a way to prevent the double call of the hook, i.e. double call of session.commit()? I tried issuing just self.begin() but that leaves the session with an open transaction (as long as I understand) and the second time I try it SA complains a transaction has already begun. Any hints on how to cope with this? sandro *:-) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Possible bug in orm/state.py
Hi, using scoped_session(sessionmaker()) to create my sessions, if I hammer a request (using Pylons) by repeatedly clicking on a link that uses the ORM somewhat extensively for the relevant request, it seems that another thread is getting involved with SQLAlchemy internals and pulling the rug out from under its feet. Here's the change I made to stop the exception from happening, but we're just wondering if I've done something else wrong or if this is an actual bug in SQLAlchemy: Index: state.py === --- state.py(revision 5974) +++ state.py(working copy) @@ -170,9 +170,14 @@ attr.impl.key in self.expired_attributes and attr.impl.key in unmodified ]) -for k in self.expired_attributes: -self.callables.pop(k, None) -del self.expired_attributes +try: +for k in self.expired_attributes: +self.callables.pop(k, None) +del self.expired_attributes +except AttributeError: +# XXX: self.expired_attributes can be del'ed by another thread +# which raises an AttributeError here +pass return ATTR_WAS_SET @property Here's the original traceback before the change: File '/home/bob/src/prospectspace/branches/sqlalchemy-eon-merge/ prospectspace/controllers/company.py', line 206 in index return self.render_response('company.mako', t_pars) File '/home/bob/src/prospectspace/branches/sqlalchemy-eon-merge/ prospectspace/lib/base.py', line 372 in render_response page = tmpl.render(**kargs) File '/home/bob/src/prospectspace/branches/sqlalchemy-eon-merge/lib/ python2.5/site-packages/Mako-0.1.8-py2.5.egg/mako/template.py', line 114 in render return runtime._render(self, self.callable_, args, data) File '/home/bob/src/prospectspace/branches/sqlalchemy-eon-merge/lib/ python2.5/site-packages/Mako-0.1.8-py2.5.egg/mako/runtime.py', line 287 in _render _render_context(template, callable_, context, *args, **_kwargs_for_callable(callable_, data)) File '/home/bob/src/prospectspace/branches/sqlalchemy-eon-merge/lib/ python2.5/site-packages/Mako-0.1.8-py2.5.egg/mako/runtime.py', line 304 in _render_context _exec_template(inherit, lclcontext, args=args, kwargs=kwargs) File '/home/bob/src/prospectspace/branches/sqlalchemy-eon-merge/lib/ python2.5/site-packages/Mako-0.1.8-py2.5.egg/mako/runtime.py', line 337 in _exec_template callable_(context, *args, **kwargs) File 'prospectmaster_mako', line 61 in render_body File '/home/bob/src/prospectspace/branches/sqlalchemy-eon-merge/lib/ python2.5/site-packages/SQLAlchemy-0.5.4p1-py2.5.egg/sqlalchemy/orm/ attributes.py', line 158 in __get__ return self.impl.get(instance_state(instance), instance_dict (instance)) File '/home/bob/src/prospectspace/branches/sqlalchemy-eon-merge/lib/ python2.5/site-packages/SQLAlchemy-0.5.4p1-py2.5.egg/sqlalchemy/orm/ attributes.py', line 374 in get value = callable_() File '/home/bob/src/prospectspace/branches/sqlalchemy-eon-merge/lib/ python2.5/site-packages/SQLAlchemy-0.5.4p1-py2.5.egg/sqlalchemy/orm/ state.py', line 175 in __call__ del self.expired_attributes AttributeError: expired_attributes And here's how I'm dealing with creating the sessions: threadlocal = threading.local() Session = scoped_session(sessionmaker(autocommit=True)) Session.metadata = None def setup_db(): if hasattr(threadlocal, 'engine'): return uri = config['main.engine.dburi'] threadlocal.engine = create_engine(uri) Session.configure(bind=threadlocal.engine) if Session.metadata is None: Session.metadata = MetaData(threadlocal.engine) model.initialise(Session.metadata) And then each request does this: setup_db() environ['dbsession'] = Session() The reason for including this bit is because I'm not convinced I'm doing it correctly, so I want to make sure that, if I am doing it wrong, this isn't what's causing the problem in SQLAlchemy, i.e. it may not be a bug at all. Please let me know if you need any more code/info - thanks a lot for any help. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Can't determine join between 'engines' and 'versions'. Please specify the 'onclause' of this join explicitly ?
On May 23, 2009, at 5:58 AM, sniipe wrote: I wrote in my first post, that I am expecting result in SQL - close to this one select e.id, e.type_id, tv.version as min_version, tv2.version as max_version from engines e join versions tv on(e.min_version_id=tv.id) join versions tv2 on(e.max_version_id=tv2.id) where '7.0.1.32' between tv.version and tv2.version;. I don't know how to call tables to check condition in between clause. do it just like SQL: tv = aliased(Version) tv2 = aliased(Version) query(Engine.id, Engine.type_id, tv.version.label('min_version'), tv2.version.label('max_version')).\ join((tv, Engine.min_version_id==tv.id)).\ join((tv2, Engine.max_version_id==tv2.id)).\ filter(literal(7.0.1.32).between(tv.version, tv2.version)) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Q: fetch value of autoincrement column
if its of any value, databases like MySQL and SQLIte do not give you any way to get at an auto-generated ID without actually INSERTing a row. Only databases that support sequences, i.e. postgres, firebird, and oracle, give you a built in way to get IDs without using INSERT. On May 23, 2009, at 8:25 AM, Adrian von Bidder wrote: On Friday 22 May 2009 23.00:05 Werner F. Bruhin wrote: What do you want to do with the autoincrement column? Often these are used for primary keys, which in turn get used as foreign keys. I want to use the id as filename; the table will cache some info that comes from the file. Using it as a filename directly has several advantages: * I don't have to care about a unique filename, since the db already takes care of that. * I don't have to create an additional column with the filename. Obviously, I can just first create the file with a temp filename, but I'd like to avoid this additional step if it's possible. Less code == fewer bugs. And I hoped that using the id from the beginning would be easily possible. Right now, I'm undecided if I should just drop portability and fetch the next value from the mytable_id_seq that postgres has generated for me with the autoincrement columns (from what I understand, even if I declare the column autoincrement, if I set the id manually it will Just Work(tm) like it should or if I should go the temporary filename route. I'm leaning towards the former because I probably will drop portability at some point in any case, since PostgreSQL just has too many cool features waiting to be used :-) (And no, I'm not going to use blobs and store the file as a large object in the db. Relational databases are not made for this, and besides it makes the db harder to maintain because db dumps get huge or the blobs have to be backed up separately, and debugging is harder because I can't use shell tools to get at the file data.) (So you see, I'm not trying to build foreign keys by hand or any such thing. I actually know relational databases quite well, it's just my knowledge of SQLAlchemy and Python which is lacking.) thanks -- vbi -- Alle Wirksamkeit ist stärker am Mittelpunkt als gegen die Peripherie zu. Raum zwischen Mars und Jupiter. -- Goethe, Maximen und Reflektionen, Nr. 1336 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Self Reference within Joined Inheritance
use inherit_condition: import sqlalchemy from sqlalchemy import create_engine from sqlalchemy import String, Column, Integer, ForeignKey from sqlalchemy.orm import relation, backref from sqlalchemy.ext.declarative import declarative_base engine = create_engine('mysql://scott:ti...@localhost/test', echo=True) Base = declarative_base() # B is the superclas of S class B(Base): __tablename__ = 'b' id = Column(Integer, primary_key=True) type = Column(String(1)) __mapper_args__ = {'polymorphic_on': type, 'polymorphic_identity':'b'} __table_args__ = {'mysql_engine':'InnoDB'} # S is a subclass of B and also references an object of the B hierarchy via a foreign key class S(B): __tablename__ = 's' id = Column(Integer, ForeignKey('b.id'), primary_key=True) b_id = Column(Integer, ForeignKey('b.id')) b = relation('B', backref='s_collection', primaryjoin='S.b_id==B.id') __table_args__ = {'mysql_engine':'InnoDB'} __mapper_args__ = {'polymorphic_identity': 's', 'inherit_condition':id==B.id} Base.metadata.create_all(engine) On May 23, 2009, at 9:51 AM, sven-eric wrote: Dear Group, I have a problem with setting up self referential relations within a joined table inheritance scheme and declarative mapping. Let's say I have a base class B with a derived class S. S has a self-referential many-to-one relationship to B (and with that also to all of B's derived classes). The declarative definition seems to compile fine if a database system without native support for foreign keys is used (like sqlite), but breaks down with systems like InnoDB on mysql where foreign keys are supported natively (in the latter case I get a tables have more than one foreign key constraint relationship between them. Please specify the 'onclause' of this join explicitly error). So, the following testcase works on sqlite but fails if I move to SQL and InnoDB (after setting SQL server in the engine declaration and the InnoDB table in the __table_args__ of each class). Could anyone give me advice for setting up this kind of relation on InnoDB? Thanks a lot. -sven-eric import sqlalchemy from sqlalchemy import create_engine from sqlalchemy import String, Column, Integer, ForeignKey, ForeignKeyConstraint from sqlalchemy.orm import relation, backref from sqlalchemy.ext.declarative import declarative_base engine = create_engine('sqlite:///:memory:', echo=False) Base = declarative_base() # B is the superclas of S class B(Base): __tablename__ = 'b' id = Column(Integer, primary_key=True) type = Column(String(1)) __mapper_args__ = {'polymorphic_on': type, 'polymorphic_identity': 'b'} # S is a subclass of B and also references an object of the B hierarchy via a foreign key class S(B): __tablename__ = 's' __table_args__ = (ForeignKeyConstraint(['b_id'], ['b.id'])) __mapper_args__ = {'polymorphic_identity': 's'} id = Column(Integer, ForeignKey('b.id'), primary_key=True) b_id = Column(Integer) b = relation('B', backref='s_collection', primaryjoin='S.b_id==B.id') Base.metadata.create_all(engine) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: session.autocommit, session.begin double SessionExtension call
On May 23, 2009, at 10:15 AM, Alessandro Dentella wrote: Hi, when from my pygtk application i commit, I really do:: if self.session.autocommit: self.session.begin() self.session.commit() I'm normally using session.autocommit = True as a mean to prevent all those 'idle in transaction' processes (that prevent me from changing the structure of the database - I use PostgreSQL). moreover I use autoflush=False to prevent flushing objects when I just need to ge more info from the database via normal 'select'. Everything seems to work nicely apart the fact that using this along with after_commit hook in SessionExtension, turns out in a double call of the hook. The first when I run session.begin() and the second when I run session.commit(). I can't see how that occurs. Can you please post a stack trace ? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Possible bug in orm/state.py
On May 23, 2009, at 1:47 PM, Bob Farrell wrote: Hi, using scoped_session(sessionmaker()) to create my sessions, if I hammer a request (using Pylons) by repeatedly clicking on a link that uses the ORM somewhat extensively for the relevant request, it seems that another thread is getting involved with SQLAlchemy internals and pulling the rug out from under its feet. that means you are sharing a mapped instance between threads.A mapped instance, when associated with a Session (i.e. not detatched), should be considered as an extension of that Session's internal state. The Session isn't threadsafe so you can't share a persistent instance between threads. If you are using some kind of persistent/ cached instances, make sure they are detatched from their original Session first, or merge() the instances in to the Session local to the request before using them (you can send the dont_load=True flag to merge() if you want to cut down on potentially needless SELECT statements). And here's how I'm dealing with creating the sessions: threadlocal = threading.local() Session = scoped_session(sessionmaker(autocommit=True)) Session.metadata = None def setup_db(): if hasattr(threadlocal, 'engine'): return uri = config['main.engine.dburi'] threadlocal.engine = create_engine(uri) Session.configure(bind=threadlocal.engine) if Session.metadata is None: Session.metadata = MetaData(threadlocal.engine) model.initialise(Session.metadata) the threading.local() is unnecessary...unless you are planning for the same application to be run with different .ini files in each thread which would be extremely unusual.scoped_session() already handles the thread local part for you as far as Sessions are concerned, and Engine objects are threadsafe. Session = scoped_session(sessionmaker(autocommit=True)) metadata = None def setup_db(): global metadata if metadata is not None: return uri = config['main.engine.dburi'] engine = create_engine(uri) Session.configure(bind=engine) metadata = MetaData(engine) model.initialise(metadata) if OTOH you had some reason for the threadlocal engines, then you dont want to use Session.configure, which configures the whole scoped_session(). You'd want to say Session(bind=my_threadlocal_engine). --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---