[sqlalchemy] Update from an identical joined table with SQLAlchemy
I have two tables in SQLAlchemy which are identical and I want to update one from the other where the rows have the same primary key, and I want to do it in an efficient way. I tried joining the tables on the primary key, but SQLAlchemy doesn't appear to support updates on joined tables at the moment (except using a subquery for every column, which was too inefficient). I'd be keen to hear any suggestions of a good way to do this in a way that is portable across different backends. Currently we have some raw sql (for MS SQL) that generates something like the following: update [dbo].[table_1] set col_1 = table_2.col_1 , col_2 = table_2.col_2 from [dbo].[table_1] inner join table_2 on ([dbo].[table_1].id_1 = table_2.id_1 and [dbo].[table_1].id_2 = table_2.id_2) Many thanks Ed -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out.
Re: [sqlalchemy] 0.7.10 build test failures
im trying to recall if I've seen that one before. how are tests being run? this error does not occur in any environment for me. On Jul 14, 2013, at 6:18 PM, Hans-Peter Jansen h...@urpla.net wrote: On Sonntag, 14. Juli 2013 17:38:12 Michael Bayer wrote: you can get that right here: www.sqlalchemy.org/changelog/CHANGES_0_7_11 Thanks, great. That fixed all tests, but this is left still: [ 156s] -- [ 156s] Ran 4075 tests in 148.973s [ 156s] [ 156s] OK (SKIP=140) [ 156s] Error in atexit._run_exitfuncs: [ 156s] Traceback (most recent call last): [ 156s] File /usr/lib64/python2.7/atexit.py, line 24, in _run_exitfuncs [ 156s] func(*targs, **kargs) [ 156s] File /usr/lib64/python2.7/multiprocessing/util.py, line 284, in _exit_function [ 156s] info('process shutting down') [ 156s] TypeError: 'NoneType' object is not callable [ 156s] Error in sys.exitfunc: [ 156s] Traceback (most recent call last): [ 156s] File /usr/lib64/python2.7/atexit.py, line 24, in _run_exitfuncs [ 156s] func(*targs, **kargs) [ 156s] File /usr/lib64/python2.7/multiprocessing/util.py, line 284, in _exit_function [ 156s] info('process shutting down') [ 156s] TypeError: 'NoneType' object is not callable Pete -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out.
Re: [sqlalchemy] Update from an identical joined table with SQLAlchemy
On Jul 15, 2013, at 11:06 AM, Ed Singleton singleto...@gmail.com wrote: I have two tables in SQLAlchemy which are identical and I want to update one from the other where the rows have the same primary key, and I want to do it in an efficient way. I tried joining the tables on the primary key, but SQLAlchemy doesn't appear to support updates on joined tables at the moment (except using a subquery for every column, which was too inefficient). SQLAlchemy supports this for those backends which also do, SQL Server is included, you just wouldn't use the JOIN keyword, instead use an implicit join. Example: addresses.update(). values(email_address=users.c.name). where(users.c.id == addresses.c.user_id). where(users.c.name == 'ed') -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out.
Re: [sqlalchemy] 0.7.10 build test failures
oh I know, you're running python setup.py test. I'd strongly recommend running with the sqla_nose.py runner directly, however this issue is a setuptools one, adding this to setup.py will fix: # Hack to prevent TypeError: 'NoneType' object is not callable error # in multiprocessing/util.py _exit_function when running `python # setup.py test` (see # http://www.eby-sarna.com/pipermail/peak/2010-May/003357.html) try: import multiprocessing except ImportError: pass Seems like this might not be in the current master so this can be added. On Jul 15, 2013, at 1:08 PM, Michael Bayer mike...@zzzcomputing.com wrote: im trying to recall if I've seen that one before. how are tests being run? this error does not occur in any environment for me. On Jul 14, 2013, at 6:18 PM, Hans-Peter Jansen h...@urpla.net wrote: On Sonntag, 14. Juli 2013 17:38:12 Michael Bayer wrote: you can get that right here: www.sqlalchemy.org/changelog/CHANGES_0_7_11 Thanks, great. That fixed all tests, but this is left still: [ 156s] -- [ 156s] Ran 4075 tests in 148.973s [ 156s] [ 156s] OK (SKIP=140) [ 156s] Error in atexit._run_exitfuncs: [ 156s] Traceback (most recent call last): [ 156s] File /usr/lib64/python2.7/atexit.py, line 24, in _run_exitfuncs [ 156s] func(*targs, **kargs) [ 156s] File /usr/lib64/python2.7/multiprocessing/util.py, line 284, in _exit_function [ 156s] info('process shutting down') [ 156s] TypeError: 'NoneType' object is not callable [ 156s] Error in sys.exitfunc: [ 156s] Traceback (most recent call last): [ 156s] File /usr/lib64/python2.7/atexit.py, line 24, in _run_exitfuncs [ 156s] func(*targs, **kargs) [ 156s] File /usr/lib64/python2.7/multiprocessing/util.py, line 284, in _exit_function [ 156s] info('process shutting down') [ 156s] TypeError: 'NoneType' object is not callable Pete -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out.
Re: [sqlalchemy] 0.7.10 build test failures
issue http://www.sqlalchemy.org/trac/ticket/2782 is added to merge this to all three branches On Jul 15, 2013, at 1:14 PM, Michael Bayer mike...@zzzcomputing.com wrote: oh I know, you're running python setup.py test. I'd strongly recommend running with the sqla_nose.py runner directly, however this issue is a setuptools one, adding this to setup.py will fix: # Hack to prevent TypeError: 'NoneType' object is not callable error # in multiprocessing/util.py _exit_function when running `python # setup.py test` (see # http://www.eby-sarna.com/pipermail/peak/2010-May/003357.html) try: import multiprocessing except ImportError: pass Seems like this might not be in the current master so this can be added. On Jul 15, 2013, at 1:08 PM, Michael Bayer mike...@zzzcomputing.com wrote: im trying to recall if I've seen that one before. how are tests being run? this error does not occur in any environment for me. On Jul 14, 2013, at 6:18 PM, Hans-Peter Jansen h...@urpla.net wrote: On Sonntag, 14. Juli 2013 17:38:12 Michael Bayer wrote: you can get that right here: www.sqlalchemy.org/changelog/CHANGES_0_7_11 Thanks, great. That fixed all tests, but this is left still: [ 156s] -- [ 156s] Ran 4075 tests in 148.973s [ 156s] [ 156s] OK (SKIP=140) [ 156s] Error in atexit._run_exitfuncs: [ 156s] Traceback (most recent call last): [ 156s] File /usr/lib64/python2.7/atexit.py, line 24, in _run_exitfuncs [ 156s] func(*targs, **kargs) [ 156s] File /usr/lib64/python2.7/multiprocessing/util.py, line 284, in _exit_function [ 156s] info('process shutting down') [ 156s] TypeError: 'NoneType' object is not callable [ 156s] Error in sys.exitfunc: [ 156s] Traceback (most recent call last): [ 156s] File /usr/lib64/python2.7/atexit.py, line 24, in _run_exitfuncs [ 156s] func(*targs, **kargs) [ 156s] File /usr/lib64/python2.7/multiprocessing/util.py, line 284, in _exit_function [ 156s] info('process shutting down') [ 156s] TypeError: 'NoneType' object is not callable Pete -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out.
[sqlalchemy] aliasing a CTE expression
(using sqlalchemy version 7.10 postgres), I'm attempting to join a CTE twice in the same select statement. see here http://www.sqlfiddle.com/#!12/c487e/14 for a SQLFiddle, which shows the full select statement which I'm trying to build but essentially the problem I'm facing is how to create aliases for the `WITH` such that a second join is able to reference the first -- the emboldened 'AS alias' below, highlights the problem. WITH semester_days AS ( ... blah ... ) SELECT ... blah ... FROM students as S JOIN semester_days *as SD_start* On SD_start.day_date = S.start_date JOIN semester_days *as SD_end* On SD_end.day_number = SD_start.day_number + (7 * S.n_weeks) Attempting to use: my_cte = select([ ... ]).cte(name='semester_days') followed by: cte_start = my_cte.alias('start_date') cte_end = my_cte.alias('end_date') ... in ether a .select_from() appended to the select clause, or in a `from_obj` within the clause, simply blows the `with` statement off the top of the select. (if that makes any sense) Is there any way to accomplish this? Many thanks, Rob -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out.
Re: [sqlalchemy] aliasing a CTE expression
On Jul 15, 2013, at 1:58 PM, Rob robert.sudwa...@gmail.com wrote: (using sqlalchemy version 7.10 postgres), I'm attempting to join a CTE twice in the same select statement. see here for a SQLFiddle, which shows the full select statement which I'm trying to build but essentially the problem I'm facing is how to create aliases for the `WITH` such that a second join is able to reference the first -- the emboldened 'AS alias' below, highlights the problem. WITH semester_days AS ( ... blah ... ) SELECT ... blah ... FROM students as S JOIN semester_days as SD_start On SD_start.day_date = S.start_date JOIN semester_days as SD_end On SD_end.day_number = SD_start.day_number + (7 * S.n_weeks) Attempting to use: my_cte = select([ ... ]).cte(name='semester_days') followed by: cte_start = my_cte.alias('start_date') cte_end = my_cte.alias('end_date') ... in ether a .select_from() appended to the select clause, or in a `from_obj` within the clause, simply blows the `with` statement off the top of the select. (if that makes any sense) Is there any way to accomplish this? I had to reconstruct your query fully in order to reproduce this, just for now make sure you use a version of the CTE without an extra alias() assigned, in addition to the aliased usage. That is, use alias(SD_end) but don't use alias(SD_start) (or vice versa). Ticket http://www.sqlalchemy.org/trac/ticket/2783 has been added. from sqlalchemy import * from sqlalchemy.orm import * from sqlalchemy.ext.declarative import declarative_base Base = declarative_base() class Semester(Base): __tablename__ = 'semesters' id = Column(Integer, primary_key=True) start_date = Column(Date) end_date = Column(Date) class Student(Base): __tablename__ = 'students' id = Column(Integer, primary_key=True) start_date = Column(Date) n_weeks = Column(Integer) S = Student.__table__.alias(S) s1 = select([ Semester.id.label(semester_id), func.generate_series( Semester.start_date, Semester.end_date, 1 day).label(day_date) ]).alias(day_series) semester_days = select([ s1.c.semester_id, func.row_number().over().label(day_number), s1.c.day_date]).order_by(s1.c.day_date).cte(semester_days) # if you alias this, then the CTE doesn't render SD_start = semester_days #.alias(SD_start) SD_end = semester_days.alias(SD_end) s2 = select([ S.c.id.label(student_id), S.c.start_date, SD_start.c.semester_id.label(start_semester_id), S.c.n_weeks, SD_end.c.day_date.label(end_date), SD_end.c.semester_id.label(end_semester_id) ]).select_from( S.join(SD_start, S.c.start_date == SD_start.c.day_date). join(SD_end, SD_end.c.day_number == SD_start.c.day_number + (7 * S.c.n_weeks)) ).order_by(S.c.start_date) print s2 -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out.
[sqlalchemy] Re: aliasing a CTE expression
Thank you *so* much... And apologies for not having included the object set up ... I thought it'd obscure more than it showed... But for the record, the `generate_series` and `row_number() over()` are spot on :D On Monday, 15 July 2013 18:58:00 UTC+1, Rob wrote: (using sqlalchemy version 7.10 postgres), I'm attempting to join a CTE twice in the same select statement. see here http://www.sqlfiddle.com/#!12/c487e/14 for a SQLFiddle, which shows the full select statement which I'm trying to build but essentially the problem I'm facing is how to create aliases for the `WITH` such that a second join is able to reference the first -- the emboldened 'AS alias' below, highlights the problem. WITH semester_days AS ( ... blah ... ) SELECT ... blah ... FROM students as S JOIN semester_days *as SD_start* On SD_start.day_date = S.start_date JOIN semester_days *as SD_end* On SD_end.day_number = SD_start.day_number + (7 * S.n_weeks) Attempting to use: my_cte = select([ ... ]).cte(name='semester_days') followed by: cte_start = my_cte.alias('start_date') cte_end = my_cte.alias('end_date') ... in ether a .select_from() appended to the select clause, or in a `from_obj` within the clause, simply blows the `with` statement off the top of the select. (if that makes any sense) Is there any way to accomplish this? Many thanks, Rob -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out.
[sqlalchemy] Query caching allows two objects representing same underlying data in session?
I'm having this weird problem using the query caching recipes in which two instances of a model representing the same underlying dataset will both get into the session. I know this is happening because I put all of the models in a set() and there are two instances with the same underlying database row id. I was under the impression that the session itself would handle the case that an object coming from the query cache is already in the session, preventing duplication. Is this not the case? Thanks, Amir -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out.
Re: [sqlalchemy] 0.7.10 build test failures
On Montag, 15. Juli 2013 13:19:55 Michael Bayer wrote: issue http://www.sqlalchemy.org/trac/ticket/2782 is added to merge this to all three branches patch for 0.7 attached to the ticket. tested-by: Hans-Peter Jansen h...@urpla.net Thanks, Mike. This nice experience will encourage me to report some more issues, and tackle upgrading the package to 0.8.xx. Cheers, Pete On Jul 15, 2013, at 1:14 PM, Michael Bayer mike...@zzzcomputing.com wrote: oh I know, you're running python setup.py test. I'd strongly recommend running with the sqla_nose.py runner directly, however this issue is a setuptools one, adding this to setup.py will fix: # Hack to prevent TypeError: 'NoneType' object is not callable error # in multiprocessing/util.py _exit_function when running `python # setup.py test` (see # http://www.eby-sarna.com/pipermail/peak/2010-May/003357.html) try: import multiprocessing except ImportError: pass Seems like this might not be in the current master so this can be added. On Jul 15, 2013, at 1:08 PM, Michael Bayer mike...@zzzcomputing.com wrote: im trying to recall if I've seen that one before. how are tests being run? this error does not occur in any environment for me. On Jul 14, 2013, at 6:18 PM, Hans-Peter Jansen h...@urpla.net wrote: On Sonntag, 14. Juli 2013 17:38:12 Michael Bayer wrote: you can get that right here: www.sqlalchemy.org/changelog/CHANGES_0_7_11 Thanks, great. That fixed all tests, but this is left still: [ 156s] -- [ 156s] Ran 4075 tests in 148.973s [ 156s] [ 156s] OK (SKIP=140) [ 156s] Error in atexit._run_exitfuncs: [ 156s] Traceback (most recent call last): [ 156s] File /usr/lib64/python2.7/atexit.py, line 24, in _run_exitfuncs [ 156s] func(*targs, **kargs) [ 156s] File /usr/lib64/python2.7/multiprocessing/util.py, line 284, in _exit_function [ 156s] info('process shutting down') [ 156s] TypeError: 'NoneType' object is not callable [ 156s] Error in sys.exitfunc: [ 156s] Traceback (most recent call last): [ 156s] File /usr/lib64/python2.7/atexit.py, line 24, in _run_exitfuncs [ 156s] func(*targs, **kargs) [ 156s] File /usr/lib64/python2.7/multiprocessing/util.py, line 284, in _exit_function [ 156s] info('process shutting down') [ 156s] TypeError: 'NoneType' object is not callable Pete -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out.
Re: [sqlalchemy] Query caching allows two objects representing same underlying data in session?
On Jul 15, 2013, at 4:51 PM, Amir Elaguizy aelag...@gmail.com wrote: I'm having this weird problem using the query caching recipes in which two instances of a model representing the same underlying dataset will both get into the session. I know this is happening because I put all of the models in a set() and there are two instances with the same underlying database row id. I was under the impression that the session itself would handle the case that an object coming from the query cache is already in the session, preventing duplication. Is this not the case? well you need to be using the merge() aspect of it, which will reconcile an existing identity that's already in the session. the recipe as written uses merge_result() so will ensure this, yes.This only deals with the identity map though, if you have an object pending with a given identity, its not in the identity map. I'd advise against heavy usage of cached queries overlapping with lots of pending objects within the same sets because things can get very crazy. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out.
Re: [sqlalchemy] Joinedload and duplicate relations
On Jul 13, 2013, at 11:37 AM, Paul Johnston paul@gmail.com wrote: Hi, I've just been debugging a slow running query: products = db.Product.query.options(sao.joinedload_all('variations.channels'), sao.joinedload_all('variations.specifics')).all() The second joinedload_all should just have been joinedload. It was causing variations to be included twice in the joinedload, presumably causing a cross product that made the query very slow. I can't imagine there's any legitimate need to include a relation twice, so it would be helpful if SQLAlchemy issued an error for this condition. joinedload()/joinedload_all() against somekey essentially puts a token in a dictionary that represents somekey. It's unique, and there's no mechanism by which it would include variations as two distinct FROM targets given the form you have above. There shouldn't be any issue saying joinedload_all(a.b)/joinedload_all(a.c). A simple test illustrates this, this works at least as far back as 0.7 if not further: from sqlalchemy import * from sqlalchemy.orm import * from sqlalchemy.ext.declarative import declarative_base Base = declarative_base() class A(Base): __tablename__ = 'a' id = Column(Integer, primary_key=True) bs = relationship(B) class B(Base): __tablename__ = 'b' id = Column(Integer, primary_key=True) a_id = Column(Integer, ForeignKey('a.id')) cs = relationship(C) ds = relationship(D) class C(Base): __tablename__ = 'c' id = Column(Integer, primary_key=True) b_id = Column(Integer, ForeignKey('b.id')) class D(Base): __tablename__ = 'd' id = Column(Integer, primary_key=True) b_id = Column(Integer, ForeignKey('b.id')) sess = Session() print sess.query(A).options(joinedload_all(bs.cs), joinedload_all(bs.ds)) in the output, we see that b is stated as a FROM target only once and is present in all three ON clauses: SELECT a.id AS a_id, c_1.id AS c_1_id, c_1.b_id AS c_1_b_id, d_1.id AS d_1_id, d_1.b_id AS d_1_b_id, b_1.id AS b_1_id, b_1.a_id AS b_1_a_id FROM a LEFT OUTER JOIN b AS b_1 ON a.id = b_1.a_id LEFT OUTER JOIN c AS c_1 ON b_1.id = c_1.b_id LEFT OUTER JOIN d AS d_1 ON b_1.id = d_1.b_id -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out.