[sqlalchemy] Re: sqlalchemy rocks my socks off!
+1 On Jan 13, 5:08 pm, rdlowrey rdlow...@gmail.com wrote: To Michael Bayer: sqlalchemy simplifies my life every day and makes me vastly more productive! Many thanks. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: Objects are wrongly getting removed/expunged from session
On Jan 15, 11:56 am, Tvrtko qvx3...@gmail.com wrote: First, let me say that I'm using sqlalchemy version 0.5.8. We have a large and complex application and changing to sa 0.6 is expensive, so I would like, if possible, to find a solution for 0.5. I'm using sqlalchemy since version 0.1. Problem: objects are one moment inside a session an the other moment are not. Setup: Session is in weak_identity_map=False mode: maker = sessionmaker(autoflush=True, autocommit=False, weak_identity_map=False, extension=[ZopeTransactionExtension(), EpiSessionExtension()]) DBSession = scoped_session(maker) All objects are immediately added to session: campaign = Campaign(...) DBSession.add(campaign) Plus, the reference to object is kept locally. Plus, the object is assigned to another relation property: qitem = QItem(campaign=campaign, ...) DBSession.add(qitem) Now, the problem occurs at two different random points. Both time the garbage collector is collecting the garbage before the problem presents itself. This may have nothing to do with a problem. Everything happens inside a single thread. First case: campaign = Campaign(...) DBSession.add(campaign) ... # called indirectly because of lots of queries DBSession.flush() print campaign.id # 206 # A: the (Campaign, 206) is inside the keys print DBSession.identity_map.keys() contact = DBSession.query(...).one() # garbage is collected while this runs # I can see it because of gc.set_debug(...) # However, campaign is referenced locally and gc # leaves it, but maybe some other internal SA object # is collected. # B: the (Campaign, 206) is no longer present print DBSession.identity_map.keys() Actually, this case is the same as the second case, so let's just ignore it. The objects are not expunged from the session without a reason as the example might suggest. Objects are removed because the automatic `flush()` inside `query()` failed and rolled back the transaction (and removed objects from session). B print was done inside exception handler. So, it really is the same case as the second one: Second case, very similar, but problem appears at a later point: campaign = Campaign(...) DBSession.add(campaign) DBSession.flush() print campaign.id # 207 contact = DBSession.query(...).one() # Also, assign campaign to relational property of `qitem` qitem = QItem(campaign=campaign, ...) DBSession.add(qitem) # The (Campaign, 207) is inside the keys print DBSession.identity_map.keys() DBSession.flush() # called via commit by web controller # DB raises exception CONTACT_ID cannot be NULL Somehow, the `qitem.campaign` relation property behaves as if it is not set at all, and SA doesn't prepare `QITEM.CAMPAIGN_ID` properly. The exception is raised because the `campaign` object again got expunged from session for some reason, even though the `qitem` itself holds a reference to it. Well, maybe it wasn't expunged at that point, I dont have a proof, but the `CAMPAIGN_ID` was set to NULL as if the `campaign` property was not set. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: Objects are wrongly getting removed/expunged from session
I've drilled down all the way up to `mapper._save_obj()`. I examined
the `state.dict` for `qitem`.
Good case:
qitem state dict = {'campaign': Campaign 233, u'Test',
'campaign_id': 233, ...}
And for the bad case:
qitem state dict = {'campaign': Campaign 234, u'Test',
'campaign_id': None, ...}
The `campaign` property is *relation* to `Campaign` entity. Campaign
is previously flushed and has id. But, for some reason the
`campaign_id` is missing inside `qitem` state event though the
`campaign` is present.
I can probably work around by saying:
qitem.campaign_id = campaign.id
but then I must rely on campaign being flushed which might not allways
be the case. Yes, I can flush it manually, but I like to depend on
relations functioning properly.
P.S. I am not restarting or changing application in any way, just
refreshing a web page, which sometime fails and sometime succeeds.
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] one to may Commit
Hello.
A have this model.
How to make Order() and Orderitems() objects during one commit()
I'm trying
no = Order()
noi = Orderitem()
no.orderitems.append(noi)
Session.add(no)
Session.commit()
but get an error
IntegrityError: (IntegrityError) null value in column order_id
violates not-null constraint 'INSERT INTO
orders_table = Table('orders', metadata,
Column('id', types.Integer, primary_key=True, autoincrement=True),
Column('created', types.DateTime, default=now()),
)
orderitems_table = Table('orderitems', metadata,
Column('id', types.Integer, primary_key=True, autoincrement=True),
Column('order_id', types.Integer, ForeignKey('orders.id'),
nullable=False),
Column('sum', types.Float, default=0),
Column('created', types.DateTime, default=now()),
)
class Order(BaseModel):
pass
class Orderitem(BaseModel):
pass
mapper(Order, orders_table,
properties={'orderitems': relation(Orderitem, backref='order',
primaryjoin=orderitems_table.c.order_id==orders_table.c.id),
})
mapper(Orderitem, orderitems_table)
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Re: Objects are wrongly getting removed/expunged from session
So, you have A-B, an exception occurs, you do the rollback, then you're
somehow trying to continue on within the web request ? The code examples
here are out of context snippets and I don't see any exception handling
happening so they don't really tell me much.When the flush fails, the
internal state is completely expired. Campaign, if pending when the
transaction started, now transient again, qitem is also transient. Previous
flushes within that transaction have no effect on this.Campaign and qitem
would retain their relationship to each other, campaign_id would probably
remain present, but if you were to re-add the two, it would be overwritten on
the next flush.
In my development, if a web request has an exception, that web request is over,
so the use case here is a little confusing to me to start with, and I don't
really understand why you need to manually flush or not since I don't have a
clear illustration of the issue.
On Jan 15, 2011, at 9:27 AM, Tvrtko wrote:
I've drilled down all the way up to `mapper._save_obj()`. I examined
the `state.dict` for `qitem`.
Good case:
qitem state dict = {'campaign': Campaign 233, u'Test',
'campaign_id': 233, ...}
And for the bad case:
qitem state dict = {'campaign': Campaign 234, u'Test',
'campaign_id': None, ...}
The `campaign` property is *relation* to `Campaign` entity. Campaign
is previously flushed and has id. But, for some reason the
`campaign_id` is missing inside `qitem` state event though the
`campaign` is present.
I can probably work around by saying:
qitem.campaign_id = campaign.id
but then I must rely on campaign being flushed which might not allways
be the case. Yes, I can flush it manually, but I like to depend on
relations functioning properly.
P.S. I am not restarting or changing application in any way, just
refreshing a web page, which sometime fails and sometime succeeds.
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: Objects are wrongly getting removed/expunged from session
On Jan 15, 4:35 pm, Michael Bayer mike...@zzzcomputing.com wrote:
So, you have A-B, an exception occurs, you do the rollback, then you're
somehow trying to continue on within the web request ? The code examples
here are out of context snippets and I don't see any exception handling
happening so they don't really tell me much. When the flush fails, the
internal state is completely expired. Campaign, if pending when the
transaction started, now transient again, qitem is also transient. Previous
flushes within that transaction have no effect on this. Campaign and qitem
would retain their relationship to each other, campaign_id would probably
remain present, but if you were to re-add the two, it would be overwritten on
the next flush.
You are right. I presented the issues in a confusing manner because I
was not exactly sure what was happening in the first place.
I am *not* continuing with request after exception. I was reraising
exception after a debug print. Unfortunately that debug print has lead
me in the wrong direction of thinking that objects were expunged
before the flush. That is not the case.
What actually happens is that save mechanism sometimes gets underlying
ID from related object end sometimes it doesn't. Note that campaign
has ID, and that campaign is associated with qitem and still,
qitem.contact_id is not allways set during a save/flush.
Debug print from inside `mapper._save_obj()`:
Good case:
qitem state dict = {'campaign': Campaign 233, u'Test',
'campaign_id': 233, ...}
And for the bad case:
qitem state dict = {'campaign': Campaign 234, u'Test',
'campaign_id': None, ...}
In my development, if a web request has an exception, that web request is
over, so the use case here is a little confusing to me to start with, and I
don't really understand why you need to manually flush or not since I don't
have a clear illustration of the issue.
I never explicitly flush. I don't like it. I was only saying that if I
want to make a *workaround* and say:
qitem.campaign = campaign # sometimes has no effect
qitem.campaign_id = campaign.id # need flush for id
I have to call flush() before I have an ID.
On Jan 15, 2011, at 9:27 AM, Tvrtko wrote:
I've drilled down all the way up to `mapper._save_obj()`. I examined
the `state.dict` for `qitem`.
Good case:
qitem state dict = {'campaign': Campaign 233, u'Test',
'campaign_id': 233, ...}
And for the bad case:
qitem state dict = {'campaign': Campaign 234, u'Test',
'campaign_id': None, ...}
The `campaign` property is *relation* to `Campaign` entity. Campaign
is previously flushed and has id. But, for some reason the
`campaign_id` is missing inside `qitem` state event though the
`campaign` is present.
I can probably work around by saying:
qitem.campaign_id = campaign.id
but then I must rely on campaign being flushed which might not allways
be the case. Yes, I can flush it manually, but I like to depend on
relations functioning properly.
P.S. I am not restarting or changing application in any way, just
refreshing a web page, which sometime fails and sometime succeeds.
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group
athttp://groups.google.com/group/sqlalchemy?hl=en.
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] one to may Commit
More information is needed. Test case, psycopg2, SQLA 0.6.6, works fine:
from sqlalchemy import *
from sqlalchemy.orm import *
metadata= MetaData()
orders_table = Table('orders', metadata,
Column('id', Integer, primary_key=True, autoincrement=True),
Column('created', DateTime, default=func.now()),
)
orderitems_table = Table('orderitems', metadata,
Column('id', Integer, primary_key=True, autoincrement=True),
Column('order_id', Integer, ForeignKey('orders.id'),nullable=False),
Column('sum', Float, default=0),
Column('created', DateTime, default=func.now()),
)
class Order(object):
pass
class Orderitem(object):
pass
mapper(Order, orders_table,
properties={
'orderitems': relation(Orderitem, backref='order',
primaryjoin=orderitems_table.c.order_id==orders_table.c.id),
})
mapper(Orderitem, orderitems_table)
e = create_engine('postgresql://scott:tiger@localhost/test', echo=True)
metadata.drop_all(e)
metadata.create_all(e)
session = Session(e)
no = Order()
noi = Orderitem()
no.orderitems.append(noi)
session.add(no)
session.commit()
output of the commit:
BEGIN (implicit)
INSERT INTO orders (created) VALUES (now()) RETURNING orders.id
{}
INSERT INTO orderitems (order_id, sum, created) VALUES (%(order_id)s, %(sum)s,
now()) RETURNING orderitems.id
{'order_id': 1, 'sum': 0}
COMMIT
On Jan 15, 2011, at 4:02 AM, borisov wrote:
Hello.
A have this model.
How to make Order() and Orderitems() objects during one commit()
I'm trying
no = Order()
noi = Orderitem()
no.orderitems.append(noi)
Session.add(no)
Session.commit()
but get an error
IntegrityError: (IntegrityError) null value in column order_id
violates not-null constraint 'INSERT INTO
orders_table = Table('orders', metadata,
Column('id', types.Integer, primary_key=True, autoincrement=True),
Column('created', types.DateTime, default=now()),
)
orderitems_table = Table('orderitems', metadata,
Column('id', types.Integer, primary_key=True, autoincrement=True),
Column('order_id', types.Integer, ForeignKey('orders.id'),
nullable=False),
Column('sum', types.Float, default=0),
Column('created', types.DateTime, default=now()),
)
class Order(BaseModel):
pass
class Orderitem(BaseModel):
pass
mapper(Order, orders_table,
properties={'orderitems': relation(Orderitem, backref='order',
primaryjoin=orderitems_table.c.order_id==orders_table.c.id),
})
mapper(Orderitem, orderitems_table)
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Re: Objects are wrongly getting removed/expunged from session
On Jan 15, 2011, at 11:10 AM, Tvrtko wrote:
On Jan 15, 4:35 pm, Michael Bayer mike...@zzzcomputing.com wrote:
So, you have A-B, an exception occurs, you do the rollback, then you're
somehow trying to continue on within the web request ? The code examples
here are out of context snippets and I don't see any exception handling
happening so they don't really tell me much.When the flush fails, the
internal state is completely expired. Campaign, if pending when the
transaction started, now transient again, qitem is also transient.
Previous flushes within that transaction have no effect on this.Campaign
and qitem would retain their relationship to each other, campaign_id would
probably remain present, but if you were to re-add the two, it would be
overwritten on the next flush.
You are right. I presented the issues in a confusing manner because I
was not exactly sure what was happening in the first place.
I am *not* continuing with request after exception. I was reraising
exception after a debug print. Unfortunately that debug print has lead
me in the wrong direction of thinking that objects were expunged
before the flush. That is not the case.
What actually happens is that save mechanism sometimes gets underlying
ID from related object end sometimes it doesn't. Note that campaign
has ID, and that campaign is associated with qitem and still,
qitem.contact_id is not allways set during a save/flush.
Debug print from inside `mapper._save_obj()`:
Good case:
qitem state dict = {'campaign': Campaign 233, u'Test',
'campaign_id': 233, ...}
And for the bad case:
qitem state dict = {'campaign': Campaign 234, u'Test',
'campaign_id': None, ...}
OK, then I need a full reproducing test case which shows how that result
occurs.Its not a known issue off the top of my head. Also, please note
that I am in no way suggesting you upgrade your application to 0.6, however, if
you test your development code with 0.6 and the problem goes away, that would
indicate something specific to 0.5 might be the cause.I would suggest that
it may be related to how your relationships are set up, such that a dependency
between QItem and Campaign is not correctly established, leading the ORM to
sometimes handle one or the other first.
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: Objects are wrongly getting removed/expunged from session
This is my first such case in 5-6 years of using the SA. Usually the
problem was with my code. This could also be the case now, but it
escapes me where I made a mistake.
Thank you for your time. I will consider this closed for now and move
on using a workaround of explicitly populating campaign_id. I guess I
will try 0.6 or even 0.7 in the next few months.
Thanks,
Tvrtko
P.S. for the curious, my relations go like this:
from elixir import Entity, Field, ...
class Campaign(Entity):
id = Field(Integer, Sequence('cc_campaign_seq'),
colname='campaign_id', primary_key=True)
qitems = OneToMany('QItem')
class QItem(Entity):
id = Field(Integer, Sequence('cc_qitem_seq'), colname='qitem_id',
primary_key=True)
campaign = ManyToOne('Campaign', colname='campaign_id',
required=True)
On Jan 15, 5:32 pm, Michael Bayer mike...@zzzcomputing.com wrote:
On Jan 15, 2011, at 11:10 AM, Tvrtko wrote:
On Jan 15, 4:35 pm, Michael Bayer mike...@zzzcomputing.com wrote:
So, you have A-B, an exception occurs, you do the rollback, then you're
somehow trying to continue on within the web request ? The code
examples here are out of context snippets and I don't see any exception
handling happening so they don't really tell me much. When the flush
fails, the internal state is completely expired. Campaign, if pending
when the transaction started, now transient again, qitem is also
transient. Previous flushes within that transaction have no effect on
this. Campaign and qitem would retain their relationship to each other,
campaign_id would probably remain present, but if you were to re-add the
two, it would be overwritten on the next flush.
You are right. I presented the issues in a confusing manner because I
was not exactly sure what was happening in the first place.
I am *not* continuing with request after exception. I was reraising
exception after a debug print. Unfortunately that debug print has lead
me in the wrong direction of thinking that objects were expunged
before the flush. That is not the case.
What actually happens is that save mechanism sometimes gets underlying
ID from related object end sometimes it doesn't. Note that campaign
has ID, and that campaign is associated with qitem and still,
qitem.contact_id is not allways set during a save/flush.
Debug print from inside `mapper._save_obj()`:
Good case:
qitem state dict = {'campaign': Campaign 233, u'Test',
'campaign_id': 233, ...}
And for the bad case:
qitem state dict = {'campaign': Campaign 234, u'Test',
'campaign_id': None, ...}
OK, then I need a full reproducing test case which shows how that result
occurs. Its not a known issue off the top of my head. Also, please note
that I am in no way suggesting you upgrade your application to 0.6, however,
if you test your development code with 0.6 and the problem goes away, that
would indicate something specific to 0.5 might be the cause. I would
suggest that it may be related to how your relationships are set up, such
that a dependency between QItem and Campaign is not correctly established,
leading the ORM to sometimes handle one or the other first.
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Re: Objects are wrongly getting removed/expunged from session
On Jan 15, 2011, at 11:53 AM, Tvrtko wrote:
This is my first such case in 5-6 years of using the SA. Usually the
problem was with my code. This could also be the case now, but it
escapes me where I made a mistake.
not suggesting you made a mistake. suggesting that some artifact of your
relationship setup may be causing the UOW to make the wrong decision. the
setup below seems very simple so it remains a mystery what would be causing the
issue.
Thank you for your time. I will consider this closed for now and move
on using a workaround of explicitly populating campaign_id. I guess I
will try 0.6 or even 0.7 in the next few months.
Thanks,
Tvrtko
P.S. for the curious, my relations go like this:
from elixir import Entity, Field, ...
class Campaign(Entity):
id = Field(Integer, Sequence('cc_campaign_seq'),
colname='campaign_id', primary_key=True)
qitems = OneToMany('QItem')
class QItem(Entity):
id = Field(Integer, Sequence('cc_qitem_seq'), colname='qitem_id',
primary_key=True)
campaign = ManyToOne('Campaign', colname='campaign_id',
required=True)
On Jan 15, 5:32 pm, Michael Bayer mike...@zzzcomputing.com wrote:
On Jan 15, 2011, at 11:10 AM, Tvrtko wrote:
On Jan 15, 4:35 pm, Michael Bayer mike...@zzzcomputing.com wrote:
So, you have A-B, an exception occurs, you do the rollback, then you're
somehow trying to continue on within the web request ? The code
examples here are out of context snippets and I don't see any exception
handling happening so they don't really tell me much.When the flush
fails, the internal state is completely expired. Campaign, if pending
when the transaction started, now transient again, qitem is also
transient. Previous flushes within that transaction have no effect on
this.Campaign and qitem would retain their relationship to each other,
campaign_id would probably remain present, but if you were to re-add the
two, it would be overwritten on the next flush.
You are right. I presented the issues in a confusing manner because I
was not exactly sure what was happening in the first place.
I am *not* continuing with request after exception. I was reraising
exception after a debug print. Unfortunately that debug print has lead
me in the wrong direction of thinking that objects were expunged
before the flush. That is not the case.
What actually happens is that save mechanism sometimes gets underlying
ID from related object end sometimes it doesn't. Note that campaign
has ID, and that campaign is associated with qitem and still,
qitem.contact_id is not allways set during a save/flush.
Debug print from inside `mapper._save_obj()`:
Good case:
qitem state dict = {'campaign': Campaign 233, u'Test',
'campaign_id': 233, ...}
And for the bad case:
qitem state dict = {'campaign': Campaign 234, u'Test',
'campaign_id': None, ...}
OK, then I need a full reproducing test case which shows how that result
occurs.Its not a known issue off the top of my head. Also, please note
that I am in no way suggesting you upgrade your application to 0.6, however,
if you test your development code with 0.6 and the problem goes away, that
would indicate something specific to 0.5 might be the cause.I would
suggest that it may be related to how your relationships are set up, such
that a dependency between QItem and Campaign is not correctly established,
leading the ORM to sometimes handle one or the other first.
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: Objects are wrongly getting removed/expunged from session
On Jan 15, 5:57 pm, Michael Bayer mike...@zzzcomputing.com wrote: On Jan 15, 2011, at 11:53 AM, Tvrtko wrote: This is my first such case in 5-6 years of using the SA. Usually the problem was with my code. This could also be the case now, but it escapes me where I made a mistake. not suggesting you made a mistake. Don't worry. I was trying to give you an indirect compliment by saying the problem was most often with my own code :) I DO have unusual relationships in entire code base, but the Campaign/ QItem relationship is straightforward. One of the other relationships could be causing the problem. I'll see what happens with SA 0.7 when it comes out. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: Objects are wrongly getting removed/expunged from session
On Jan 15, 5:53 pm, Tvrtko qvx3...@gmail.com wrote:
This is my first such case in 5-6 years of using the SA. Usually the
problem was with my code. This could also be the case now, but it
escapes me where I made a mistake.
Thank you for your time. I will consider this closed for now and move
on using a workaround of explicitly populating campaign_id. I guess I
will try 0.6 or even 0.7 in the next few months.
I couldn't resist.
I'm pleased to say that it works just fine in SA 0.6
Thanks,
Tvrtko
P.S. for the curious, my relations go like this:
from elixir import Entity, Field, ...
class Campaign(Entity):
id = Field(Integer, Sequence('cc_campaign_seq'),
colname='campaign_id', primary_key=True)
qitems = OneToMany('QItem')
class QItem(Entity):
id = Field(Integer, Sequence('cc_qitem_seq'), colname='qitem_id',
primary_key=True)
campaign = ManyToOne('Campaign', colname='campaign_id',
required=True)
On Jan 15, 5:32 pm, Michael Bayer mike...@zzzcomputing.com wrote:
On Jan 15, 2011, at 11:10 AM, Tvrtko wrote:
On Jan 15, 4:35 pm, Michael Bayer mike...@zzzcomputing.com wrote:
So, you have A-B, an exception occurs, you do the rollback, then you're
somehow trying to continue on within the web request ? The code
examples here are out of context snippets and I don't see any exception
handling happening so they don't really tell me much. When the flush
fails, the internal state is completely expired. Campaign, if pending
when the transaction started, now transient again, qitem is also
transient. Previous flushes within that transaction have no effect on
this. Campaign and qitem would retain their relationship to each
other, campaign_id would probably remain present, but if you were to
re-add the two, it would be overwritten on the next flush.
You are right. I presented the issues in a confusing manner because I
was not exactly sure what was happening in the first place.
I am *not* continuing with request after exception. I was reraising
exception after a debug print. Unfortunately that debug print has lead
me in the wrong direction of thinking that objects were expunged
before the flush. That is not the case.
What actually happens is that save mechanism sometimes gets underlying
ID from related object end sometimes it doesn't. Note that campaign
has ID, and that campaign is associated with qitem and still,
qitem.contact_id is not allways set during a save/flush.
Debug print from inside `mapper._save_obj()`:
Good case:
qitem state dict = {'campaign': Campaign 233, u'Test',
'campaign_id': 233, ...}
And for the bad case:
qitem state dict = {'campaign': Campaign 234, u'Test',
'campaign_id': None, ...}
OK, then I need a full reproducing test case which shows how that result
occurs. Its not a known issue off the top of my head. Also, please note
that I am in no way suggesting you upgrade your application to 0.6,
however, if you test your development code with 0.6 and the problem goes
away, that would indicate something specific to 0.5 might be the cause.
I would suggest that it may be related to how your relationships are set
up, such that a dependency between QItem and Campaign is not correctly
established, leading the ORM to sometimes handle one or the other first.
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Re: Objects are wrongly getting removed/expunged from session
On Jan 15, 2011, at 8:37 PM, Tvrtko wrote:
On Jan 15, 5:53 pm, Tvrtko qvx3...@gmail.com wrote:
This is my first such case in 5-6 years of using the SA. Usually the
problem was with my code. This could also be the case now, but it
escapes me where I made a mistake.
Thank you for your time. I will consider this closed for now and move
on using a workaround of explicitly populating campaign_id. I guess I
will try 0.6 or even 0.7 in the next few months.
I couldn't resist.
I'm pleased to say that it works just fine in SA 0.6
I rewrote the UOW entirely from scratch in 0.6. The last remaining
incompatibilities were gone by 0.6.4 or so.Wait til you see what it does in
0.7.
Thanks,
Tvrtko
P.S. for the curious, my relations go like this:
from elixir import Entity, Field, ...
class Campaign(Entity):
id = Field(Integer, Sequence('cc_campaign_seq'),
colname='campaign_id', primary_key=True)
qitems = OneToMany('QItem')
class QItem(Entity):
id = Field(Integer, Sequence('cc_qitem_seq'), colname='qitem_id',
primary_key=True)
campaign = ManyToOne('Campaign', colname='campaign_id',
required=True)
On Jan 15, 5:32 pm, Michael Bayer mike...@zzzcomputing.com wrote:
On Jan 15, 2011, at 11:10 AM, Tvrtko wrote:
On Jan 15, 4:35 pm, Michael Bayer mike...@zzzcomputing.com wrote:
So, you have A-B, an exception occurs, you do the rollback, then you're
somehow trying to continue on within the web request ? The code
examples here are out of context snippets and I don't see any exception
handling happening so they don't really tell me much.When the flush
fails, the internal state is completely expired. Campaign, if pending
when the transaction started, now transient again, qitem is also
transient. Previous flushes within that transaction have no effect on
this.Campaign and qitem would retain their relationship to each
other, campaign_id would probably remain present, but if you were to
re-add the two, it would be overwritten on the next flush.
You are right. I presented the issues in a confusing manner because I
was not exactly sure what was happening in the first place.
I am *not* continuing with request after exception. I was reraising
exception after a debug print. Unfortunately that debug print has lead
me in the wrong direction of thinking that objects were expunged
before the flush. That is not the case.
What actually happens is that save mechanism sometimes gets underlying
ID from related object end sometimes it doesn't. Note that campaign
has ID, and that campaign is associated with qitem and still,
qitem.contact_id is not allways set during a save/flush.
Debug print from inside `mapper._save_obj()`:
Good case:
qitem state dict = {'campaign': Campaign 233, u'Test',
'campaign_id': 233, ...}
And for the bad case:
qitem state dict = {'campaign': Campaign 234, u'Test',
'campaign_id': None, ...}
OK, then I need a full reproducing test case which shows how that result
occurs.Its not a known issue off the top of my head. Also, please note
that I am in no way suggesting you upgrade your application to 0.6,
however, if you test your development code with 0.6 and the problem goes
away, that would indicate something specific to 0.5 might be the cause.
I would suggest that it may be related to how your relationships are set
up, such that a dependency between QItem and Campaign is not correctly
established, leading the ORM to sometimes handle one or the other first.
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] attributes.get_history() from mapper extension's before_update()
Is it safe trust attributes.get_history(instance, attrname) from mapper extension's before_update()? I assume this is not reset until later? Kent -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Pass query with as parameter (avoid creating a method and hardcoding a query)
Hello list... I would like to allow the users to perform certain queries without me (or well... my server) knowing in advance what those queries are going to be (without hard-coding the query). For instance: I have a “Product” class. One of it's fields is manufacturer and another is model class Product(declarativeBase): def __init__(self): self.model = self.manufacturer = I would like the user be able to input an string with a query, such as “Product.model != 'foo' or Product.model != 'bar'” or: Product.model == 'foo' Product.manufacturer == 'bar' I have created a little Python module (queryTree) that tokenizes the string and generates a tree for that kind of queries. For the last one mentioned above, it would be something like: sqlalchemy.and_ / \ == == / \ / \ Product.model foo Product.manufacturer bar 1) The “” string can be converted to (stored as) the sqlalchemy.and_ method 2) The fields of Product are sqlalchemy.orm.synonym(s). If I pass my tree module the class I'm going to perform the query for, it can call getattr(cls, model) and get the synonym (I mean: get the Product.model synonym itself instead of the “model” string) 3) Equally, the comparators are get with getattr(Product, __eq__) or getattr(Product, __ne__) so I can store in the tree node the comparator function instead of the string “==” or “!=” But when I try to run the query: from mylibs.product import Product queryString = Product.model == 'foo' Product.manufacturer == 'bar' session.query(Product.Product).filter(queryTree.getQuery(queryString, Product.Product)) I get an exception: File /home/hbr/Documents/my-cms/backlib/product/ProductManager.py, line 62, in getByCustomFilter retval = Database.session.query(Product.Product).filter(queryTokenizer.getQuery()).all() File string, line 1, in lambda File /home/hbr/.buildout/eggs/SQLAlchemy-0.6.5-py2.6.egg/sqlalchemy/orm/query.py, line 52, in generate fn(self, *args[1:], **kw) File /home/hbr/.buildout/eggs/SQLAlchemy-0.6.5-py2.6.egg/sqlalchemy/orm/query.py, line 942, in filter filter() argument must be of type ArgumentError: filter() argument must be of type sqlalchemy.sql.ClauseElement or string I could easily modify my query generator module to get the query string...: “and_(Product.model == 'foo', Product.manufacturer == 'bar')” so a call to eval(“and_(Product.model == 'foo', Product.manufacturer == 'bar')”) would probably work, but I'd like to avoid the use of eval because of the security issues it usually implies. With some other tests, I've got some other exceptions that made me realize that I could possibly modify somehow the nodes of my tree until getting something that is accepted by MySQL as a valid query, but that's kind of cheating... I'd like to use pure SqlAlchemy if possible (I trust SqlAlchemy more than my programming skills) :-D Thank you in advance. Every hint will be deeply appreciated. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
