Re: [sqlalchemy] session.query().get() is unsupported during flush for getting an object that was just added?
On Jan 26, 2012, at 11:28 AM, Kent Bower wrote: I think I understand why, during a flush(), if I use session.query().get() for an item that was just added during this flush, I don't get the persistent object I might expect because the session still has it as pending even though, logically, it is already persistent. I don't suppose you have any desire to support that, huh? The use case would be related to the future ticket http://www.sqlalchemy.org/trac/ticket/1939 (and http://www.sqlalchemy.org/trac/ticket/2350). Attached is a script demonstrating the issue I've hit. I can work around it with some difficulty, but I wanted your input and thoughts. No, there's no plans to support this case at all; you're using the Session inside of a mapper event, which is just not supported, and can never be due to the nature of the unit of work. The most recent docstrings try to be very explicit about this: http://docs.sqlalchemy.org/en/latest/orm/events.html#sqlalchemy.orm.events.MapperEvents.before_update I guess I have to add session.query() and get() in there as well. The way the flush works is not as straightforward as persist object A; persist object B; persist object C - that is, these are not atomic operations inside the flush.It's more like, Perform step X for objects A, B, and C; perform step Y for objects A, B and C. This is basically batching, and is necessary since it is vastly more efficient than atomically completing each object one at a time. Also, some decisions are needed by Y which can't always be made until X has completed for objects involved in dependencies. A side effect of batching is that if we provide a hook that emits after X and before Y, you're being exposed to the objects in an unusual state. Hence, the hooks that are in the middle like that are only intended to emit SQL on the given Connection; not to do anything ORM level beyond assigning column-based values on the immediate object.As always, before_flush() is where ORM-level manipulations are intended to be placed. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] session.query().get() is unsupported during flush for getting an object that was just added?
Fair enough. I had enough understanding of what must be going on to know flush isn't straightforward, but I'm still glad I asked. Sorry for having not read the documents very well and thanks for your answer, because from it, I surmise that before_flush() *is* safe for session operations, which is very good to understand more clearly. Thanks. On 1/26/2012 12:06 PM, Michael Bayer wrote: On Jan 26, 2012, at 11:28 AM, Kent Bower wrote: I think I understand why, during a flush(), if I use session.query().get() for an item that was just added during this flush, I don't get the persistent object I might expect because the session still has it as pending even though, logically, it is already persistent. I don't suppose you have any desire to support that, huh? The use case would be related to the future ticket http://www.sqlalchemy.org/trac/ticket/1939 (and http://www.sqlalchemy.org/trac/ticket/2350). Attached is a script demonstrating the issue I've hit. I can work around it with some difficulty, but I wanted your input and thoughts. No, there's no plans to support this case at all; you're using the Session inside of a mapper event, which is just not supported, and can never be due to the nature of the unit of work. The most recent docstrings try to be very explicit about this: http://docs.sqlalchemy.org/en/latest/orm/events.html#sqlalchemy.orm.events.MapperEvents.before_update I guess I have to add session.query() and get() in there as well. The way the flush works is not as straightforward as persist object A; persist object B; persist object C - that is, these are not atomic operations inside the flush.It's more like, Perform step X for objects A, B, and C; perform step Y for objects A, B and C. This is basically batching, and is necessary since it is vastly more efficient than atomically completing each object one at a time. Also, some decisions are needed by Y which can't always be made until X has completed for objects involved in dependencies. A side effect of batching is that if we provide a hook that emits after X and before Y, you're being exposed to the objects in an unusual state. Hence, the hooks that are in the middle like that are only intended to emit SQL on the given Connection; not to do anything ORM level beyond assigning column-based values on the immediate object.As always, before_flush() is where ORM-level manipulations are intended to be placed. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] session.query().get() is unsupported during flush for getting an object that was just added?
yup, before_flush is made for that, and I've for some time had some vague plans to add some more helpers there so you could get events local to certain kinds of objects in certain kinds of states, meaning it would look a lot like before_update. But looping through .new, .dirty, and .deleted is how to do it for now. On Jan 26, 2012, at 12:12 PM, Kent wrote: Fair enough. I had enough understanding of what must be going on to know flush isn't straightforward, but I'm still glad I asked. Sorry for having not read the documents very well and thanks for your answer, because from it, I surmise that before_flush() *is* safe for session operations, which is very good to understand more clearly. Thanks. On 1/26/2012 12:06 PM, Michael Bayer wrote: On Jan 26, 2012, at 11:28 AM, Kent Bower wrote: I think I understand why, during a flush(), if I use session.query().get() for an item that was just added during this flush, I don't get the persistent object I might expect because the session still has it as pending even though, logically, it is already persistent. I don't suppose you have any desire to support that, huh? The use case would be related to the future ticket http://www.sqlalchemy.org/trac/ticket/1939 (and http://www.sqlalchemy.org/trac/ticket/2350). Attached is a script demonstrating the issue I've hit. I can work around it with some difficulty, but I wanted your input and thoughts. No, there's no plans to support this case at all; you're using the Session inside of a mapper event, which is just not supported, and can never be due to the nature of the unit of work. The most recent docstrings try to be very explicit about this: http://docs.sqlalchemy.org/en/latest/orm/events.html#sqlalchemy.orm.events.MapperEvents.before_update I guess I have to add session.query() and get() in there as well. The way the flush works is not as straightforward as persist object A; persist object B; persist object C - that is, these are not atomic operations inside the flush.It's more like, Perform step X for objects A, B, and C; perform step Y for objects A, B and C. This is basically batching, and is necessary since it is vastly more efficient than atomically completing each object one at a time. Also, some decisions are needed by Y which can't always be made until X has completed for objects involved in dependencies. A side effect of batching is that if we provide a hook that emits after X and before Y, you're being exposed to the objects in an unusual state. Hence, the hooks that are in the middle like that are only intended to emit SQL on the given Connection; not to do anything ORM level beyond assigning column-based values on the immediate object.As always, before_flush() is where ORM-level manipulations are intended to be placed. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] session.query().get() is unsupported during flush for getting an object that was just added?
So, as a typical example of where it seems very natural to use
before_update, suppose you need to automatically update the not null
sequence of a related table. This but to get the sequence you need to
loop over the parent table's collection.
You want the sequence to be human friendly (natural primary key) and
also you want to be able to sort by sequence guaranteed in order without
the possibility of a database sequence wrap around. So you want the
sequence 1,2,3...
This seems extremely well fit for before_insert, like this:
==
parents_table = Table(parents, metadata,
Column(id, Integer, primary_key=True),
)
children_table = Table(children, metadata,
Column(parentid, Integer, ForeignKey('parents.id'),),
Column(sequence, Integer, primary_key=True),
)
class Parent(object):
pass
class Child(object):
pass
mapper(Parent, parents_table,
properties={'children': relationship(Child,
cascade='all,delete-orphan',
backref='parent')
})
mapper(Child, children_table)
@event.listens_for(Child, 'before_insert')
def set_sequence(mapper, connection, instance):
if instance.sequence is None:
instance.sequence = (max(c.sequence for c in
instance.parent.children) or 0) + 1
==
But this reaches across relationships, so that is actually not desired
here, is that correct?
For this, you would loop over session.new in before_update, is that how
you would approach this requirement?
On 1/26/2012 12:34 PM, Michael Bayer wrote:
yup, before_flush is made for that, and I've for some time had some vague plans
to add some more helpers there so you could get events local to certain kinds
of objects in certain kinds of states, meaning it would look a lot like
before_update. But looping through .new, .dirty, and .deleted is how to do it
for now.
On Jan 26, 2012, at 12:12 PM, Kent wrote:
Fair enough. I had enough understanding of what must be going on to know flush
isn't straightforward, but I'm still glad I asked. Sorry for having not read
the documents very well and thanks for your answer, because from it, I surmise
that before_flush() *is* safe for session operations, which is very good to
understand more clearly.
Thanks.
On 1/26/2012 12:06 PM, Michael Bayer wrote:
On Jan 26, 2012, at 11:28 AM, Kent Bower wrote:
I think I understand why, during a flush(), if I use session.query().get() for
an item that was just added during this flush, I don't get the persistent
object I might expect because the session still has it as pending even though,
logically, it is already persistent.
I don't suppose you have any desire to support that, huh? The use case would
be related to the future ticket http://www.sqlalchemy.org/trac/ticket/1939 (and
http://www.sqlalchemy.org/trac/ticket/2350).
Attached is a script demonstrating the issue I've hit. I can work around it
with some difficulty, but I wanted your input and thoughts.
No, there's no plans to support this case at all; you're using the Session
inside of a mapper event, which is just not supported, and can never be due to
the nature of the unit of work. The most recent docstrings try to be very
explicit about this:
http://docs.sqlalchemy.org/en/latest/orm/events.html#sqlalchemy.orm.events.MapperEvents.before_update
I guess I have to add session.query() and get() in there as well.
The way the flush works is not as straightforward as persist object A; persist object B;
persist object C - that is, these are not atomic operations inside the flush.It's more
like, Perform step X for objects A, B, and C; perform step Y for objects A, B and C.
This is basically batching, and is necessary since it is vastly more efficient than atomically
completing each object one at a time. Also, some decisions are needed by Y which can't always be
made until X has completed for objects involved in dependencies.
A side effect of batching is that if we provide a hook that emits after X and
before Y, you're being exposed to the objects in an unusual state. Hence, the
hooks that are in the middle like that are only intended to emit SQL on the
given Connection; not to do anything ORM level beyond assigning column-based
values on the immediate object.As always, before_flush() is where ORM-level
manipulations are intended to be placed.
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
Re: [sqlalchemy] session.query().get() is unsupported during flush for getting an object that was just added?
On Jan 26, 2012, at 1:21 PM, Kent wrote:
So, as a typical example of where it seems very natural to use
before_update, suppose you need to automatically update the not null
sequence of a related table. This but to get the sequence you need to loop
over the parent table's collection.
You want the sequence to be human friendly (natural primary key) and also
you want to be able to sort by sequence guaranteed in order without the
possibility of a database sequence wrap around. So you want the sequence
1,2,3...
This seems extremely well fit for before_insert, like this:
==
parents_table = Table(parents, metadata,
Column(id, Integer, primary_key=True),
)
children_table = Table(children, metadata,
Column(parentid, Integer, ForeignKey('parents.id'),),
Column(sequence, Integer, primary_key=True),
)
class Parent(object):
pass
class Child(object):
pass
mapper(Parent, parents_table,
properties={'children': relationship(Child,
cascade='all,delete-orphan',
backref='parent')
})
mapper(Child, children_table)
@event.listens_for(Child, 'before_insert')
def set_sequence(mapper, connection, instance):
if instance.sequence is None:
instance.sequence = (max(c.sequence for c in instance.parent.children)
or 0) + 1
==
But this reaches across relationships, so that is actually not desired here,
is that correct?
that is correct.
For this, you would loop over session.new in before_update, is that how you
would approach this requirement?
If the value is based on what's already been INSERTed for previous rows, I'd
emit a SQL statement to get at the value.If it's based on some kind of
natural consideration that isn't dependent on the outcome of an INSERT
statement, then you can do the looping above within the before_flush() event
and assign everything at once.Basically you need to batch the same way
the UOW itself does.
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
