Re: [sqlalchemy] Event listener for when query results get matched with pre-existing objects on the session

[Mike Bayer]

On Fri, Oct 20, 2023, at 10:46 AM, 'Tony Cosentini' via sqlalchemy wrote:
> Oh I see, thanks for clarifying.
> 
> I'm trying to detect cases where we depend on the autoflush behavior. For 
> example, in the sample above, when the query runs with no_autoflush, we won't 
> get back any results (when previously we would get back one row). It's a 
> fairly large codebase, so I was trying to automate finding these cases in 
> order to add explicit flush calls when needed or just pass in the pending 
> object instead of running a query.


there's not really an event for an object that just gets returned by a query 
from the identity map.you would need to do something more drastic like a 
do_orm_execute() hook that runs queries internally and then looks at all the 
returned objects.  i wouldnt put that in production.







> 
> Tony
> 
> On Fri, Oct 20, 2023 at 10:42 PM Mike Bayer 
>  wrote:
>> __
>> 
>> 
>> On Fri, Oct 20, 2023, at 9:50 AM, 'Tony Cosentini' via sqlalchemy wrote:
>>> Weird, I did try that but I can't seem to trigger it.
>>> 
>>> Here's a self-contained test: 
>>> https://gist.github.com/tonycosentini/4dee3478695d032ca67707b5e26739b6
>> 
>> 
>> the object was not affected in that query (that is, not mutated).  if you 
>> change the query to this:
>> 
>>  table_one_instances = session.query(TableOne).populate_existing().all()
>> 
>> then the refresh event is triggered.
>> 
>> Im not really following what kind of bug you are trying to detect.
>> 
>> 
>>> 
>>> I have the event listener set to just crash, but it never triggers.
>>> 
>>> On Fri, Oct 20, 2023 at 9:20 PM Mike Bayer 
>>>  wrote:
 __
 
 
 On Fri, Oct 20, 2023, at 8:08 AM, 'Tony Cosentini' via sqlalchemy wrote:
> Hi,
> 
> Is there any way to listen for an event for when a query result gets 
> merged into a pre-existing object in the session?
 
 this is the refresh event:
 
 https://docs.sqlalchemy.org/en/20/orm/events.html#sqlalchemy.orm.InstanceEvents.refresh
 
 
> 
> I'm working on disabling autoflush for some of our codebase (mostly to 
> cut down on queries on a high QPS path), but before doing that I want to 
> write a utility to detect when we read data that was flushed via 
> autoflush to cut down on any potential bugs.
> 
> What I'd like to do is this (while autoflush is still enabled):
> 
> table_one = TableOne(name='test')
> session.add(table_one) # Track this object as pending a flush via the 
> before_* listeners, this is working as expected.
> 
> table_two_instances = session.query(TableTwo).all() # All good, doesn't 
> do anything with the table_one instance created earlier
> 
> table_one_instances = session.query(TableOne).all() # I would like to log 
> a warning here as the results of this query depend on a flush happening. 
> What I'm hoping to do is detect that one of the rows coming back is the 
> same object that was flagged earlier, but I can't see to find the correct 
> event to use.
 
 you probably want to also use refresh_flush also, which will be invoked 
 for column defaults that are populated on the object
 
 https://docs.sqlalchemy.org/en/20/orm/events.html#sqlalchemy.orm.InstanceEvents.refresh_flush
 
 
 
 
> 
> Is this possible? I can't seem to find the appropriate event that would 
> trigger when the results from the .all() query get merged back in with 
> the existing objects in the session.
 
 that's the refresh event
 
 
 
 
 --
 SQLAlchemy -
 The Python SQL Toolkit and Object Relational Mapper
  
 http://www.sqlalchemy.org/
  
 To post example code, please provide an MCVE: Minimal, Complete, and 
 Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
 description.
 ---
 You received this message because you are subscribed to a topic in the 
 Google Groups "sqlalchemy" group.
 To unsubscribe from this topic, visit 
 https://groups.google.com/d/topic/sqlalchemy/9bevy3Kpql4/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to 
 sqlalchemy+unsubscr...@googlegroups.com.
 To view this discussion on the web visit 
 https://groups.google.com/d/msgid/sqlalchemy/c5b89f67-e3fa-45f2-8774-463d154514f3%40app.fastmail.com
  
 .
>>> 
>>> 
>>> --
>>> SQLAlchemy -
>>> The Python SQL Toolkit and Object Relational Mapper
>>>  
>>> http://www.sqlalchemy.org/
>>>  
>>> To post example code, please provide an MCVE: Minimal, Complete, and 
>>> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
>>> description.
>>> ---
>>> You received this message because you are subscribed to the Google Groups 
>>> "sqlalchemy" group.
>>> To 

Re: [sqlalchemy] Event listener for when query results get matched with pre-existing objects on the session

['Tony Cosentini' via sqlalchemy]

Oh I see, thanks for clarifying.

I'm trying to detect cases where we depend on the autoflush behavior. For
example, in the sample above, when the query runs with no_autoflush, we
won't get back any results (when previously we would get back one row).
It's a fairly large codebase, so I was trying to automate finding these
cases in order to add explicit flush calls when needed or just pass in the
pending object instead of running a query.

Tony

On Fri, Oct 20, 2023 at 10:42 PM Mike Bayer <
mike_not_on_goo...@zzzcomputing.com> wrote:

>
>
> On Fri, Oct 20, 2023, at 9:50 AM, 'Tony Cosentini' via sqlalchemy wrote:
>
> Weird, I did try that but I can't seem to trigger it.
>
> Here's a self-contained test:
> https://gist.github.com/tonycosentini/4dee3478695d032ca67707b5e26739b6
>
>
>
> the object was not affected in that query (that is, not mutated).  if you
> change the query to this:
>
>  table_one_instances =
> session.query(TableOne).populate_existing().all()
>
> then the refresh event is triggered.
>
> Im not really following what kind of bug you are trying to detect.
>
>
>
> I have the event listener set to just crash, but it never triggers.
>
> On Fri, Oct 20, 2023 at 9:20 PM Mike Bayer <
> mike_not_on_goo...@zzzcomputing.com> wrote:
>
>
>
>
> On Fri, Oct 20, 2023, at 8:08 AM, 'Tony Cosentini' via sqlalchemy wrote:
>
> Hi,
>
> Is there any way to listen for an event for when a query result gets
> merged into a pre-existing object in the session?
>
>
> this is the refresh event:
>
>
> https://docs.sqlalchemy.org/en/20/orm/events.html#sqlalchemy.orm.InstanceEvents.refresh
>
>
>
> I'm working on disabling autoflush for some of our codebase (mostly to cut
> down on queries on a high QPS path), but before doing that I want to write
> a utility to detect when we read data that was flushed via autoflush to cut
> down on any potential bugs.
>
> What I'd like to do is this (while autoflush is still enabled):
>
> table_one = TableOne(name='test')
> session.add(table_one) # Track this object as pending a flush via the
> before_* listeners, this is working as expected.
>
> table_two_instances = session.query(TableTwo).all() # All good, doesn't do
> anything with the table_one instance created earlier
>
> table_one_instances = session.query(TableOne).all() # I would like to log
> a warning here as the results of this query depend on a flush happening.
> What I'm hoping to do is detect that one of the rows coming back is the
> same object that was flagged earlier, but I can't see to find the correct
> event to use.
>
>
> you probably want to also use refresh_flush also, which will be invoked
> for column defaults that are populated on the object
>
>
> https://docs.sqlalchemy.org/en/20/orm/events.html#sqlalchemy.orm.InstanceEvents.refresh_flush
>
>
>
>
>
> Is this possible? I can't seem to find the appropriate event that would
> trigger when the results from the .all() query get merged back in with the
> existing objects in the session.
>
>
> that's the refresh event
>
>
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "sqlalchemy" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/sqlalchemy/9bevy3Kpql4/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sqlalchemy/c5b89f67-e3fa-45f2-8774-463d154514f3%40app.fastmail.com
> 
> .
>
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sqlalchemy/CAEx_o%2BAhhUSPDYuGL4TFkTvxjHYfKEWUnBMxh2PzCDw3GJPaGg%40mail.gmail.com
> 
> .
>
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> 

Re: [sqlalchemy] Event listener for when query results get matched with pre-existing objects on the session

[Mike Bayer]

On Fri, Oct 20, 2023, at 9:50 AM, 'Tony Cosentini' via sqlalchemy wrote:
> Weird, I did try that but I can't seem to trigger it.
> 
> Here's a self-contained test: 
> https://gist.github.com/tonycosentini/4dee3478695d032ca67707b5e26739b6


the object was not affected in that query (that is, not mutated).  if you 
change the query to this:

 table_one_instances = session.query(TableOne).populate_existing().all()

then the refresh event is triggered.

Im not really following what kind of bug you are trying to detect.


> 
> I have the event listener set to just crash, but it never triggers.
> 
> On Fri, Oct 20, 2023 at 9:20 PM Mike Bayer 
>  wrote:
>> __
>> 
>> 
>> On Fri, Oct 20, 2023, at 8:08 AM, 'Tony Cosentini' via sqlalchemy wrote:
>>> Hi,
>>> 
>>> Is there any way to listen for an event for when a query result gets merged 
>>> into a pre-existing object in the session?
>> 
>> this is the refresh event:
>> 
>> https://docs.sqlalchemy.org/en/20/orm/events.html#sqlalchemy.orm.InstanceEvents.refresh
>> 
>> 
>>> 
>>> I'm working on disabling autoflush for some of our codebase (mostly to cut 
>>> down on queries on a high QPS path), but before doing that I want to write 
>>> a utility to detect when we read data that was flushed via autoflush to cut 
>>> down on any potential bugs.
>>> 
>>> What I'd like to do is this (while autoflush is still enabled):
>>> 
>>> table_one = TableOne(name='test')
>>> session.add(table_one) # Track this object as pending a flush via the 
>>> before_* listeners, this is working as expected.
>>> 
>>> table_two_instances = session.query(TableTwo).all() # All good, doesn't do 
>>> anything with the table_one instance created earlier
>>> 
>>> table_one_instances = session.query(TableOne).all() # I would like to log a 
>>> warning here as the results of this query depend on a flush happening. What 
>>> I'm hoping to do is detect that one of the rows coming back is the same 
>>> object that was flagged earlier, but I can't see to find the correct event 
>>> to use.
>> 
>> you probably want to also use refresh_flush also, which will be invoked for 
>> column defaults that are populated on the object
>> 
>> https://docs.sqlalchemy.org/en/20/orm/events.html#sqlalchemy.orm.InstanceEvents.refresh_flush
>> 
>> 
>> 
>> 
>>> 
>>> Is this possible? I can't seem to find the appropriate event that would 
>>> trigger when the results from the .all() query get merged back in with the 
>>> existing objects in the session.
>> 
>> that's the refresh event
>> 
>> 
>> 
>> 
>> -- 
>> SQLAlchemy - 
>> The Python SQL Toolkit and Object Relational Mapper
>>  
>> http://www.sqlalchemy.org/
>>  
>> To post example code, please provide an MCVE: Minimal, Complete, and 
>> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
>> description.
>> --- 
>> You received this message because you are subscribed to a topic in the 
>> Google Groups "sqlalchemy" group.
>> To unsubscribe from this topic, visit 
>> https://groups.google.com/d/topic/sqlalchemy/9bevy3Kpql4/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to 
>> sqlalchemy+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/sqlalchemy/c5b89f67-e3fa-45f2-8774-463d154514f3%40app.fastmail.com
>>  
>> .
> 
> 
> -- 
> SQLAlchemy - 
> The Python SQL Toolkit and Object Relational Mapper
>  
> http://www.sqlalchemy.org/
>  
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> --- 
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/CAEx_o%2BAhhUSPDYuGL4TFkTvxjHYfKEWUnBMxh2PzCDw3GJPaGg%40mail.gmail.com
>  
> .

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/1047d2d1-6964-4d03-a978-dbac1b3e57ec%40app.fastmail.com.

Re: [sqlalchemy] Event listener for when query results get matched with pre-existing objects on the session

['Tony Cosentini' via sqlalchemy]

Weird, I did try that but I can't seem to trigger it.

Here's a self-contained test:
https://gist.github.com/tonycosentini/4dee3478695d032ca67707b5e26739b6

I have the event listener set to just crash, but it never triggers.

On Fri, Oct 20, 2023 at 9:20 PM Mike Bayer <
mike_not_on_goo...@zzzcomputing.com> wrote:

>
>
> On Fri, Oct 20, 2023, at 8:08 AM, 'Tony Cosentini' via sqlalchemy wrote:
>
> Hi,
>
> Is there any way to listen for an event for when a query result gets
> merged into a pre-existing object in the session?
>
>
> this is the refresh event:
>
>
> https://docs.sqlalchemy.org/en/20/orm/events.html#sqlalchemy.orm.InstanceEvents.refresh
>
>
>
> I'm working on disabling autoflush for some of our codebase (mostly to cut
> down on queries on a high QPS path), but before doing that I want to write
> a utility to detect when we read data that was flushed via autoflush to cut
> down on any potential bugs.
>
> What I'd like to do is this (while autoflush is still enabled):
>
> table_one = TableOne(name='test')
> session.add(table_one) # Track this object as pending a flush via the
> before_* listeners, this is working as expected.
>
> table_two_instances = session.query(TableTwo).all() # All good, doesn't do
> anything with the table_one instance created earlier
>
> table_one_instances = session.query(TableOne).all() # I would like to log
> a warning here as the results of this query depend on a flush happening.
> What I'm hoping to do is detect that one of the rows coming back is the
> same object that was flagged earlier, but I can't see to find the correct
> event to use.
>
>
> you probably want to also use refresh_flush also, which will be invoked
> for column defaults that are populated on the object
>
>
> https://docs.sqlalchemy.org/en/20/orm/events.html#sqlalchemy.orm.InstanceEvents.refresh_flush
>
>
>
>
>
> Is this possible? I can't seem to find the appropriate event that would
> trigger when the results from the .all() query get merged back in with the
> existing objects in the session.
>
>
> that's the refresh event
>
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "sqlalchemy" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/sqlalchemy/9bevy3Kpql4/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sqlalchemy/c5b89f67-e3fa-45f2-8774-463d154514f3%40app.fastmail.com
> 
> .
>

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAEx_o%2BAhhUSPDYuGL4TFkTvxjHYfKEWUnBMxh2PzCDw3GJPaGg%40mail.gmail.com.

Re: [sqlalchemy] Event listener for when query results get matched with pre-existing objects on the session

[Mike Bayer]

On Fri, Oct 20, 2023, at 8:08 AM, 'Tony Cosentini' via sqlalchemy wrote:
> Hi,
> 
> Is there any way to listen for an event for when a query result gets merged 
> into a pre-existing object in the session?

this is the refresh event:

https://docs.sqlalchemy.org/en/20/orm/events.html#sqlalchemy.orm.InstanceEvents.refresh


> 
> I'm working on disabling autoflush for some of our codebase (mostly to cut 
> down on queries on a high QPS path), but before doing that I want to write a 
> utility to detect when we read data that was flushed via autoflush to cut 
> down on any potential bugs.
> 
> What I'd like to do is this (while autoflush is still enabled):
> 
> table_one = TableOne(name='test')
> session.add(table_one) # Track this object as pending a flush via the 
> before_* listeners, this is working as expected.
> 
> table_two_instances = session.query(TableTwo).all() # All good, doesn't do 
> anything with the table_one instance created earlier
> 
> table_one_instances = session.query(TableOne).all() # I would like to log a 
> warning here as the results of this query depend on a flush happening. What 
> I'm hoping to do is detect that one of the rows coming back is the same 
> object that was flagged earlier, but I can't see to find the correct event to 
> use.

you probably want to also use refresh_flush also, which will be invoked for 
column defaults that are populated on the object

https://docs.sqlalchemy.org/en/20/orm/events.html#sqlalchemy.orm.InstanceEvents.refresh_flush




> 
> Is this possible? I can't seem to find the appropriate event that would 
> trigger when the results from the .all() query get merged back in with the 
> existing objects in the session.

that's the refresh event

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/c5b89f67-e3fa-45f2-8774-463d154514f3%40app.fastmail.com.

[sqlalchemy] Event listener for when query results get matched with pre-existing objects on the session

['Tony Cosentini' via sqlalchemy]

Hi,

Is there any way to listen for an event for when a query result gets merged 
into a pre-existing object in the session?

I'm working on disabling autoflush for some of our codebase (mostly to cut 
down on queries on a high QPS path), but before doing that I want to write 
a utility to detect when we read data that was flushed via autoflush to cut 
down on any potential bugs.

What I'd like to do is this (while autoflush is still enabled):

table_one = TableOne(name='test')
session.add(table_one) # Track this object as pending a flush via the 
before_* listeners, this is working as expected.

table_two_instances = session.query(TableTwo).all() # All good, doesn't do 
anything with the table_one instance created earlier

table_one_instances = session.query(TableOne).all() # I would like to log a 
warning here as the results of this query depend on a flush happening. What 
I'm hoping to do is detect that one of the rows coming back is the same 
object that was flagged earlier, but I can't see to find the correct event 
to use.

Is this possible? I can't seem to find the appropriate event that would 
trigger when the results from the .all() query get merged back in with the 
existing objects in the session.

Thanks,
Tony

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/13ed894a-0a70-407b-a568-9cc17fd58042n%40googlegroups.com.