Re: [sqlalchemy] Event listener for when query results get matched with pre-existing objects on the session

[Mike Bayer]

great, glad to know that new API is finding itself useful.

On Sun, Oct 22, 2023, at 9:32 PM, 'Tony Cosentini' via sqlalchemy wrote:
> do_orm_execute() (and freezing the results) totally allows for what I'm 
> trying to do! Also I forgot to mention earlier, but this is only going to run 
> in tests.
> 
> Thanks again for all the help,
> Tony
> 
> On Friday, October 20, 2023 at 11:10:23 PM UTC+8 Mike Bayer wrote:
>> 
>> 
>> On Fri, Oct 20, 2023, at 10:46 AM, 'Tony Cosentini' via sqlalchemy wrote:
>>> Oh I see, thanks for clarifying.
>>> 
>>> I'm trying to detect cases where we depend on the autoflush behavior. For 
>>> example, in the sample above, when the query runs with no_autoflush, we 
>>> won't get back any results (when previously we would get back one row). 
>>> It's a fairly large codebase, so I was trying to automate finding these 
>>> cases in order to add explicit flush calls when needed or just pass in the 
>>> pending object instead of running a query.
>> 
>> 
>> there's not really an event for an object that just gets returned by a query 
>> from the identity map.you would need to do something more drastic like a 
>> do_orm_execute() hook that runs queries internally and then looks at all the 
>> returned objects.  i wouldnt put that in production.
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>>> 
>>> 
>>> Tony
>>> 
>>> On Fri, Oct 20, 2023 at 10:42 PM Mike Bayer  
>>> wrote:
 __
 
 
 On Fri, Oct 20, 2023, at 9:50 AM, 'Tony Cosentini' via sqlalchemy wrote:
> Weird, I did try that but I can't seem to trigger it.
> 
> Here's a self-contained test: 
> https://gist.github.com/tonycosentini/4dee3478695d032ca67707b5e26739b6
 
 
 the object was not affected in that query (that is, not mutated).  if you 
 change the query to this:
 
  table_one_instances = 
 session.query(TableOne).populate_existing().all()
 
 then the refresh event is triggered.
 
 Im not really following what kind of bug you are trying to detect.
 
 
> 
> I have the event listener set to just crash, but it never triggers.
> 
> On Fri, Oct 20, 2023 at 9:20 PM Mike Bayer 
>  wrote:
>> __
>> 
>> 
>> On Fri, Oct 20, 2023, at 8:08 AM, 'Tony Cosentini' via sqlalchemy wrote:
>>> Hi,
>>> 
>>> Is there any way to listen for an event for when a query result gets 
>>> merged into a pre-existing object in the session?
>> 
>> this is the refresh event:
>> 
>> https://docs.sqlalchemy.org/en/20/orm/events.html#sqlalchemy.orm.InstanceEvents.refresh
>> 
>> 
>>> 
>>> I'm working on disabling autoflush for some of our codebase (mostly to 
>>> cut down on queries on a high QPS path), but before doing that I want 
>>> to write a utility to detect when we read data that was flushed via 
>>> autoflush to cut down on any potential bugs.
>>> 
>>> What I'd like to do is this (while autoflush is still enabled):
>>> 
>>> table_one = TableOne(name='test')
>>> session.add(table_one) # Track this object as pending a flush via the 
>>> before_* listeners, this is working as expected.
>>> 
>>> table_two_instances = session.query(TableTwo).all() # All good, doesn't 
>>> do anything with the table_one instance created earlier
>>> 
>>> table_one_instances = session.query(TableOne).all() # I would like to 
>>> log a warning here as the results of this query depend on a flush 
>>> happening. What I'm hoping to do is detect that one of the rows coming 
>>> back is the same object that was flagged earlier, but I can't see to 
>>> find the correct event to use.
>> 
>> you probably want to also use refresh_flush also, which will be invoked 
>> for column defaults that are populated on the object
>> 
>> https://docs.sqlalchemy.org/en/20/orm/events.html#sqlalchemy.orm.InstanceEvents.refresh_flush
>> 
>> 
>> 
>> 
>>> 
>>> Is this possible? I can't seem to find the appropriate event that would 
>>> trigger when the results from the .all() query get merged back in with 
>>> the existing objects in the session.
>> 
>> that's the refresh event
>> 
>> 
>> 
>> 
>> --
>> SQLAlchemy -
>> The Python SQL Toolkit and Object Relational Mapper
>>  
>> http://www.sqlalchemy.org/
>>  
>> To post example code, please provide an MCVE: Minimal, Complete, and 
>> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
>> description.
>> ---
>> You received this message because you are subscribed to a topic in the 
>> Google Groups "sqlalchemy" group.
>> To unsubscribe from this topic, visit 
>> https://groups.google.com/d/topic/sqlalchemy/9bevy3Kpql4/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to 
>> sqlalchemy+...@googlegroups.com.
>> To view this discussion on the web 

Re: [sqlalchemy] Event listener for when query results get matched with pre-existing objects on the session

['Tony Cosentini' via sqlalchemy]

do_orm_execute() (and freezing the results) totally allows for what I'm 
trying to do! Also I forgot to mention earlier, but this is only going to 
run in tests.

Thanks again for all the help,
Tony

On Friday, October 20, 2023 at 11:10:23 PM UTC+8 Mike Bayer wrote:

>
>
> On Fri, Oct 20, 2023, at 10:46 AM, 'Tony Cosentini' via sqlalchemy wrote:
>
> Oh I see, thanks for clarifying.
>
> I'm trying to detect cases where we depend on the autoflush behavior. For 
> example, in the sample above, when the query runs with no_autoflush, we 
> won't get back any results (when previously we would get back one row). 
> It's a fairly large codebase, so I was trying to automate finding these 
> cases in order to add explicit flush calls when needed or just pass in the 
> pending object instead of running a query.
>
>
>
> there's not really an event for an object that just gets returned by a 
> query from the identity map.you would need to do something more drastic 
> like a do_orm_execute() hook that runs queries internally and then looks at 
> all the returned objects.  i wouldnt put that in production.
>
>
>
>
>
>
>
>
> Tony
>
> On Fri, Oct 20, 2023 at 10:42 PM Mike Bayer  
> wrote:
>
>
>
>
> On Fri, Oct 20, 2023, at 9:50 AM, 'Tony Cosentini' via sqlalchemy wrote:
>
> Weird, I did try that but I can't seem to trigger it.
>
> Here's a self-contained test: 
> https://gist.github.com/tonycosentini/4dee3478695d032ca67707b5e26739b6
>
>
>
> the object was not affected in that query (that is, not mutated).  if you 
> change the query to this:
>
>  table_one_instances = 
> session.query(TableOne).populate_existing().all()
>
> then the refresh event is triggered.
>
> Im not really following what kind of bug you are trying to detect.
>
>
>
> I have the event listener set to just crash, but it never triggers.
>
> On Fri, Oct 20, 2023 at 9:20 PM Mike Bayer  
> wrote:
>
>
>
>
> On Fri, Oct 20, 2023, at 8:08 AM, 'Tony Cosentini' via sqlalchemy wrote:
>
> Hi,
>
> Is there any way to listen for an event for when a query result gets 
> merged into a pre-existing object in the session?
>
>
> this is the refresh event:
>
>
> https://docs.sqlalchemy.org/en/20/orm/events.html#sqlalchemy.orm.InstanceEvents.refresh
>
>
>
> I'm working on disabling autoflush for some of our codebase (mostly to cut 
> down on queries on a high QPS path), but before doing that I want to write 
> a utility to detect when we read data that was flushed via autoflush to cut 
> down on any potential bugs.
>
> What I'd like to do is this (while autoflush is still enabled):
>
> table_one = TableOne(name='test')
> session.add(table_one) # Track this object as pending a flush via the 
> before_* listeners, this is working as expected.
>
> table_two_instances = session.query(TableTwo).all() # All good, doesn't do 
> anything with the table_one instance created earlier
>
> table_one_instances = session.query(TableOne).all() # I would like to log 
> a warning here as the results of this query depend on a flush happening. 
> What I'm hoping to do is detect that one of the rows coming back is the 
> same object that was flagged earlier, but I can't see to find the correct 
> event to use.
>
>
> you probably want to also use refresh_flush also, which will be invoked 
> for column defaults that are populated on the object
>
>
> https://docs.sqlalchemy.org/en/20/orm/events.html#sqlalchemy.orm.InstanceEvents.refresh_flush
>
>
>
>
>
> Is this possible? I can't seem to find the appropriate event that would 
> trigger when the results from the .all() query get merged back in with the 
> existing objects in the session.
>
>
> that's the refresh event
>
>
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>  
> http://www.sqlalchemy.org/
>  
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to a topic in the 
> Google Groups "sqlalchemy" group.
> To unsubscribe from this topic, visit 
> https://groups.google.com/d/topic/sqlalchemy/9bevy3Kpql4/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to 
> sqlalchemy+...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/c5b89f67-e3fa-45f2-8774-463d154514f3%40app.fastmail.com
>  
> 
> .
>
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>  
> http://www.sqlalchemy.org/
>  
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from