Re: [sqlalchemy] Large increase in memory use when upgrading from 1.3 to 1.4

2023-01-04 Thread 'Tony Cosentini' via sqlalchemy
Somehow I never followed up on this one, sorry :( We upgraded to 1.4.44 and indeed did see a big decrease in memory usage. However, with how we are using gunicorn there's still a noticable jump in memory usage with the cache enabled, however it's not nearly as large as before. We're still

Re: [sqlalchemy] Large increase in memory use when upgrading from 1.3 to 1.4

2022-11-13 Thread 'Tony Cosentini' via sqlalchemy
Oh wow, this sounds like it would definitely impact us. I'm out this week travelling, but I will definitely upgrade + re-enable the cache and report back. Thanks! On Sat, Nov 12, 2022 at 3:17 PM Mike Bayer wrote: > 1.4.44 is released with this change. if you can try it out with your >

Re: [sqlalchemy] Large increase in memory use when upgrading from 1.3 to 1.4

2022-11-12 Thread Mike Bayer
1.4.44 is released with this change. if you can try it out with your application and let me know if you see improvements in memory use for your memory-intensive case, that would be much appreciated! thanks https://www.sqlalchemy.org/blog/2022/11/12/sqlalchemy-1.4.44-released/ On Fri, Nov

Re: [sqlalchemy] Large increase in memory use when upgrading from 1.3 to 1.4

2022-11-11 Thread Mike Bayer
We've identified a significant source of memory over-use in the 1.4 caching system, on the particular environment where it was discovered, an extremely long and complex query nonetheless created a cache key that used orders of magnitude more memory than the statement itself. A fix for this

Re: [sqlalchemy] Large increase in memory use when upgrading from 1.3 to 1.4

2022-11-03 Thread Mike Bayer
On Thu, Nov 3, 2022, at 3:11 AM, 'Tony Cosentini' via sqlalchemy wrote: > Hey, sorry for the crazy delay on this! > > We ended up turning off the cache in everything via query_cache_size and > memory usage returned to previous levels. We also didn't see any noticeable > change in CPU usage in

Re: [sqlalchemy] Large increase in memory use when upgrading from 1.3 to 1.4

2022-11-03 Thread 'Tony Cosentini' via sqlalchemy
Hey, sorry for the crazy delay on this! We ended up turning off the cache in everything via query_cache_size and memory usage returned to previous levels. We also didn't see any noticeable change in CPU usage in our web servers. We did see a pretty noticable perf regression in a worker job

Re: [sqlalchemy] Large increase in memory use when upgrading from 1.3 to 1.4

2022-10-21 Thread 'Tony Cosentini' via sqlalchemy
Hi Mike, Thanks for such a fast reply! We tried setting query_cache_size on a canary environment today, will be rolling it out widely on servers on Monday and can report back on if it has a noticeable impact. After thinking about this more, I think our situation might exacerbate things a bit,

Re: [sqlalchemy] Large increase in memory use when upgrading from 1.3 to 1.4

2022-10-20 Thread Mike Bayer
On Fri, Oct 21, 2022, at 12:00 AM, 'Tony Cosentini' via sqlalchemy wrote: > Hi, > > We recently upgraded our application (a Flask web app) from SQLAlchemy 1.3.19 > to 1.4.41. > > Overall things are stable, but we have noticed a very large increase in > memory use: > Screen Shot 2022-10-21 at

[sqlalchemy] Large increase in memory use when upgrading from 1.3 to 1.4

2022-10-20 Thread 'Tony Cosentini' via sqlalchemy
Hi, We recently upgraded our application (a Flask web app) from SQLAlchemy 1.3.19 to 1.4.41. Overall things are stable, but we have noticed a very large increase in memory use: [image: Screen Shot 2022-10-21 at 11.26.18 AM.png] Is this from the new query caching feature? I'm planning on