Re: [sqlalchemy] Caching intermediate results in baked queries

2018-11-16 Thread Martijn van Oosterhout
t;> s1 = select([b1]) > >>> print(s1) > SELECT :param_1 AS anon_1 > >>> s1.compile().construct_params({'%(140062366895800 param)s': 5}) > {'param_1': 5} > > > Ah, understood. That's a pretty neat trick actually, might see if we can incorporate that somewhere.

Re: [sqlalchemy] Caching intermediate results in baked queries

2018-11-15 Thread Martijn van Oosterhout
ameters. The API for the user does resemble what you do with normally subclassing Query, which is nice because it means the users are getting baked queries, without actually knowing they're getting them. The saved parameters are automatically bound to the Result object on execution. Hope this is cl

Re: [sqlalchemy] Caching intermediate results in baked queries

2018-11-14 Thread Martijn van Oosterhout
Hoi Mike, On Wednesday, November 14, 2018 at 11:30:18 PM UTC+1, Mike Bayer wrote: > > I have some big ideas for the baked concept which may or may not > impact this area as well. But they are only ideas for now, I have > many explicit SQLAlchemy tasks that I have to get done before I can >

Re: [sqlalchemy] Caching intermediate results in baked queries

2018-11-14 Thread Martijn van Oosterhout
tentially how much savings you'd really get, because I'm not sure > it's that much. If you have a limited number of Ns, then it's > probably not worth it. > > > > -- Martijn van Oosterhout http://svana.org/kleptog/ -- SQLAlchemy - The Python SQL Toolkit and Object Relational Ma

[sqlalchemy] Caching intermediate results in baked queries

2018-11-13 Thread Martijn van Oosterhout
Hoi, We're starting to do some pretty complicated things with the SQLAlchemy bakery (because we have some pretty complicated queries) and I have something I'd like to verify. We have queries that are constructed like so (each letter is a step): A B C .. M N0 A B C .. M N1 A B C .. M N2 A B C

Re: [sqlalchemy] Baked queries vs before_execute hook

2017-10-12 Thread Martijn van Oosterhout
On Thursday, October 12, 2017 at 5:47:53 PM UTC+2, Mike Bayer wrote: > > > It sounds like you are getting back inconsistent SQL for the same > query based on some external context that is not being considered as > part of the cache key. This would indicate that you are probably > modifying the

[sqlalchemy] Baked queries vs before_execute hook

2017-10-12 Thread Martijn van Oosterhout
Hi, Recently we've been looking into the baked query feature as a method of speeding up query compilation. We also use a before_execute hook to modify the query before execution to handle permission related stuff. One thing that turned up was that when using a baked query that it cached the

Re: [sqlalchemy] Combining yield_per and eager loading

2016-07-20 Thread Martijn van Oosterhout
Ok, so this is what I have for today. It works, and handles all kinds of corner cases and yet it's not quite what I want. It does everything as a joinedload. It's much easier to use now though. You can do things like: q = Session.query(Animal) for animal in yielded_load(q,

Re: [sqlalchemy] Combining yield_per and eager loading

2016-07-20 Thread Martijn van Oosterhout
On 19 July 2016 at 23:22, Mike Bayer <mike...@zzzcomputing.com> wrote: > > > On 07/19/2016 05:20 PM, Martijn van Oosterhout wrote: > >> >> >> Thanks. On the way home though I had a thought: wouldn't it be simpler >> to run the original query with yiel

Re: [sqlalchemy] Combining yield_per and eager loading

2016-07-19 Thread Martijn van Oosterhout
On 19 July 2016 at 18:42, Mike Bayer <mike...@zzzcomputing.com> wrote: > > > On 07/19/2016 12:33 PM, Martijn van Oosterhout wrote: > >> On Sunday, July 17, 2016 at 8:47:11 AM UTC+2, Martijn van Oosterhout >> wrote: >> >> But in any ca

Re: [sqlalchemy] Combining yield_per and eager loading

2016-07-19 Thread Martijn van Oosterhout
On Sunday, July 17, 2016 at 8:47:11 AM UTC+2, Martijn van Oosterhout wrote: > > > I'll play a bit and see what I can get to work. Thanks again. > > So, I have a chance to play and got something that actually works quite nicely, see below. Two things: - I switched to referencing

Re: [sqlalchemy] Combining yield_per and eager loading

2016-07-17 Thread Martijn van Oosterhout
On 15 Jul 2016 23:57, "Mike Bayer" wrote: > > > > OK, but the IN loading you're doing, that's good too, I'd like to add a loader which uses that someday, and you're right, if we did, we could make it work with yield_per too due to the nature of it. If you want to use

Re: [sqlalchemy] Combining yield_per and eager loading

2016-07-15 Thread Martijn van Oosterhout
oks like it will work transparently with eager loading. It basically does the above, but skips the cursor and replaces it with queries on ranges of the primary key (which is shorter and probably more efficient to boot). Thanks for the tip! -- Martijn van Oosterhout <klep...@gmail.com> http

[sqlalchemy] Combining yield_per and eager loading

2016-07-15 Thread Martijn van Oosterhout
ewhere that might help? Thanks for any ideas, Have a nice day, -- Martijn van Oosterhout -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sql

[sqlalchemy] (Micro-)feature request: sessionmaker.configure(class_=foo)

2016-07-10 Thread Martijn van Oosterhout
and global, that leads to issues... It's not a really important feature, but it would allow us to clean up some warts here are there. Have a nice day, -- Martijn van Oosterhout -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubs

Re: [sqlalchemy] 1.1.0b1: construct has no default compilation handler.

2016-06-28 Thread Martijn van Oosterhout
On 28 June 2016 at 15:47, Mike Bayer <mike...@zzzcomputing.com> wrote: > > > On 06/28/2016 09:09 AM, Martijn van Oosterhout wrote: > >> >> That sqlite reference looks a bit weird, did I miss some initialisation >> somewhere? >> > > It looks like th

Re: [sqlalchemy] 1.1.0b1: construct has no default compilation handler.

2016-06-28 Thread Martijn van Oosterhout
primary_key=True, nullable=False, server_default=DefaultClause('{}', for_update=False))}, 'pdb': , 'fn': None, 'compiler': } (Pdb) p self.specs {'sqlite': } (Pdb) p self.specs['default'] *** KeyError: KeyError('default',) That sqlite reference looks a bit weird, did I miss some initialisation

Re: [sqlalchemy] Re: Tracking query performance per Session

2016-06-27 Thread Martijn van Oosterhout
sion have an .info attribute > which is a dictionary you can use for memoizations. > > -- Martijn van Oosterhout <klep...@gmail.com> http://svana.org/kleptog/ -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscrib

[sqlalchemy] 1.1.0b1: construct has no default compilation handler.

2016-06-27 Thread Martijn van Oosterhout
this helps, -- Martijn van Oosterhout -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group,

Re: [sqlalchemy] "RuntimeError: dictionary changed size during iteration" during configure_mappers

2016-06-27 Thread Martijn van Oosterhout
up that way in the beginning it can be painful to rework... -- Martijn van Oosterhout -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...

[sqlalchemy] Re: Tracking query performance per Session

2016-06-27 Thread Martijn van Oosterhout
On Tuesday, June 21, 2016 at 11:25:36 PM UTC+2, Martijn van Oosterhout wrote: > > Hi, > > In an effort to find the bottlenecks in our system we'd like to collect > statistics about which queries take a long time, or return a lot of rows, > or paths which are triggeri

[sqlalchemy] Tracking query performance per Session

2016-06-21 Thread Martijn van Oosterhout
to extract this information reliably? There is the after_begin() event on a Session which has both the Session and the Connection, but there doesn't appear to be a reliable moment when the Session is no longer related to a connection. Or am I missing something? Thanks in advance, -- Martijn van

[sqlalchemy] EXTRACT() not working as expected in PostgreSQL with TIMESTAMP WITH TIMEZONE

2013-06-04 Thread Martijn van Oosterhout
one. But is there a workaround to suppress the cast? Thanks in advance, -- Martijn van Oosterhout klep...@gmail.com -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email

[sqlalchemy] Re: EXTRACT() not working as expected in PostgreSQL with TIMESTAMP WITH TIMEZONE

2013-06-04 Thread Martijn van Oosterhout
about all the builtin casts, but for now it's fine. Have a nice day, On Tuesday, June 4, 2013 10:53:22 AM UTC+2, Martijn van Oosterhout wrote: snip I think it's a bug and if people agree I can file one. But is there a workaround to suppress the cast? Thanks in advance, -- Martijn van