On Tue, Nov 13, 2018 at 3:49 PM Martijn van Oosterhout
<klep...@gmail.com> wrote:
>
> Hoi,
>
> We're starting to do some pretty complicated things with the SQLAlchemy
> bakery (because we have some pretty complicated queries) and I have something
> I'd like to verify.
>
> We have queries that are constructed like so (each letter is a step):
>
> A B C .. M N0
> A B C .. M N1
> A B C .. M N2
> A B C .. M N3
>
> So we have lots of steps and only at the end does something change. Not all
> the time, but much more often than the steps A to M. The way the bakery is
> set up only the last step is actually cached, meaning that steps A to M get
> done each time the last step changes.
>
> However, suppose we could, after step M, do a kind of checkpoint so that at
> least that part is done only once. AFAICT it would work like the "spoil"
> method, except leave spoil as False. So something like:
>
> class BakedQuery(object):
> def checkpoint(self):
> _check_point = self._clone()
> _check_point._cache_key += ('_query_only', )
> self.steps = [_check_point._retrieve_baked_query]
> return self
>
>
> It's not a great name I agree, but the alternatives I can think of are mark,
> save, stuff like that.
>
> Anyway, AFAICT this should just work. The assumption is that the query
> construction takes significant time, but I think we are sure of that. When an
> N4 comes, after executing the checkpoint after M, the last step simply
> extracts the cached intermediate step, applies N4 and we're done.
The two things that the BakedQuery caches are the QueryContext, which
is the thing that Query creates when everything it needs is ready to
go and is by far the most expensive part of the Query, and then
indirectly it caches the string SQL which also cannot be generated
until everything is ready to go. That is, there's not much to cache
before you're ready to generate the SQL. If you're looking to just
cache the overhead of calling the join() or filter() method, that is
something, but this would need some new mechanism of having that be
cached as well within the bakery, e.g. a Query by itself that's the
result of some series of steps. It might be something you'd want
to play with if you really want to hack on the internals of the
BakedQuery but I'd recommend doing some code profiling up front to see
potentially how much savings you'd really get, because I'm not sure
it's that much. If you have a limited number of Ns, then it's
probably not worth it.
>
> Am I missing something? Is this something that could be considered an
> submittable improvement?
>
> Thanks in advance,
> Martijn
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To post to this group, send email to sqlalchemy@googlegroups.com.
> Visit this group at https://groups.google.com/group/sqlalchemy.
> For more options, visit https://groups.google.com/d/optout.
--
SQLAlchemy -
The Python SQL Toolkit and Object Relational Mapper
http://www.sqlalchemy.org/
To post example code, please provide an MCVE: Minimal, Complete, and Verifiable
Example. See http://stackoverflow.com/help/mcve for a full description.
---
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.
