Re: [sqlalchemy] Query and compiled_cache

2013-06-07 Thread Claudio Freire
On Thu, Jun 6, 2013 at 2:20 PM, Michael Bayer mike...@zzzcomputing.com wrote: diff --git a/lib/sqlalchemy/orm/query.py b/lib/sqlalchemy/orm/query.py index c2ec72c..b458975 100644 --- a/lib/sqlalchemy/orm/query.py +++ b/lib/sqlalchemy/orm/query.py @@ -40,12 +40,15 @@ from ..sql import (

Re: [sqlalchemy] Query and compiled_cache

2013-06-07 Thread Claudio Freire
On Fri, Jun 7, 2013 at 1:56 PM, Claudio Freire klaussfre...@gmail.com wrote: This is my current BakedQuery: class BakedQuery(sqlalchemy.orm.query.Query): F**k gmail again. Why the F+!k doesn't it show me that it'll bork all whitespace before I hit send... I smell a bug report coming...

Re: [sqlalchemy] Query and compiled_cache

2013-06-06 Thread Claudio Freire
On Tue, Jun 4, 2013 at 5:26 PM, Claudio Freire klaussfre...@gmail.com wrote: On Sun, Jun 2, 2013 at 9:41 PM, Claudio Freire klaussfre...@gmail.com wrote: So the whole thing is rolled up into the named thing I referred to also, so that there's no need to keep a Query object hanging around,

Re: [sqlalchemy] Query and compiled_cache

2013-06-06 Thread Michael Bayer
On Jun 6, 2013, at 1:03 PM, Claudio Freire klaussfre...@gmail.com wrote: On Tue, Jun 4, 2013 at 5:26 PM, Claudio Freire klaussfre...@gmail.com wrote: On Sun, Jun 2, 2013 at 9:41 PM, Claudio Freire klaussfre...@gmail.com wrote: So the whole thing is rolled up into the named thing I referred

Re: [sqlalchemy] Query and compiled_cache

2013-06-04 Thread Claudio Freire
On Sun, Jun 2, 2013 at 9:41 PM, Claudio Freire klaussfre...@gmail.comwrote: So the whole thing is rolled up into the named thing I referred to also, so that there's no need to keep a Query object hanging around, when we say bake() we're really just referring to a position in the code

Re: [sqlalchemy] Query and compiled_cache

2013-06-02 Thread Claudio Freire
On Sat, Jun 1, 2013 at 10:59 PM, Michael Bayer mike...@zzzcomputing.comwrote: The recipe on the wiki also has the issue that it isn't even caching anything to do with the QueryContext, including all of this information regarding eager joins which is pretty important. Your modifications try to

Re: [sqlalchemy] Query and compiled_cache

2013-06-02 Thread Michael Bayer
Well, if that works, it certainly covers my needs so there would be no pressing need to incorporate it into the core. I'll let you know tomorrow. It's so straightforward at this point I'm leaning towards some more cleanup and adding to 0.9. I'll probably cache the compiled statements

Re: [sqlalchemy] Query and compiled_cache

2013-06-02 Thread Claudio Freire
On Mon, Jun 3, 2013 at 1:55 AM, Michael Bayer mike...@zzzcomputing.comwrote: Well, if that works, it certainly covers my needs so there would be no pressing need to incorporate it into the core. I'll let you know tomorrow. It's so straightforward at this point I'm leaning towards some more

Re: [sqlalchemy] Query and compiled_cache

2013-06-01 Thread Michael Bayer
the detached error is because even though the baked query emits the correct SQL with the LEFT OUTER JOIN, there are callables that are present inside the QueryContext that are tailored to look specifically for a particular alias() of the mapped Address table, which is not the same alias()

Re: [sqlalchemy] Query and compiled_cache

2013-05-31 Thread Claudio Freire
On Thu, May 30, 2013 at 7:04 PM, Michael Bayer mike...@zzzcomputing.com wrote: The hashing thing really has to start as a core concept first. It's a big job but would be very helpful for caching scenarios and would allow us to build this feature on Query without too much difficulty. The

Re: [sqlalchemy] Query and compiled_cache

2013-05-31 Thread Michael Bayer
On Friday, May 31, 2013 10:18:41 AM UTC-4, Klauss wrote: On Thu, May 30, 2013 at 7:04 PM, Michael Bayer mik...@zzzcomputing.comjavascript: wrote: The hashing thing really has to start as a core concept first. It's a big job but would be very helpful for caching scenarios and

Re: [sqlalchemy] Query and compiled_cache

2013-05-31 Thread Claudio Freire
On Fri, May 31, 2013 at 11:29 AM, Michael Bayer mike...@zzzcomputing.com wrote: On Friday, May 31, 2013 10:18:41 AM UTC-4, Klauss wrote: On Thu, May 30, 2013 at 7:04 PM, Michael Bayer mik...@zzzcomputing.com wrote: The hashing thing really has to start as a core concept first. It's a

Re: [sqlalchemy] Query and compiled_cache

2013-05-31 Thread Michael Bayer
On May 31, 2013, at 10:51 AM, Claudio Freire klaussfre...@gmail.com wrote: On Fri, May 31, 2013 at 11:29 AM, Michael Bayer mike...@zzzcomputing.com wrote: On Friday, May 31, 2013 10:18:41 AM UTC-4, Klauss wrote: On Thu, May 30, 2013 at 7:04 PM, Michael Bayer mik...@zzzcomputing.com

Re: [sqlalchemy] Query and compiled_cache

2013-05-31 Thread Claudio Freire
On Fri, May 31, 2013 at 12:46 PM, Michael Bayer mike...@zzzcomputing.com wrote: this hash works for backends that can render LIMIT as a bound parameter. It will *not* work for SQL server which cannot render LIMIT as a bound parameter. If the hash is determined at the level of Query, we *do

Re: [sqlalchemy] Query and compiled_cache

2013-05-31 Thread Michael Bayer
On May 31, 2013, at 11:59 AM, Claudio Freire klaussfre...@gmail.com wrote: On Fri, May 31, 2013 at 12:46 PM, Michael Bayer mike...@zzzcomputing.com wrote: this hash works for backends that can render LIMIT as a bound parameter. It will *not* work for SQL server which cannot render LIMIT as

Re: [sqlalchemy] Query and compiled_cache

2013-05-31 Thread Claudio Freire
On Thu, May 30, 2013 at 3:28 PM, Claudio Freire klaussfre...@gmail.com wrote: On Thu, May 30, 2013 at 2:25 PM, Michael Bayer mike...@zzzcomputing.com wrote: On May 30, 2013, at 11:25 AM, Claudio Freire klaussfre...@gmail.com wrote: That way, one could use the second form up there and benefit

Re: [sqlalchemy] Query and compiled_cache

2013-05-31 Thread Claudio Freire
On Fri, May 31, 2013 at 4:03 PM, Claudio Freire klaussfre...@gmail.comwrote: With: class CacheableQuery(sqlalchemy.orm.query.Query): def __init__(self, *p, **kw): self._cached_context = None self._cached_context_labels = None super(CacheableQuery, self).__init__(*p, **kw)

Re: [sqlalchemy] Query and compiled_cache

2013-05-31 Thread Michael Bayer
can you just attach a working .py script On May 31, 2013, at 3:06 PM, Claudio Freire klaussfre...@gmail.com wrote: On Fri, May 31, 2013 at 4:03 PM, Claudio Freire klaussfre...@gmail.com wrote: With: class CacheableQuery(sqlalchemy.orm.query.Query): def __init__(self, *p, **kw):

Re: [sqlalchemy] Query and compiled_cache

2013-05-31 Thread Claudio Freire
On Fri, May 31, 2013 at 4:44 PM, Michael Bayer mike...@zzzcomputing.comwrote: can you just attach a working .py script How does that work without a database? -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and

Re: [sqlalchemy] Query and compiled_cache

2013-05-31 Thread Claudio Freire
On Fri, May 31, 2013 at 4:47 PM, Claudio Freire klaussfre...@gmail.comwrote: On Fri, May 31, 2013 at 4:44 PM, Michael Bayer mike...@zzzcomputing.comwrote: can you just attach a working .py script How does that work without a database? Ok, I took one of SQLA's tests, and make it break

[sqlalchemy] Query and compiled_cache

2013-05-30 Thread Claudio Freire
I know this has been discussed a great deal already, but I've been noticing this: SomeClass.query() .filter_by(id = blah) .execution_options(compiled_cache = _orm_query_cache) .options( orm.joinedload(blah), orm.joinedload(blah, bleh), orm.joinedload(blah,

Re: [sqlalchemy] Query and compiled_cache

2013-05-30 Thread Michael Bayer
On May 30, 2013, at 11:25 AM, Claudio Freire klaussfre...@gmail.com wrote: That way, one could use the second form up there and benefit from query hashing, because session/param binding wouldn't change the hash, and it would be a cache hit. Has it been explored already? Or maybe there's

Re: [sqlalchemy] Query and compiled_cache

2013-05-30 Thread Claudio Freire
On Thu, May 30, 2013 at 2:25 PM, Michael Bayer mike...@zzzcomputing.com wrote: On May 30, 2013, at 11:25 AM, Claudio Freire klaussfre...@gmail.com wrote: That way, one could use the second form up there and benefit from query hashing, because session/param binding wouldn't change the hash,

Re: [sqlalchemy] Query and compiled_cache

2013-05-30 Thread Michael Bayer
On May 30, 2013, at 2:28 PM, Claudio Freire klaussfre...@gmail.com wrote: 4. it would have a super crapload of very complete and clean unit tests. Ehm... I would imagine all the current tests involving Query would cover most of it. A few more cache-specific tests could be added surely,

Re: [sqlalchemy] Query and compiled_cache

2013-05-30 Thread Michael Bayer
my first 25 seconds of looking at this reveals that if you want to be able to generate a hash, this has to go all the way down to everything. query.filter(X == Y) means you need a hash for X == Y too.These hashes are definitely going to be determined using a traversal scheme for sure: q

Re: [sqlalchemy] Query and compiled_cache

2013-05-30 Thread Claudio Freire
On Thu, May 30, 2013 at 3:45 PM, Michael Bayer mike...@zzzcomputing.com wrote: On May 30, 2013, at 2:28 PM, Claudio Freire klaussfre...@gmail.com wrote: 4. it would have a super crapload of very complete and clean unit tests. Ehm... I would imagine all the current tests involving Query

Re: [sqlalchemy] Query and compiled_cache

2013-05-30 Thread Michael Bayer
On May 30, 2013, at 2:28 PM, Claudio Freire klaussfre...@gmail.com wrote: On Thu, May 30, 2013 at 2:25 PM, Michael Bayer mike...@zzzcomputing.com wrote: If you want to work on a feature that is actually going to change SQLAlchemy, (and would that be before or after you finish #2720? :)

Re: [sqlalchemy] Query and compiled_cache

2013-05-30 Thread Michael Bayer
On May 30, 2013, at 3:10 PM, Claudio Freire klaussfre...@gmail.com wrote: On Thu, May 30, 2013 at 3:45 PM, Michael Bayer mike...@zzzcomputing.com wrote: On May 30, 2013, at 2:28 PM, Claudio Freire klaussfre...@gmail.com wrote: 4. it would have a super crapload of very complete and

Re: [sqlalchemy] Query and compiled_cache

2013-05-30 Thread Claudio Freire
On Thu, May 30, 2013 at 4:19 PM, Michael Bayer mike...@zzzcomputing.com wrote: In case I'm not clear, this would not be cached if I were to take id(internals) query(Blah).filter(blah).join(blah).first() But I don't care, because that's expensive on its own. result caching (because we're

Re: [sqlalchemy] Query and compiled_cache

2013-05-30 Thread Michael Bayer
a very brief example of this, which if you keep digging in you can see how tricky it gets (fast), is like: from sqlalchemy.sql import column, table t1 = table('t1', column('x'), column('y')) t2 = table('t1', column('x'), column('y')) t3 = table('t2', column('p'), column('r')) t4 =

Re: [sqlalchemy] Query and compiled_cache

2013-05-30 Thread Michael Bayer
my next thought is, if something isn't distinctly hashable, then it should cancel being hashable entirely. this patch shows it using a symbol unhashable: https://gist.github.com/zzzeek/5681612 . If any construct has an unhashable inside of it, then that construct is unhashable too. The