[sqlalchemy] Re: [PATCH] Filtered one_to_many relationships (Experimental)
On 4/17/07, Michael Bayer [EMAIL PROTECTED] wrote: On Apr 10, 2007, at 10:22 AM, Gaetan de Menten wrote: By the way, should I create a ticket for this? ive created ticket #541 for this, I had already created ticket #531 for this. Sorry for not mentioning it here (I thought you'd see it). Anyway, it can be closed too now :). and implemented a slightly refined version of the patch you provided, including the classmethod as well as a generative method + unit tests for both. I added docs and examples for the generative version, so you can see the use cases i was getting at. it will throw an error if you give it input that makes no sense (i.e. two classes that are unrelated). the pattern looks like (assuming the user-address paradigm) : # without specifying the property session.query(Address).with_parent(someuser).list() # specifying the property session.query(Address).with_parent(someuser, property='addresses').filter_by(email_address='[EMAIL PROTECTED]').list() That's just great! I was going to do it eventually but it seems like you beat me to it ;-). Thanks a lot. -- Gaëtan de Menten http://openhex.org --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: [PATCH] Filtered one_to_many relationships (Experimental)
On Apr 17, 2007, at 2:49 PM, Gaetan de Menten wrote: I had already created ticket #531 for this. Sorry for not mentioning it here (I thought you'd see it). Anyway, it can be closed too now :). oh crap, sorry, i did a quick search thru track for filtered and nothing came up. yeah im starting to lose track of tickets now... --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: [PATCH] Filtered one_to_many relationships (Experimental)
On Apr 13, 2007, at 12:51 PM, [EMAIL PROTECTED] wrote: haven't thought yet of where/how to hack this... i may have to abandon *-to-many-relations() alltogether, as i don't want/need them loaded - only filtered at view-time. or can i make some super- (or sub-) relation thing (propertyLoader?) that does the above? I currently do it using a property that returns a Query object, which works very well. the extra method with_parent() i want to add to Query would make it easier to implement also. class MyClass(object): def _get_children(self): return object_session(self).query (SomeChildClass).with_parent(self) children = property(_get_children) then: m = session.query(MyClass).get(1) somechildren = m.children[3:5] someotherchildren = m.children.filter_by(foo='bar').filter_by (x==y).list() --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to [EMAIL PROTECTED] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: [PATCH] Filtered one_to_many relationships (Experimental)
On 4/10/07, Michael Bayer [EMAIL PROTECTED] wrote:
hm, why is from_attr a classmethod ?
Because that way, you don't have to specify the related class at all,
and you can specify the parameters as args not kwargs. See my first
initial remark:
* I've implemented Query.from_attr, instead of adding new keywords to
the Query constructor, because I think:
Query.from_attr(someuser, 'addresses')
looks better, is shorter and is more readable than:
Query('Address', instance=someuser, attr_name='addresses')
not very consistent with all the other generative methods ?
True, but I think it makes more sense this way (see below).
can we have a regular generative method as well ?
If you really want one, I'll gladly do it, but I don't think it makes
sense because that method can possibly change the mapper.
So, first I'd need to duplicate part of what is in the __init__
method, which doesn't feel right.
And second, I think it could be quite confusing for a user. Imagine
that Query(A).from_attr(inst, 'rel') could return instances of B (or
whatever class is attached to the relation)... Probably not what you'd
expect. So, if we go down that route, it would probably be a good idea
to check that the mapper in the cloned query is the same than the one
we get from the relation.
And by the way, from_attr doesn't sound like a generative method, so
if you want it, what about filter_from_attr, or something similar?
--
Gaëtan de Menten
http://openhex.org
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: [PATCH] Filtered one_to_many relationships (Experimental)
is from_attr makeing sense for plain atributes, e.g. integers ot
whatever?
if no, maybe choose something like from_relation or filter_relation or
filter_relation_tomany or similar if it is expected to only work over
relations - and multiple-instances relations; i.e. it is useless over
single pointer-like 1:1 relation - so the name would suggest the
proper target-type.
On Wednesday 11 April 2007 11:50:53 Gaetan de Menten wrote:
On 4/10/07, Michael Bayer [EMAIL PROTECTED] wrote:
hm, why is from_attr a classmethod ?
Because that way, you don't have to specify the related class at
all, and you can specify the parameters as args not kwargs. See my
first initial remark:
* I've implemented Query.from_attr, instead of adding new keywords
to the Query constructor, because I think:
Query.from_attr(someuser, 'addresses')
looks better, is shorter and is more readable than:
Query('Address', instance=someuser, attr_name='addresses')
not very consistent with all the other generative methods ?
True, but I think it makes more sense this way (see below).
can we have a regular generative method as well ?
If you really want one, I'll gladly do it, but I don't think it
makes sense because that method can possibly change the mapper.
So, first I'd need to duplicate part of what is in the __init__
method, which doesn't feel right.
And second, I think it could be quite confusing for a user. Imagine
that Query(A).from_attr(inst, 'rel') could return instances of B
(or whatever class is attached to the relation)... Probably not
what you'd expect. So, if we go down that route, it would probably
be a good idea to check that the mapper in the cloned query is the
same than the one we get from the relation.
And by the way, from_attr doesn't sound like a generative method,
so if you want it, what about filter_from_attr, or something
similar?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: [PATCH] Filtered one_to_many relationships (Experimental)
On 4/11/07, svilen [EMAIL PROTECTED] wrote:
is from_attr makeing sense for plain atributes, e.g. integers ot
whatever?
You got a point here. It doesn't work (or even make sense) on plain attributes.
if no, maybe choose something like from_relation
Fine with me.
or filter_relation or
filter_relation_tomany or similar if it is expected to only work over
relations - and multiple-instances relations; i.e. it is useless over
single pointer-like 1:1 relation - so the name would suggest the
proper target-type.
It works fine for to_one relations. I'm not sure if anybody will
ever use it on such relations, but since it works as expected, I see
no reason to artificially constrain the thing to to_many
relationships.
On Wednesday 11 April 2007 11:50:53 Gaetan de Menten wrote:
On 4/10/07, Michael Bayer [EMAIL PROTECTED] wrote:
hm, why is from_attr a classmethod ?
Because that way, you don't have to specify the related class at
all, and you can specify the parameters as args not kwargs. See my
first initial remark:
* I've implemented Query.from_attr, instead of adding new keywords
to the Query constructor, because I think:
Query.from_attr(someuser, 'addresses')
looks better, is shorter and is more readable than:
Query('Address', instance=someuser, attr_name='addresses')
not very consistent with all the other generative methods ?
True, but I think it makes more sense this way (see below).
can we have a regular generative method as well ?
If you really want one, I'll gladly do it, but I don't think it
makes sense because that method can possibly change the mapper.
So, first I'd need to duplicate part of what is in the __init__
method, which doesn't feel right.
And second, I think it could be quite confusing for a user. Imagine
that Query(A).from_attr(inst, 'rel') could return instances of B
(or whatever class is attached to the relation)... Probably not
what you'd expect. So, if we go down that route, it would probably
be a good idea to check that the mapper in the cloned query is the
same than the one we get from the relation.
And by the way, from_attr doesn't sound like a generative method,
so if you want it, what about filter_from_attr, or something
similar?
--
Gaëtan de Menten
http://openhex.org
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: [PATCH] Filtered one_to_many relationships (Experimental)
On 3/31/07, Michael Bayer [EMAIL PROTECTED] wrote:
On Mar 31, 2007, at 1:17 PM, Gaetan de Menten wrote:
That's approximately what I did in my patch with the new params
keyword argument, except I only implemented the set operation, not
the add operation on the params. Anyway, what can/should I do to get
this included? Do you have any advice/pointers on how to do the same
for eager attributes? (or will you implement it yourself?)
im totally into a series of engine/execution patches/refactorings
right now, so for properties that have lazy=False, there is still a
LazyLoader strategy there...you should just call
property._get_strategy(LazyLoader) in all cases to get at it.
In case anybody is interested, here is my patch slightly modified with
what you suggest above. Now it works wonders for both lazy and eager
relationships. There is something ugly about it though: imports. I
have to import the LazyLoader class from the orm.strategies module,
but that module imports query, so what I did is import the LazyLoader
class inside the from_attr method to avoid a circular import
problem.
By the way, should I create a ticket for this?
--
Gaëtan de Menten
http://openhex.org
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
Index: orm/query.py
===
--- orm/query.py(revision 2493)
+++ orm/query.py(working copy)
@@ -8,6 +8,7 @@
from sqlalchemy.orm import mapper, class_mapper
from sqlalchemy.orm.interfaces import OperationContext, SynonymProperty
+
__all__ = ['Query', 'QueryContext', 'SelectionContext']
class Query(object):
@@ -42,12 +43,36 @@
self._distinct = kwargs.pop('distinct', False)
self._offset = kwargs.pop('offset', None)
self._limit = kwargs.pop('limit', None)
-self._criterion = None
+self._criterion = kwargs.pop('criterion', None)
+self._params = kwargs.pop('params', {})
self._joinpoint = self.mapper
self._from_obj = [self.table]
for opt in util.flatten_iterator(self.with_options):
opt.process_query(self)
+
+def from_attr(cls, instance, attr_name):
+from sqlalchemy.orm.strategies import LazyLoader
+
+prop = instance.mapper.props[attr_name]
+loader = prop._get_strategy(LazyLoader)
+
+# the following code is taken from strategies.py
+# this gets the values of the columns referenced by the property
+# for this specific instance
+params = {}
+allparams = True
+for col, bind in loader.lazybinds.iteritems():
+params[bind.key] = loader.parent.get_attr_by_column(instance, col)
+if params[bind.key] is None:
+allparams = False
+break
+
+if not allparams:
+return None
+
+return Query(prop.mapper, criterion=loader.lazywhere, params=params)
+from_attr = classmethod(from_attr)
def _clone(self):
q = Query.__new__(Query)
@@ -71,6 +96,7 @@
q._from_obj = list(self._from_obj)
q._joinpoint = self._joinpoint
q._criterion = self._criterion
+q._params = self._params
return q
def _get_session(self):
@@ -694,8 +720,10 @@
method, which takes the executed statement's ResultProxy
directly.
-
-result = self.session.execute(self.mapper, clauseelement,
params=params)
+final_params = self._params.copy()
+if params is not None:
+final_params.update(params)
+result = self.session.execute(self.mapper, clauseelement,
params=final_params)
try:
return self.instances(result, **kwargs)
finally:
[sqlalchemy] Re: [PATCH] Filtered one_to_many relationships (Experimental)
can u give some exampe, how this is supposed to be used (finaly)? as relation and/or directly as Query() On Tuesday 10 April 2007 17:22:45 Gaetan de Menten wrote: On 3/31/07, Michael Bayer [EMAIL PROTECTED] wrote: On Mar 31, 2007, at 1:17 PM, Gaetan de Menten wrote: That's approximately what I did in my patch with the new params keyword argument, except I only implemented the set operation, not the add operation on the params. Anyway, what can/should I do to get this included? Do you have any advice/pointers on how to do the same for eager attributes? (or will you implement it yourself?) im totally into a series of engine/execution patches/refactorings right now, so for properties that have lazy=False, there is still a LazyLoader strategy there...you should just call property._get_strategy(LazyLoader) in all cases to get at it. In case anybody is interested, here is my patch slightly modified with what you suggest above. Now it works wonders for both lazy and eager relationships. There is something ugly about it though: imports. I have to import the LazyLoader class from the orm.strategies module, but that module imports query, so what I did is import the LazyLoader class inside the from_attr method to avoid a circular import problem. By the way, should I create a ticket for this? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: [PATCH] Filtered one_to_many relationships (Experimental)
On 4/10/07, svilen [EMAIL PROTECTED] wrote:
can u give some exampe, how this is supposed to be used (finaly)?
as relation and/or directly as Query()
Attached is a simple example. The setup is done using Elixir but the
actual Query.from_attr usage would be the same with plain SA. If you
really need a plain SA example, just ask.
On Tuesday 10 April 2007 17:22:45 Gaetan de Menten wrote:
On 3/31/07, Michael Bayer [EMAIL PROTECTED] wrote:
On Mar 31, 2007, at 1:17 PM, Gaetan de Menten wrote:
That's approximately what I did in my patch with the new
params keyword argument, except I only implemented the set
operation, not the add operation on the params. Anyway, what
can/should I do to get this included? Do you have any
advice/pointers on how to do the same for eager attributes? (or
will you implement it yourself?)
im totally into a series of engine/execution patches/refactorings
right now, so for properties that have lazy=False, there is
still a LazyLoader strategy there...you should just call
property._get_strategy(LazyLoader) in all cases to get at it.
In case anybody is interested, here is my patch slightly modified
with what you suggest above. Now it works wonders for both lazy and
eager relationships. There is something ugly about it though:
imports. I have to import the LazyLoader class from the
orm.strategies module, but that module imports query, so what I did
is import the LazyLoader class inside the from_attr method to avoid
a circular import problem.
By the way, should I create a ticket for this?
--
Gaëtan de Menten
http://openhex.org
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
from sqlalchemy import *
from elixir import *
class A(Entity):
has_field('name', String(30))
has_many('b', of_kind='B', lazy=False)
class B(Entity):
has_field('name', String(30))
has_field('extra', Integer)
belongs_to('a', of_kind='A')
metadata.connect('sqlite:///')
create_all()
a1 = A(name='a1')
b1 = B(name='b1', a=a1, extra=10)
b2 = B(name='b2', a=a1)
b3 = B(name='b3', a=a1, extra=5)
b4 = B(name='b4', a=a1, extra=4)
b5 = B(name='b5', a=a1, extra=7)
objectstore.flush()
objectstore.clear()
#metadata.engine.echo = True
a = A.get_by(name='a1')
print normal
q = Query.from_attr(a, 'b')
for b in q:
print b.name, b.extra
print filtered extra 8
for b in q.filter(B.c.extra 8):
print b.name, b.extra
print ordered by extra
for b in q.order_by(B.c.extra):
print b.name, b.extra
[sqlalchemy] Re: [PATCH] Filtered one_to_many relationships (Experimental)
On Apr 10, 2007, at 10:22 AM, Gaetan de Menten wrote: In case anybody is interested, here is my patch slightly modified with what you suggest above. Now it works wonders for both lazy and eager relationships. There is something ugly about it though: imports. I have to import the LazyLoader class from the orm.strategies module, but that module imports query, so what I did is import the LazyLoader class inside the from_attr method to avoid a circular import problem. importing inside a method is perfectly fine, ill try to see if theres a workaround but sometimes there really isnt... By the way, should I create a ticket for this? yes. otherwise ill completely forget about it. i might have to rush out some extra features this weekend to get 0.3.7 out, so this one might not be in until 0.3.8. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: [PATCH] Filtered one_to_many relationships (Experimental)
hm, why is from_attr a classmethod ? not very consistent with all the other generative methods ? can we have a regular generative method as well ? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: [PATCH] Filtered one_to_many relationships (Experimental)
On Mar 31, 2007, at 1:17 PM, Gaetan de Menten wrote: That's approximately what I did in my patch with the new params keyword argument, except I only implemented the set operation, not the add operation on the params. Anyway, what can/should I do to get this included? Do you have any advice/pointers on how to do the same for eager attributes? (or will you implement it yourself?) im totally into a series of engine/execution patches/refactorings right now, so for properties that have lazy=False, there is still a LazyLoader strategy there...you should just call property._get_strategy(LazyLoader) in all cases to get at it. the strategy attribute youre looking at there should really be named _default_strategyand should probably be privately held and accessed via a property (with a docstring, yes), and _get_strategy() should become public (and docstring'ed, yes). the interfaces module youre working with in orm is, trust me, or just go look at the 0.2 source...a work of art compared to how this whole area used to function. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: [PATCH] Filtered one_to_many relationships (Experimental)
On 3/26/07, Michael Bayer [EMAIL PROTECTED] wrote:
(and whether that be a .query() or Query() or SelectResults not big
difference imo.)
i vote Query().
I tried to implement it but I couldn't do it the way I wanted to. The
problem is: how do I construct a clause from a clause with bind
parameters + a dictionary containing the values for said bind
parameters? I've only seen bind parameters resolved at execution time.
Is it possible to resolve them earlier? In the attached patch, I used
a workaround which is to store the bind parameters in the query
itself, and then use them whenever the query is executed.
Two remarks:
* I've implemented Query.from_attr, instead of adding new keywords to
the Query constructor, because I think:
Query.from_attr(someuser, 'addresses')
looks better, is shorter and is more readable than:
Query('Address', instance=someuser, attr_name='addresses')
* It only works for lazy attributes. I don't think there is any reason
we couldn't make it work for eager attributes, but by looking at the
eagerloader code, I couldn't figure how to do it.
--
Gaëtan de Menten
http://openhex.org
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
Index: orm/query.py
===
--- orm/query.py(revision 2444)
+++ orm/query.py(working copy)
@@ -42,12 +42,32 @@
self._distinct = kwargs.pop('distinct', False)
self._offset = kwargs.pop('offset', None)
self._limit = kwargs.pop('limit', None)
-self._criterion = None
+self._criterion = kwargs.pop('criterion', None)
+self._params = kwargs.pop('params', {})
self._joinpoint = self.mapper
self._from_obj = [self.table]
for opt in util.flatten_iterator(self.with_options):
opt.process_query(self)
+
+def from_attr(cls, instance, attr_name):
+prop = instance.mapper.props[attr_name]
+loader = prop.strategy
+#TODO: make it work for eager loader too.
+# code taken from strategies.py
+params = {}
+allparams = True
+for col, bind in loader.lazybinds.iteritems():
+params[bind.key] = loader.parent.get_attr_by_column(instance, col)
+if params[bind.key] is None:
+allparams = False
+break
+
+if not allparams:
+return None
+
+return Query(prop.mapper, criterion=loader.lazywhere, params=params)
+from_attr = classmethod(from_attr)
def _clone(self):
q = Query.__new__(Query)
@@ -71,6 +91,7 @@
q._from_obj = list(self._from_obj)
q._joinpoint = self._joinpoint
q._criterion = self._criterion
+q._params = self._params
return q
def _get_session(self):
@@ -690,8 +711,10 @@
method, which takes the executed statement's ResultProxy
directly.
-
-result = self.session.execute(self.mapper, clauseelement,
params=params)
+final_params = self._params.copy()
+if params is not None:
+final_params.update(params)
+result = self.session.execute(self.mapper, clauseelement,
params=final_params)
try:
return self.instances(result, **kwargs)
finally:
[sqlalchemy] Re: [PATCH] Filtered one_to_many relationships (Experimental)
On 3/23/07, Michael Bayer [EMAIL PROTECTED] wrote:
OK, this is actually something people have asked for a lot. in the
beginning, recently, etc. also for different reasons...i.e.
convenience, or performance, etc. So, first off let me start by
illustrating how this use case is done right now. Assuming your
Address mapper has a backref user to the User mapper, its just:
for address in session.query(Address).filter_by(user=someuser).filter
(address_table.c.postcode == 5000):
print address.street
Once again, I discover a better way to do something I did the hard way.
But, the join conditions and bind params which
have been calculated by LazyLoader are just sitting there, they can
be currently pulled out with a little non-API attribute access but
Ive no problem with adding some API-level accessors to get at the
Query object calculated for a particular property (i.e. what you are
using in your patch internally).
Yes, please do, that'd probably solve the problem nicely (see below
how I see things).
now lets look at the way your patch does it.
addresses = user.addresses.filter(address_table.c.postcode == 5000)
seems easy. right ? remember that user is now in the session.
anyone else that queries for user will get that same User
instance. but the rest of the app is going to assume normal
relationship semantics on that collectionwhich means:
How I envisioned things, this wouldn't be a problem, because
user.addresses.filter(xxx) would return a new, independant list, which
doesn't affect user.addresses, and is not affected if user.addresses
has already been accessed or not. This is not what my patch does, I
know. Sorry for not explaining this in my first mail.
print someaddress in user.addresses # -- FAIL - the address is not
present
user.addresses.remove(someaddress) # -- ERROR - the address is not
present
user.addresses.insert(5, someotheraddress) # -- FAIL - the list is
incomplete, ordering will be incorrect
This is only a matter of getattr and __contains__ triggering init,
right? (At least if we exclude the other problems pointed above).
session.flush() # -- FAIL - we have to figure out what items were
added/removed/unchanged from the collection...but the data's
incomplete !
I don't master SQLAlchemy internals but I don't see how that is
different from when the collection is complete?
so as long as we can agree on the its a read-only thing aspect of
this, we're good to go. otherwise you have to define for me how all
those mutating operations are going to work (and even then, its
additional core complexity im not sure if i can add to my support-load).
I'm fine with the readonly aspect of it. What I don't like is the fact
you have to create a readonly relation (lazyloader/whatever/...) in
advance (ie in your mapper), which is IMHO just a dupe of the normal
relation and pollutes the mapper. You'd end up with mappers like this:
mapper(SomeClass, table, properties={
'addresses':relation(Address)
'addresses2':lazyloader(Address)
})
which is pretty much as ugly as you can get.
On the other hand, I think that combined with a quick way to have
predefined filters it might be a nice addition anyway:
mapper(SomeClass, table, properties={
'addresses': relation(Address)
'local_addresses': lazyloader(Address,
filter=address.c.postcode==5000)
})
But it does in no case replace the dynamic non-polluting use-case
I'd like to have. What I had in mind is to reuse normal relations to
get a query. It feels much more natural and cleaner to me. And I think
the best compromise would be something along the lines of:
user.addresses: # use standard relation = read/write
user.addresses.filter(XXX): # returns a query = read only
the code would probably be cleaner if we did something more explicit like:
user.addresses.query # returns the query object that you can filter, etc...
though, as a user, I'd prefer the first solution.
Wouldn't that be possible? I think it should be. You only need to keep
the deferred approach of the InstrumentedList that I demonstrated in
my patch, so that the whole list is not fetched before we get the
query object, which would ruin the whole idea. Of course it was only a
proof-of-concept patch, but I think it should be fixable.
--
Gaëtan de Menten
http://openhex.org
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: [PATCH] Filtered one_to_many relationships (Experimental)
And by the way, if you agree with that direction of things, I'd
happily work on a patch for the query-on-relation thing.
On 3/26/07, Gaetan de Menten [EMAIL PROTECTED] wrote:
On 3/23/07, Michael Bayer [EMAIL PROTECTED] wrote:
OK, this is actually something people have asked for a lot. in the
beginning, recently, etc. also for different reasons...i.e.
convenience, or performance, etc. So, first off let me start by
illustrating how this use case is done right now. Assuming your
Address mapper has a backref user to the User mapper, its just:
for address in
session.query(Address).filter_by(user=someuser).filter
(address_table.c.postcode == 5000):
print address.street
Once again, I discover a better way to do something I did the hard way.
But, the join conditions and bind params which
have been calculated by LazyLoader are just sitting there, they can
be currently pulled out with a little non-API attribute access but
Ive no problem with adding some API-level accessors to get at the
Query object calculated for a particular property (i.e. what you are
using in your patch internally).
Yes, please do, that'd probably solve the problem nicely (see below
how I see things).
now lets look at the way your patch does it.
addresses = user.addresses.filter(address_table.c.postcode == 5000)
seems easy. right ? remember that user is now in the session.
anyone else that queries for user will get that same User
instance. but the rest of the app is going to assume normal
relationship semantics on that collectionwhich means:
How I envisioned things, this wouldn't be a problem, because
user.addresses.filter(xxx) would return a new, independant list, which
doesn't affect user.addresses, and is not affected if user.addresses
has already been accessed or not. This is not what my patch does, I
know. Sorry for not explaining this in my first mail.
print someaddress in user.addresses # -- FAIL - the address is not
present
user.addresses.remove(someaddress) # -- ERROR - the address is not
present
user.addresses.insert(5, someotheraddress) # -- FAIL - the list is
incomplete, ordering will be incorrect
This is only a matter of getattr and __contains__ triggering init,
right? (At least if we exclude the other problems pointed above).
session.flush() # -- FAIL - we have to figure out what items were
added/removed/unchanged from the collection...but the data's
incomplete !
I don't master SQLAlchemy internals but I don't see how that is
different from when the collection is complete?
so as long as we can agree on the its a read-only thing aspect of
this, we're good to go. otherwise you have to define for me how all
those mutating operations are going to work (and even then, its
additional core complexity im not sure if i can add to my support-load).
I'm fine with the readonly aspect of it. What I don't like is the fact
you have to create a readonly relation (lazyloader/whatever/...) in
advance (ie in your mapper), which is IMHO just a dupe of the normal
relation and pollutes the mapper. You'd end up with mappers like this:
mapper(SomeClass, table, properties={
'addresses':relation(Address)
'addresses2':lazyloader(Address)
})
which is pretty much as ugly as you can get.
On the other hand, I think that combined with a quick way to have
predefined filters it might be a nice addition anyway:
mapper(SomeClass, table, properties={
'addresses': relation(Address)
'local_addresses': lazyloader(Address,
filter=address.c.postcode==5000)
})
But it does in no case replace the dynamic non-polluting use-case
I'd like to have. What I had in mind is to reuse normal relations to
get a query. It feels much more natural and cleaner to me. And I think
the best compromise would be something along the lines of:
user.addresses: # use standard relation = read/write
user.addresses.filter(XXX): # returns a query = read only
the code would probably be cleaner if we did something more explicit like:
user.addresses.query # returns the query object that you can filter, etc...
though, as a user, I'd prefer the first solution.
Wouldn't that be possible? I think it should be. You only need to keep
the deferred approach of the InstrumentedList that I demonstrated in
my patch, so that the whole list is not fetched before we get the
query object, which would ruin the whole idea. Of course it was only a
proof-of-concept patch, but I think it should be fixable.
--
Gaëtan de Menten
http://openhex.org
--
Gaëtan de Menten
http://openhex.org
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To
[sqlalchemy] Re: [PATCH] Filtered one_to_many relationships (Experimental)
On Mar 26, 2007, at 6:15 AM, Gaetan de Menten wrote: though, as a user, I'd prefer the first solution. Wouldn't that be possible? I think it should be. You only need to keep the deferred approach of the InstrumentedList that I demonstrated in my patch, so that the whole list is not fetched before we get the query object, which would ruin the whole idea. Of course it was only a proof-of-concept patch, but I think it should be fixable. my various issues with the deferred thing are as follows. note that im not putting these out there as this is why we arent doing it, im putting it out there as this is why it makes me uncomfortable. the current use case of has my lazy list been loaded? is: addresses in myobject.__dict__ with deferred list, now we have to have a list element actually present there (but still loaded, maybe not). so detecting when an attribute requires its callable fired off or not gets thorny...also breaks code for those who do it the above way, but also we need to add some new way to accomplish the above, which will probably have to be some messy function call like attribute_manager.is_loaded (myobject, addresses). generally theres all sorts of places where we want to get at the attribute and fire it off, not fire it off (internally known as passive), etc. and the awareness of the deferred list there would have to be more deeply embedded throughout the attributes module for everything to keep working. so my discomfort grows that we are changing the APIs of attributes.py, probably including its public API, just to suit something that IMHO is strictly for visual appeal. also, while it looks cleaner, there is still a semantic co-mingling that im not very comfortable with. i.e. that a collection of persisted objects on a parent class doubles as a database query object. to me the meaning of those two things is entirely different, and i think the decline of an API starts with minor conflation of concepts. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: [PATCH] Filtered one_to_many relationships (Experimental)
i vote Query(). No surprise there ;-) Me too, +1 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: [PATCH] Filtered one_to_many relationships (Experimental)
OK, this is actually something people have asked for a lot. in the
beginning, recently, etc. also for different reasons...i.e.
convenience, or performance, etc. So, first off let me start by
illustrating how this use case is done right now. Assuming your
Address mapper has a backref user to the User mapper, its just:
for address in session.query(Address).filter_by(user=someuser).filter
(address_table.c.postcode == 5000):
print address.street
and of course, if you dont have the backref, then we go back to the
regular SQL stuff:
for address in session.query(Address).filter
(user.id==someuser.id).filter(address_table.c.postcode == 5000):
print address.street
in the second example, we are figuring out the lazy criterion
ourselves. in most cases this is pretty easy to figure out. but we
do have some exotic mappings these days, primarily because of
polymorphic union queries, and it might not be as trivial to make up
the join condition. But, the join conditions and bind params which
have been calculated by LazyLoader are just sitting there, they can
be currently pulled out with a little non-API attribute access but
Ive no problem with adding some API-level accessors to get at the
Query object calculated for a particular property (i.e. what you are
using in your patch internally).
So if you want to take the above and make it all slick, all you have
to do is stick the above query inside a property:
class User(object):
def _get_addresses(self):
return object_session(self).query(Address).filter
(user.id==someuser.id)
addresses = property(_get_addresses)
then you can just say pretty much what you want:
for address in someuser.addresses.filter(address_table.c.postcode ==
5000):
print address.street
plus, the addresses object is a Query ! so you already have the
full interface, and it has __iter__ and __getitem__() ! so the list
operations on the base object work just fine (and they even work like
SQLObjects)
for address in someuser.addresses: # __iter__() fires off the query !
for address in someuser.addresses[3:5] # slice applies OFFSET and
LIMIT !
now lets look at the way your patch does it.
addresses = user.addresses.filter(address_table.c.postcode == 5000)
seems easy. right ? remember that user is now in the session.
anyone else that queries for user will get that same User
instance. but the rest of the app is going to assume normal
relationship semantics on that collectionwhich means:
print user.addresess # -- FAIL - this code assumes the list is
complete, its not, its already been loaded incompletely
print someaddress in user.addresses # -- FAIL - the address is not
present
user.addresses.remove(someaddress) # -- ERROR - the address is not
present
user.addresses.insert(5, someotheraddress) # -- FAIL - the list is
incomplete, ordering will be incorrect
session.flush() # -- FAIL - we have to figure out what items were
added/removed/unchanged from the collection...but the data's
incomplete !
This is an issue introduced by our usage of sessions, identity maps,
etc., things that simpler ORMs dont have to worry about. So thats
the part of this I dont want to get into (and im hoping you dont,
either). Yes, the use case you want is desireable, and ive no
problem adding in hooks to make it possible. but No, I dont think it
should be injected into the base attribute lazy-loading operation.
i think the semantics of a Query for related items and that of a
mapped relationship are just plain different, even though
implementation-wise on the view side at least they are both just a
SQL query. with the primary issue being that all mutating/uow-
related operations go out the window with an incomplete collection.
from a performance point of view, the manipulation of huge
collections could be achieved via loading stub objects which is
something discussed long ago, which is how Hibernate addresses it
(called extra-lazy loading). so if we ever did that, thats how we
would do it (i.e. a complete collection of incomplete objects).
anyway back to the query side, Ive even been considering making folks
happy with this and adding a new kind of mapped property called, i
dont knowquery_relation(), relquery(), lazyloader(),
something like that. and, right in the core, how often does that
happen ? it would copy the setup style of relation() but be a lot
more view oriented.
mapper(SomeClass, table, properties={
'addresses':lazyloader(Address)
})
and there you go ! does the same thing as the manual property code I
illustrated does, returns a Query from which you can do whatever.
We also had a guy who wanted to do a more complex primaryjoin
