[sqlalchemy] Re: eagerloading polymorphic mapper
On Jan 16, 2008, at 2:24 AM, [EMAIL PROTECTED] wrote: yeahhh i know i dont see the wider sql picture... i may put all this as a ticket to remind.. maybe one day you'll be in better mood (;-) it can certainly be done but id like to at least first get eager loads from the *base* polymorphic mapper working...query has been steadily heading towards a more sophisticated and singular notion (such as, now you can do your query.select_from(any selectable).filter() which you wanted months ago) and i think theres a certain order things need to be done. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: eagerloading polymorphic mapper
well, i tried it manualy and it works (sqlite). here the eagerloading query: model-description (joined_inheritance): class A: name = Text() class B(A): address = reference( Adress) nom = reference( Nomerator) class C(A): pass select * from A left outer join B on A.db_id = B.db_id left outer join C on A.db_id = C.db_id left outer join Address as a1 on p.address_id = address.db_id left outer join Nomerator as a2 on p.nom_id = a2.db_id ; or with subselects: select * from A left outer join B on A.db_id = B.db_id left outer join C on A.db_id = C.db_id left outer join ( select * from Address ) as a1 on p.address_id = address.db_id left outer join ( select * from Nomerator) as a2 on p.nom_id = a2.db_id ; also, i dont see a reason for it not to work if the (A jon B join C) is a polymunion - all the same, all columns will be present there, having None where missing. 0.4.3? On Monday 14 January 2008 18:56:16 svilen wrote: On Monday 14 January 2008 18:35:40 Michael Bayer wrote: On Jan 14, 2008, at 11:29 AM, svilen wrote: On Monday 14 January 2008 17:19:14 Michael Bayer wrote: On Jan 14, 2008, at 8:41 AM, svilen wrote: i have, say, base class A, inherited by two children B and C. B has an attribute/relation 'address', A and C do not have it. So i had a query(A).eagerload( 'address') and that did work before r3912. But later it gives an error - mapper|A has no property 'address'. Any hint how to do it now? what kind of inheritance/mapping from A-B ? i cant really imagine any way that kind of eager load could have worked since the address property of B does not (and has never) get consulted in that case. plain joined?... hmm. maybe it did not really work (eagerly) but lazy-load has fired instead... seems that's the case. anyway. some way to accomplish such thing? no ! this the same issue with the Channel-CatalogChannel thing, yes i guessed it.. your query is against A...attributes that are only on B don't enter into the equation here. this is somewhat different, my query/filter is on attributes that do exist in A; i only want the ORM to postprocess certain things... there will be 'address' column in the result-set anyway (empty or not), why it cannot be eagerloaded via B.address? But also, if youre using select_table, we dont yet support eager loads from a polymorphic-unioned mapper in any case (though we are close). it is not polymunion, joined_inh works via left-outer-join. well, no is no. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: eagerloading polymorphic mapper
On Jan 15, 2008, at 4:33 AM, svilen wrote: also, i dont see a reason for it not to work if the (A jon B join C) is a polymunion - all the same, all columns will be present there, having None where missing. 0.4.3? unlikely, I dont see how it could work from a generic standpoint. the query generates SQL based on the attributes attached to A. if it had to also loop through all the attributes of B, C, D, E, F, etc. and attempt to have all of those add their clauses to the SQL, theyd all have to assume that the selectable for A even supports receiving their joins, etc. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: eagerloading polymorphic mapper
On Tuesday 15 January 2008 17:19:49 Michael Bayer wrote: On Jan 15, 2008, at 4:33 AM, svilen wrote: also, i dont see a reason for it not to work if the (A jon B join C) is a polymunion - all the same, all columns will be present there, having None where missing. 0.4.3? unlikely, I dont see how it could work from a generic standpoint. the query generates SQL based on the attributes attached to A. if it had to also loop through all the attributes of B, C, D, E, F, etc. and attempt to have all of those add their clauses to the SQL, theyd all have to assume that the selectable for A even supports receiving their joins, etc. hmmm, specify explicitly? e.g. query(A).eagerload( B.address) joined-inh via left-outer-join is enough, no need for polymunion. IMO this will be big plus for the ORM - eagerloading polymorphical child attributes - moving further away from SQL-like-looking stuff. i dont know how the current machinery for eagerload works, but imo knowing your level of lookahead-design, it should not be hard to apply that machinery over a polymorphic mapper/query? ciao svilen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: eagerloading polymorphic mapper
On Jan 15, 2008, at 10:49 AM, svilen wrote: hmmm, specify explicitly? e.g. query(A).eagerload( B.address) joined-inh via left-outer-join is enough, no need for polymunion. uh well i supposeeagerload options dont really affect the traversal that way and it would take some non-trivial rearrangement of internals. IMO this will be big plus for the ORM - eagerloading polymorphical child attributes - moving further away from SQL-like-looking stuff. we *like* SQL ! we dont want to become OQL. i dont know how the current machinery for eagerload works, but imo knowing your level of lookahead-design, it should not be hard to apply that machinery over a polymorphic mapper/query? theres plenty of much higher priority issues than this one in the queue...considering that you can already get the results you want with this one using direct SQL. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: eagerloading polymorphic mapper
hmmm, specify explicitly?
e.g. query(A).eagerload( B.address)
joined-inh via left-outer-join is enough, no need for polymunion.
i dont know how the current machinery for eagerload works, but imo
knowing your level of lookahead-design, it should not be hard to
apply that machinery over a polymorphic mapper/query?
theres plenty of much higher priority issues than this one in the
queue...considering that you can already get the results you want with
this one using direct SQL.
right..
i've hacked something that seems to work; It's about 20 lines split in
orm.query and orm.interfaces:
- such special eagerloaders are requested as query.eagerload( B.address) -
and not just the name/path
- query-compile calling context.exec_withpath(...) iterates over all
self.mapper properties (not only select_mapper's), plus all eagerloaders of
above type (i.e. non-names). Thus the 4 cases are covered:
A has address / query(B).eagerload('address') #works before
A has address / query(A).eagerload('address') #new - did not work before
B has address / query(B).eagerload('address') #works before
B has address / query(A).eagerload(B.address) #new - not possible before
(in all these B inherits A via joined inheritance; A is polymorphic via
left-outer-joins)
i'm absolutely sure that this is not the completely right thing - that's
what i got from the machinery-src in 2 hours - but it is something as a
start... sure it needs correctness tests etc of the sorts.
g'night
svilen
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
Index: orm/query.py
===
--- orm/query.py (revision 4032)
+++ orm/query.py (working copy)
@@ -991,10 +991,20 @@
# give all the attached properties a chance to modify the query
# TODO: doing this off the select_mapper. if its the polymorphic mapper, then
# it has no relations() on it. should we compile those too into the query ? (i.e. eagerloads)
-for value in self.select_mapper.iterate_properties:
+
+for value in self.mapper.iterate_properties:
+if self._only_load_props and value.key not in self._only_load_props:
+continue
+context.exec_with_path(self.mapper, value.key, value.setup, context, only_load_props=self._only_load_props)
+for (mp, key) in self._eager_loaders:
+if isinstance( key, str): continue #plain
+value = key.property
if self._only_load_props and value.key not in self._only_load_props:
continue
-context.exec_with_path(self.select_mapper, value.key, value.setup, context, only_load_props=self._only_load_props)
+context.exec_with_path(self.mapper, key, value.setup, context, only_load_props=self._only_load_props)
# additional entities/columns, add those to selection criterion
for tup in self._entities:
Index: orm/interfaces.py
===
--- orm/interfaces.py (revision 4032)
+++ orm/interfaces.py (working copy)
@@ -594,6 +599,7 @@
raise exceptions.ArgumentError(Can't find entity %s in Query. Current list: %r % (str(mapper), [str(m) for m in [query.mapper] + query._entities]))
else:
mapper = query.mapper
+if isinstance( self.key,str):
for token in self.key.split('.'):
if current_path and token == current_path[1]:
current_path = current_path[2:]
@@ -604,6 +610,16 @@
path = build_path(mapper, prop.key, path)
l.append(path)
mapper = getattr(prop, 'mapper', None)
+else:
+ia = self.key
+key = ia.impl.key
+#from sqlalchemy.orm import class_mapper
+#mp = mapper#class_mapper( ia.impl.class_) #assert mp inherits mapper?
+self.key = key
+path = build_path(mapper, ia, path)
+l.append(path)
return l
PropertyOption.logger = logging.class_logger(PropertyOption)
[sqlalchemy] Re: eagerloading polymorphic mapper
On Jan 15, 2008, at 5:17 PM, [EMAIL PROTECTED] wrote:
hmmm, specify explicitly?
e.g. query(A).eagerload( B.address)
joined-inh via left-outer-join is enough, no need for polymunion.
i dont know how the current machinery for eagerload works, but imo
knowing your level of lookahead-design, it should not be hard to
apply that machinery over a polymorphic mapper/query?
theres plenty of much higher priority issues than this one in the
queue...considering that you can already get the results you want
with
this one using direct SQL.
right..
i've hacked something that seems to work; It's about 20 lines split in
orm.query and orm.interfaces:
- such special eagerloaders are requested as
query.eagerload( B.address) -
and not just the name/path
- query-compile calling context.exec_withpath(...) iterates over all
self.mapper properties (not only select_mapper's), plus all
eagerloaders of
above type (i.e. non-names). Thus the 4 cases are covered:
A has address / query(B).eagerload('address') #works before
A has address / query(A).eagerload('address') #new - did not work
before
B has address / query(B).eagerload('address') #works before
B has address / query(A).eagerload(B.address) #new - not possible
before
(in all these B inherits A via joined inheritance; A is polymorphic
via
left-outer-joins)
i'm absolutely sure that this is not the completely right thing -
that's
what i got from the machinery-src in 2 hours - but it is something
as a
start... sure it needs correctness tests etc of the sorts.
yeah thats the idea but it needs more work than that. for one thing
you might be hitting the same MappedProperty twice using that
iteration (therefore joining twice), and also it doesn't account for
eager loaders like eagerload(foo.bar.bat); i.e. deeper level
properties which would need to have an adjusted path sent to them
(or ignored in that part of the iteration).
also i can see a lot of cases where the eager loader from B is going
to generate invalid SQL, such as joined table inheritance with no
select_table, the query is only generated against A. B.address
is going to try generating an eager join against the B table which
isnt present, and youll get some kind of mess as a result. checking
for this condition beforehand is bound to add lots of complexity and i
only want to add features like these if they can be smoothly
integrated, not lots of extra if/thens bolted on.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: eagerloading polymorphic mapper
Michael Bayer wrote:
On Jan 15, 2008, at 5:17 PM, [EMAIL PROTECTED] wrote:
hmmm, specify explicitly?
e.g. query(A).eagerload( B.address)
joined-inh via left-outer-join is enough, no need for polymunion.
i dont know how the current machinery for eagerload works, but imo
knowing your level of lookahead-design, it should not be hard to
apply that machinery over a polymorphic mapper/query?
theres plenty of much higher priority issues than this one in the
queue...considering that you can already get the results you want
with
this one using direct SQL.
right..
i've hacked something that seems to work; It's about 20 lines split in
orm.query and orm.interfaces:
- such special eagerloaders are requested as
query.eagerload( B.address) -
and not just the name/path
- query-compile calling context.exec_withpath(...) iterates over all
self.mapper properties (not only select_mapper's), plus all
eagerloaders of
above type (i.e. non-names). Thus the 4 cases are covered:
A has address / query(B).eagerload('address') #works before
A has address / query(A).eagerload('address') #new - did not work
before
B has address / query(B).eagerload('address') #works before
B has address / query(A).eagerload(B.address) #new - not possible
before
(in all these B inherits A via joined inheritance; A is polymorphic
via
left-outer-joins)
i'm absolutely sure that this is not the completely right thing -
that's
what i got from the machinery-src in 2 hours - but it is something
as a
start... sure it needs correctness tests etc of the sorts.
yeah thats the idea but it needs more work than that.
for one thing
you might be hitting the same MappedProperty twice using that
iteration (therefore joining twice), and also it doesn't account for
eager loaders like eagerload(foo.bar.bat); i.e. deeper level
properties which would need to have an adjusted path sent to them
(or ignored in that part of the iteration).
pure textual paths are going same (old) way; it needs some
extra-syntax/API for multilevel descriptor-specified eagers; e.g
eagerload( (B.foo, 'bar', C.bat ) )
also i can see a lot of cases where the eager loader from B is going
to generate invalid SQL, such as joined table inheritance with no
select_table, the query is only generated against A. B.address
is going to try generating an eager join against the B table which
isnt present, and youll get some kind of mess as a result. checking
for this condition beforehand is bound to add lots of complexity and i
only want to add features like these if they can be smoothly
integrated, not lots of extra if/thens bolted on.
yeahhh i know i dont see the wider sql picture...
i may put all this as a ticket to remind.. maybe one day you'll be in better
mood (;-)
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: eagerloading polymorphic mapper
On Jan 14, 2008, at 8:41 AM, svilen wrote: i have, say, base class A, inherited by two children B and C. B has an attribute/relation 'address', A and C do not have it. So i had a query(A).eagerload( 'address') and that did work before r3912. But later it gives an error - mapper|A has no property 'address'. Any hint how to do it now? what kind of inheritance/mapping from A-B ? i cant really imagine any way that kind of eager load could have worked since the address property of B does not (and has never) get consulted in that case. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: eagerloading polymorphic mapper
On Monday 14 January 2008 17:19:14 Michael Bayer wrote: On Jan 14, 2008, at 8:41 AM, svilen wrote: i have, say, base class A, inherited by two children B and C. B has an attribute/relation 'address', A and C do not have it. So i had a query(A).eagerload( 'address') and that did work before r3912. But later it gives an error - mapper|A has no property 'address'. Any hint how to do it now? what kind of inheritance/mapping from A-B ? i cant really imagine any way that kind of eager load could have worked since the address property of B does not (and has never) get consulted in that case. plain joined?... hmm. maybe it did not really work (eagerly) but lazy-load has fired instead... seems that's the case. anyway. some way to accomplish such thing? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: eagerloading polymorphic mapper
On Jan 14, 2008, at 11:29 AM, svilen wrote: On Monday 14 January 2008 17:19:14 Michael Bayer wrote: On Jan 14, 2008, at 8:41 AM, svilen wrote: i have, say, base class A, inherited by two children B and C. B has an attribute/relation 'address', A and C do not have it. So i had a query(A).eagerload( 'address') and that did work before r3912. But later it gives an error - mapper|A has no property 'address'. Any hint how to do it now? what kind of inheritance/mapping from A-B ? i cant really imagine any way that kind of eager load could have worked since the address property of B does not (and has never) get consulted in that case. plain joined?... hmm. maybe it did not really work (eagerly) but lazy-load has fired instead... seems that's the case. anyway. some way to accomplish such thing? no ! this the same issue with the Channel-CatalogChannel thing, your query is against A...attributes that are only on B don't enter into the equation here.But also, if youre using select_table, we dont yet support eager loads from a polymorphic-unioned mapper in any case (though we are close). --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
