[sqlalchemy] Re: r3727 / AbstractClauseProcessor problem
On Nov 9, 2007, at 7:16 PM, [EMAIL PROTECTED] wrote: huh? it dies here. r3727 or 3760 all the same, py2.5..., did remove all *pyc the test is committed in r3763. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: r3727 / AbstractClauseProcessor problem
one more error in ACP, took me a day to find and separate.
it's very simple and very basic... ClauseAdapter does not work.
--
from sqlalchemy import *
from sqlalchemy.sql.util import ClauseAdapter
m = MetaData()
a=Table( 'a',m,
Column( 'id',Integer, primary_key=True),
Column( 'xxx_id', Integer, ForeignKey( 'a.id', name='adf',
use_alter=True ) )
)
e = (a.c.id == a.c.xxx_id)
print e
b = a.alias()
#print b.c.id == 0
r = ClauseAdapter( b, include= set([ a.c.id ]),
equivalents= { a.c.id: set([ a.c.id]) }
).traverse( e)
print e
# results
#in r3726: (OK)
a.id = a.xxx_id
a_1.id = a.xxx_id
#in r3727 - as well as in r3760:
a.id = a.xxx_id
a.id = a.xxx_id
it does generate proper thing in ClauseAdapter.conv_element()/newcol but
it gets lost in the way
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: r3727 / AbstractClauseProcessor problem
On Nov 9, 2007, at 6:49 PM, [EMAIL PROTECTED] wrote:
om sqlalchemy import *
from sqlalchemy.sql.util import ClauseAdapter
m = MetaData()
a=Table( 'a',m,
Column( 'id',Integer, primary_key=True),
Column( 'xxx_id', Integer, ForeignKey( 'a.id', name='adf',
use_alter=True ) )
)
e = (a.c.id == a.c.xxx_id)
print e
b = a.alias()
#print b.c.id == 0
r = ClauseAdapter( b, include= set([ a.c.id ]),
equivalents= { a.c.id: set([ a.c.id]) }
).traverse( e)
print e
# results
#in r3726: (OK)
a.id = a.xxx_id
a_1.id = a.xxx_id
#in r3727 - as well as in r3760:
a.id = a.xxx_id
a.id = a.xxx_id
no, it works, it just clones in all cases:
sql_util.ClauseAdapter( b, include= set([ a.c.id ]),
equivalents= { a.c.id: set([ a.c.id]) }
).traverse( e)
assert str(e) == a_1.id = a.xxx_id
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: r3727 / AbstractClauseProcessor problem
om sqlalchemy import *
from sqlalchemy.sql.util import ClauseAdapter
m = MetaData()
a=Table( 'a',m,
Column( 'id',Integer, primary_key=True),
Column( 'xxx_id', Integer, ForeignKey( 'a.id', name='adf',
use_alter=True ) )
)
e = (a.c.id == a.c.xxx_id)
print e
b = a.alias()
#print b.c.id == 0
r = ClauseAdapter( b, include= set([ a.c.id ]),
equivalents= { a.c.id: set([ a.c.id]) }
).traverse( e)
print e
# results
#in r3726: (OK)
a.id = a.xxx_id
a_1.id = a.xxx_id
#in r3727 - as well as in r3760:
a.id = a.xxx_id
a.id = a.xxx_id
no, it works, it just clones in all cases:
sql_util.ClauseAdapter( b, include= set([ a.c.id ]),
equivalents= { a.c.id: set([ a.c.id]) }
).traverse( e)
assert str(e) == a_1.id = a.xxx_id
huh? it dies here. r3727 or 3760 all the same, py2.5..., did remove all
*pyc
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: r3727 / AbstractClauseProcessor problem
On Nov 9, 2007, at 7:26 PM, Michael Bayer wrote:
On Nov 9, 2007, at 6:49 PM, [EMAIL PROTECTED] wrote:
om sqlalchemy import *
from sqlalchemy.sql.util import ClauseAdapter
m = MetaData()
a=Table( 'a',m,
Column( 'id',Integer, primary_key=True),
Column( 'xxx_id', Integer, ForeignKey( 'a.id', name='adf',
use_alter=True ) )
)
e = (a.c.id == a.c.xxx_id)
print e
b = a.alias()
#print b.c.id == 0
r = ClauseAdapter( b, include= set([ a.c.id ]),
equivalents= { a.c.id: set([ a.c.id]) }
).traverse( e)
print e
# results
#in r3726: (OK)
a.id = a.xxx_id
a_1.id = a.xxx_id
#in r3727 - as well as in r3760:
a.id = a.xxx_id
a.id = a.xxx_id
no, it works, it just clones in all cases:
sql_util.ClauseAdapter( b, include= set([ a.c.id ]),
equivalents= { a.c.id: set([ a.c.id]) }
).traverse( e)
assert str(e) == a_1.id = a.xxx_id
oof, try again, notice the assignment:
e = sql_util.ClauseAdapter( b, include= set([ a.c.id ]),
equivalents= { a.c.id: set([ a.c.id]) }
).traverse( e)
print e
assert str(e) == a_1.id = a.xxx_id
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: r3727 / AbstractClauseProcessor problem
On Nov 8, 2007, at 2:01 AM, [EMAIL PROTECTED] wrote:
i dont really understand why u need the ACP being so different to
plain
visitor; i mean cant they share some skeleton part of traversing,
while
putting all the choices (visit* vs convert; onentry/onexit; stop/dont)
in their own parts.
After all, visitor pattern is twofold, a) Guide + b) Visitor; the
Guide
doing traversing, the Visitor noting things; choice where to go
might be
in visitor and/or in guide. some times (one extreme) the visitor is
just
one dumb functor; other cases (other extreme end) the visitor is very
sofisticated and even does guiding/traversing.
Here it looks more like second case, u have most of both sides put in
the Visitor, and only small part (specific visit_* / copy_internals)
left to the actual nodes.
And to me, the skeleton is still same between ACP and ClauseVisitor.
you cant use plain visitor beacuse you are copying the whole structure
in place at the same time, and a method is deciding arbitrarily to not
copy certain elements, and instead returns an element that was set
from the outside; that element cannot be mutated since its not from
the original structure, therefore it cannot be traversed.
as it turns out, this visitor still has lots of problems which will
continue to prevent more sophisticated copy-and-replace
operations...some of the ways that a Select() just works from the
ground up just get in the way here.
if you are curious why there was a rewrite, heres the test which
needed to pass:
metadata = MetaData()
a = Table('a', metadata,
Column('id', Integer, primary_key=True))
b = Table('b', metadata,
Column('id', Integer, primary_key=True),
Column('aid', Integer, ForeignKey('a.id')),
)
c = Table('c', metadata,
Column('id', Integer, primary_key=True),
Column('bid', Integer, ForeignKey('b.id')),
)
d = Table('d', metadata,
Column('id', Integer, primary_key=True),
Column('aid', Integer, ForeignKey('a.id')),
)
j1 = a.outerjoin(b)
j2 = select([j1], use_labels=True)
j3 = c.join(j2, j2.c.b_id==c.c.bid)
j4 = j3.outerjoin(d)
self.assert_compile(j4, c JOIN (SELECT a.id AS a_id, b.id
AS b_id, b.aid AS b_aid FROM a LEFT OUTER JOIN b ON a.id = b.aid)
ON b_id = c.bid
LEFT OUTER JOIN d ON a_id = d.aid)
j5 = j3.alias('foo')
j6 = sql_util.ClauseAdapter(j5).copy_and_process([j4])[0]
# this statement takes c join(a join b), wraps it inside an
aliased select * from c join(a join b) AS foo.
# the outermost right side left outer join d stays the
same, except d joins against foo.a_id instead
# of plain a_id
self.assert_compile(j6, (SELECT c.id AS c_id, c.bid AS
c_bid, a_id AS a_id, b_id AS b_id, b_aid AS b_aid FROM
c JOIN (SELECT a.id AS a_id, b.id AS
b_id, b.aid AS b_aid FROM a LEFT OUTER JOIN b ON a.id = b.aid)
ON b_id = c.bid) AS foo
LEFT OUTER JOIN d ON foo.a_id =
d.aid)
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: r3727 / AbstractClauseProcessor problem
On Nov 8, 2007, at 11:32 AM, svilen wrote: mmmh. u can think of splitting the Visitor into 3: Guide (who traverses _everything_ given), Visitor (who does things), and intermediate Decisor, who decides where to go / what to do. But this can get complicated (slow) although it would be quite clear who does what. Also, do have both onEntry and onExit for each node; i am sure some operations will require both; first to gather info, second to make a decision about what to do with it while still at that level. i've done quite a lot of tree/expression traversers, and while readonly walking doesnot care much if on entry or on exit (except if u want depth or breadth first), replacements-in-place and copy+replace sometimes needed both entry and exit hooks, + they where conditional like in leafs. i think you should come up with your own ACP and lets take a look at it.while i can get various ACP ideas to work further and further, im still able to come up with plenty of cases where none of them work and its because the structure of a clauseelement really isnt a tree. the same node can be represented many times largely because columns reference their parent table. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: r3727 / AbstractClauseProcessor problem
Michael Bayer wrote: On Nov 8, 2007, at 11:32 AM, svilen wrote: mmmh. u can think of splitting the Visitor into 3: Guide (who traverses _everything_ given), Visitor (who does things), and intermediate Decisor, who decides where to go / what to do. But this can get complicated (slow) although it would be quite clear who does what. Also, do have both onEntry and onExit for each node; i am sure some operations will require both; first to gather info, second to make a decision about what to do with it while still at that level. i've done quite a lot of tree/expression traversers, and while readonly walking doesnot care much if on entry or on exit (except if u want depth or breadth first), replacements-in-place and copy+replace sometimes needed both entry and exit hooks, + they where conditional like in leafs. i think you should come up with your own ACP and lets take a look at it.while i can get various ACP ideas to work further and further, im still able to come up with plenty of cases where none of them work and its because the structure of a clauseelement really isnt a tree. the same node can be represented many times largely because columns reference their parent table. u mean a graph? mmm no. IMO an expression is a tree of nodes, each node points to some element of another space (metadata things+bindparams+...), and many nodes can point to same element, and eventualy many elements can hold same value (but be different elements - e.g. literal(1) and another literal(1) - not sure about whether this is usable). So traversing the (original) tree is one thing; what to do with the elements pointed by the nodes is another decision; e.g.. whether to process multiple-referred elements multiple times or just one, etc. i can try... but dont rely on me too much ;-) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: r3727 / AbstractClauseProcessor problem
heres an entirely valid SA expression:
subq = t2.select().alias('subq')
s = select([t1.c.col1, subq.c.col1], from_obj=[t1, subq,
t1.join(subq, t1.c.col1==subq.c.col2)])
the way the above works is, t1.join(subq) sends a message to the
enclosing Select to hide t1 and subq individually in the FROM
clause. this is some pretty old logic but its still whats in effect
today and it makes it easy to build up Select statements without
worrying whats already in the FROM clause.
traversing that, we will hit subq at least twice. r3755 makes more
changes to accomodate the above, where it only clones subq once.
the resulting clone still has some old subq references lying around
though...instead of trying to make the traversal crazy so that it
finds and exactly replaces every occurence of subq or subq-
referencing column with the exact correct replacement, i changed
Select so that when it constructs the FROM clause it takes into
account the old instance of subq as well as the cloned
version...since thats really the only place that multiple copies of
subq really matter in the final output.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: r3727 / AbstractClauseProcessor problem
i dont really understand why u need the ACP being so different to plain visitor; i mean cant they share some skeleton part of traversing, while putting all the choices (visit* vs convert; onentry/onexit; stop/dont) in their own parts. After all, visitor pattern is twofold, a) Guide + b) Visitor; the Guide doing traversing, the Visitor noting things; choice where to go might be in visitor and/or in guide. some times (one extreme) the visitor is just one dumb functor; other cases (other extreme end) the visitor is very sofisticated and even does guiding/traversing. Here it looks more like second case, u have most of both sides put in the Visitor, and only small part (specific visit_* / copy_internals) left to the actual nodes. And to me, the skeleton is still same between ACP and ClauseVisitor. you cant use plain visitor beacuse you are copying the whole structure in place at the same time, and a method is deciding arbitrarily to not copy certain elements, and instead returns an element that was set from the outside; that element cannot be mutated since its not from the original structure, therefore it cannot be traversed. well this is a behavior that can be controlled - to traverse the originals that are to be replaced, or not; and to traverse the replacement AFTER it has been replaced or not. as it turns out, this visitor still has lots of problems which will continue to prevent more sophisticated copy-and-replace operations...some of the ways that a Select() just works from the ground up just get in the way here. mmmh. u can think of splitting the Visitor into 3: Guide (who traverses _everything_ given), Visitor (who does things), and intermediate Decisor, who decides where to go / what to do. But this can get complicated (slow) although it would be quite clear who does what. Also, do have both onEntry and onExit for each node; i am sure some operations will require both; first to gather info, second to make a decision about what to do with it while still at that level. i've done quite a lot of tree/expression traversers, and while readonly walking doesnot care much if on entry or on exit (except if u want depth or breadth first), replacements-in-place and copy+replace sometimes needed both entry and exit hooks, + they where conditional like in leafs. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: r3727 / AbstractClauseProcessor problem
heres the structure of: select(from_obj=[t1, t2, t1.join(t2)])
select +--- t1 -+
|--- t2 |
+--- join of t1/t2 ---+
t2 and t1 both have two parents, and there are two paths to each of t1
and t2 from the head select. so its not a tree in the strict sense.
or another one:
s1 = t1.select().alias('s1')
s2 = t1.select().alias('s2')
s3 = s1.union(s2) (two paths to t1: s3-s1-t1, s3-s2-t1)
any kind of subquery which references a table at a higher level falls
into this category.
hmm. it's still a tree, just the nodes contain same things (t1). There
are no cyclic paths to _same_ node in the expression.
like x+y+23*(x+1) --- x is used/pointed twice but the expression is
still a tree.
is there any case where some t1 (or even subexpr) is translated once in
one way, and then in another branch in another way? e.g. like in the
above x+y+... first x is to be replaced with its value, but second with
its name (say because its in round brackets - in some now-invented
syntax of mine ).
i think there is some mixup between what the expression is, as grammar,
and what it actualy means, and u're trying to solve/combine both in one
thing/visitor. maybe also what it should mean _after the processing.
While they should be all separate notions, somehow.
eh, i'm just throwing ideas / alternative view points to play with...
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: r3727 / AbstractClauseProcessor problem
On Nov 8, 2007, at 2:37 PM, [EMAIL PROTECTED] wrote:
heres the structure of: select(from_obj=[t1, t2, t1.join(t2)])
select +--- t1 -+
|--- t2 |
+--- join of t1/t2 ---+
t2 and t1 both have two parents, and there are two paths to each of
t1
and t2 from the head select. so its not a tree in the strict sense.
or another one:
s1 = t1.select().alias('s1')
s2 = t1.select().alias('s2')
s3 = s1.union(s2) (two paths to t1: s3-s1-t1, s3-s2-t1)
any kind of subquery which references a table at a higher level falls
into this category.
hmm. it's still a tree, just the nodes contain same things (t1). There
are no cyclic paths to _same_ node in the expression.
yes, there are. select-t1 and select-join of t1/t2 - t1 .
like x+y+23*(x+1) --- x is used/pointed twice but the expression is
still a tree.
only if you represent it with two distinct nodes for the two
occurences of x. SA's expressions don't do it that way (usually).
is there any case where some t1 (or even subexpr) is translated once
in
one way, and then in another branch in another way? e.g. like in the
above x+y+... first x is to be replaced with its value, but second
with
its name (say because its in round brackets - in some now-invented
syntax of mine ).
there arent on my end, and its true that this scenario wouldn't be
supported with the kinds of translations im doing.
i think there is some mixup between what the expression is, as
grammar,
and what it actualy means, and u're trying to solve/combine both in
one
thing/visitor. maybe also what it should mean _after the processing.
While they should be all separate notions, somehow.
eh, i'm just throwing ideas / alternative view points to play with...
yeah, im starting to head towards being able to support two distinct
nodes for x but they mean the same thing. the current trunk does a
litle bit of that now, since the clone operation doesn't make a
completely copied result. i think at the end of the day, the only
place it matters that we have x represented by three different
objects instead of one is when Select prints out the list of FROM
clauses (it also might matter when we render the columns clause of the
select).
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: r3727 / AbstractClauseProcessor problem
On Nov 7, 2007, at 1:02 PM, svilen wrote: On Wednesday 07 November 2007 19:33:22 Michael Bayer wrote: ohyoure *extending* abstractclauseprocessor ??? well yes, thats going to change things quite a bit. I think you should study ACP in its current form; what its doing now is faithfully calling convert_element() for *every* element in the expression, and also is not copying any elements before calling convert_element() - convert_element() always gets components from the original clause only. if convert_element() returns non-None, the resulting element is assembled into the output, and traversal *stops* for the remainder of that element. this is different behavior than it was before. the reason it stops for a replaced element is because its assumed that the replacement value is not part of the expression which is being copied, and therefore should not be copied or processed itself. if its that second part of the behavior thats breaking it for you, we can add an option to switch it off (comment out line 156, stop_on.add(newelem) to produce this). this did not change things, the column is still not traversed. maybe something else also has to be changed. the traversal will hit everything except the external collections of columns on objects since this is usually not needed. to change this, set the column_collections attribute to True on the ACP object (or in your class, or anywhere). but the old version had the same option turned on so not sure how that would change things. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: r3727 / AbstractClauseProcessor problem
On Wednesday 07 November 2007 19:33:22 Michael Bayer wrote: ohyoure *extending* abstractclauseprocessor ??? well yes, thats going to change things quite a bit. I think you should study ACP in its current form; what its doing now is faithfully calling convert_element() for *every* element in the expression, and also is not copying any elements before calling convert_element() - convert_element() always gets components from the original clause only. if convert_element() returns non-None, the resulting element is assembled into the output, and traversal *stops* for the remainder of that element. this is different behavior than it was before. the reason it stops for a replaced element is because its assumed that the replacement value is not part of the expression which is being copied, and therefore should not be copied or processed itself. if its that second part of the behavior thats breaking it for you, we can add an option to switch it off (comment out line 156, stop_on.add(newelem) to produce this). this did not change things, the column is still not traversed. maybe something else also has to be changed. i want a copy of original expression, where certain things are replaced by my things, and no need to go inside them - so this stop_on as u describe is okay... unless: what u mean remainder? that the returned element is not further traversed (thats ok), or the parent of that element is not traversed anymore (not ok)? this new version of ACP can locate things besides just plain Table, Alias and Column objects; it can locate things like Joins embedded in a clause which match the target selectable. On Nov 7, 2007, at 10:45 AM, svilen wrote: On Wednesday 07 November 2007 16:57:08 Michael Bayer wrote: On Nov 7, 2007, at 2:03 AM, [EMAIL PROTECTED] wrote: - something changed in the traversing (AbstractClauseProcessor - r3727) and it does not find proper things... ACP has been entirely rewritten. if you can provide simple tests in the form that theyre present in test/sql/generative.py and/or test/sql/ selectable.py that would be helpful. I have a feeling its not missing things, its just doing it slightly differently. http://dbcook.svn.sourceforge.net/viewvc/dbcook/trunk/dbcook/misc /aggregator/ (no it does not need dbcook) $ cd dbcook/misc/aggregator/tests $ PYTHONPATH=$PYTHONPATH:../.. python convertertest.py ... FAIL: count tags per movie File tests/convertertest.py, line 73, in test1_count_tags_per_movie['oid']) ... AssertionError: ['oid'] != ['tabl', 'oid'] FAIL: count tags per movie File tests/convertertest.py, line 73, in test1_count_tags_per_movie['oid']) ... AssertionError: ['oid'] != ['tabl', 'oid'] i did print the interesting elements in my Converter.convert_element(), and the result is that a) order is slightly different - which i dont care b) 1 item is not traversed in r3727 e.g. r3626: Column tags.tabl Column tags.oid Column movies.id Column tags.tabl Column tags.oid Column movies.id Column users.id Column userpics.uid Column userpics.state r3627: Column tags.tabl Column tags.oid Column movies.id Column tags.oid Column movies.id Column users.id Column userpics.uid Column userpics.state the 2nd tags.tabl is missing, hence the assertFails ciao svilen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: r3727 / AbstractClauseProcessor problem
ohyoure *extending* abstractclauseprocessor ??? well yes, thats going to change things quite a bit. I think you should study ACP in its current form; what its doing now is faithfully calling convert_element() for *every* element in the expression, and also is not copying any elements before calling convert_element() - convert_element() always gets components from the original clause only. if convert_element() returns non-None, the resulting element is assembled into the output, and traversal *stops* for the remainder of that element. this is different behavior than it was before. the reason it stops for a replaced element is because its assumed that the replacement value is not part of the expression which is being copied, and therefore should not be copied or processed itself. if its that second part of the behavior thats breaking it for you, we can add an option to switch it off (comment out line 156, stop_on.add(newelem) to produce this). this new version of ACP can locate things besides just plain Table, Alias and Column objects; it can locate things like Joins embedded in a clause which match the target selectable. On Nov 7, 2007, at 10:45 AM, svilen wrote: On Wednesday 07 November 2007 16:57:08 Michael Bayer wrote: On Nov 7, 2007, at 2:03 AM, [EMAIL PROTECTED] wrote: - something changed in the traversing (AbstractClauseProcessor - r3727) and it does not find proper things... ACP has been entirely rewritten. if you can provide simple tests in the form that theyre present in test/sql/generative.py and/or test/sql/ selectable.py that would be helpful. I have a feeling its not missing things, its just doing it slightly differently. http://dbcook.svn.sourceforge.net/viewvc/dbcook/trunk/dbcook/misc/aggregator/ (no it does not need dbcook) $ cd dbcook/misc/aggregator/tests $ PYTHONPATH=$PYTHONPATH:../.. python convertertest.py ... FAIL: count tags per movie File tests/convertertest.py, line 73, in test1_count_tags_per_movie['oid']) ... AssertionError: ['oid'] != ['tabl', 'oid'] FAIL: count tags per movie File tests/convertertest.py, line 73, in test1_count_tags_per_movie['oid']) ... AssertionError: ['oid'] != ['tabl', 'oid'] i did print the interesting elements in my Converter.convert_element(), and the result is that a) order is slightly different - which i dont care b) 1 item is not traversed in r3727 e.g. r3626: Column tags.tabl Column tags.oid Column movies.id Column tags.tabl Column tags.oid Column movies.id Column users.id Column userpics.uid Column userpics.state r3627: Column tags.tabl Column tags.oid Column movies.id Column tags.oid Column movies.id Column users.id Column userpics.uid Column userpics.state the 2nd tags.tabl is missing, hence the assertFails ciao svilen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: r3727 / AbstractClauseProcessor problem
On Nov 7, 2007, at 11:03 AM, svilen wrote: also, i put a class ClauseVisitor( sql_util.AbstractClauseProcessor): def convert_element( me, e): return None in the beginning of the tests.sql.generative, and after ignoreing this or that error, here is similar thing: == FAIL: test_correlated_select (__main__.ClauseTest) -- Traceback (most recent call last): File sql/generative.py, line 235, in test_correlated_select self.assert_compile(Vis().traverse(s, clone=True), SELECT * FROM table1 WHERE table1.col1 = table2.col1 AND table1.col2 = :table1_col2) File /home/az/src/ver/sqlalchemy-trunk/test/testlib/testing.py, line 262, in assert_compile self.assert_(cc == result, \n' + cc + '\n does not match \n' + result + ') AssertionError: 'SELECT * FROM table1 WHERE table1.col1 = table2.col1' does not match 'SELECT * FROM table1 WHERE table1.col1 = table2.col1 AND table1.col2 = :table1_col2' here whole subexpr is gone that would be...correct ? replacing convert_element() to return None means the given clause will be copied with no changes. the original clause doesnt have the extra subexpression. On Wednesday 07 November 2007 17:45:04 svilen wrote: On Wednesday 07 November 2007 16:57:08 Michael Bayer wrote: On Nov 7, 2007, at 2:03 AM, [EMAIL PROTECTED] wrote: - something changed in the traversing (AbstractClauseProcessor - r3727) and it does not find proper things... ACP has been entirely rewritten. if you can provide simple tests in the form that theyre present in test/sql/generative.py and/or test/sql/ selectable.py that would be helpful. I have a feeling its not missing things, its just doing it slightly differently. i did print the interesting elements in my Converter.convert_element(), and the result is that a) order is slightly different - which i dont care b) 1 item is not traversed in r3727 e.g. r3626: Column tags.tabl Column tags.oid Column movies.id Column tags.tabl Column tags.oid Column movies.id Column users.id Column userpics.uid Column userpics.state r3627: Column tags.tabl Column tags.oid Column movies.id Column tags.oid Column movies.id Column users.id Column userpics.uid Column userpics.state the 2nd tags.tabl is missing, hence the assertFails ciao svilen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: r3727 / AbstractClauseProcessor problem
On Wednesday 07 November 2007 16:57:08 Michael Bayer wrote: On Nov 7, 2007, at 2:03 AM, [EMAIL PROTECTED] wrote: - something changed in the traversing (AbstractClauseProcessor - r3727) and it does not find proper things... ACP has been entirely rewritten. if you can provide simple tests in the form that theyre present in test/sql/generative.py and/or test/sql/ selectable.py that would be helpful. I have a feeling its not missing things, its just doing it slightly differently. http://dbcook.svn.sourceforge.net/viewvc/dbcook/trunk/dbcook/misc/aggregator/ (no it does not need dbcook) $ cd dbcook/misc/aggregator/tests $ PYTHONPATH=$PYTHONPATH:../.. python convertertest.py ... FAIL: count tags per movie File tests/convertertest.py, line 73, in test1_count_tags_per_movie['oid']) ... AssertionError: ['oid'] != ['tabl', 'oid'] FAIL: count tags per movie File tests/convertertest.py, line 73, in test1_count_tags_per_movie['oid']) ... AssertionError: ['oid'] != ['tabl', 'oid'] i did print the interesting elements in my Converter.convert_element(), and the result is that a) order is slightly different - which i dont care b) 1 item is not traversed in r3727 e.g. r3626: Column tags.tabl Column tags.oid Column movies.id Column tags.tabl Column tags.oid Column movies.id Column users.id Column userpics.uid Column userpics.state r3627: Column tags.tabl Column tags.oid Column movies.id Column tags.oid Column movies.id Column users.id Column userpics.uid Column userpics.state the 2nd tags.tabl is missing, hence the assertFails ciao svilen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: r3727 / AbstractClauseProcessor problem
OK i found some more things that i think is probably screwing you up. will keep you posted. On Nov 7, 2007, at 10:45 AM, svilen wrote: On Wednesday 07 November 2007 16:57:08 Michael Bayer wrote: On Nov 7, 2007, at 2:03 AM, [EMAIL PROTECTED] wrote: - something changed in the traversing (AbstractClauseProcessor - r3727) and it does not find proper things... ACP has been entirely rewritten. if you can provide simple tests in the form that theyre present in test/sql/generative.py and/or test/sql/ selectable.py that would be helpful. I have a feeling its not missing things, its just doing it slightly differently. http://dbcook.svn.sourceforge.net/viewvc/dbcook/trunk/dbcook/misc/aggregator/ (no it does not need dbcook) $ cd dbcook/misc/aggregator/tests $ PYTHONPATH=$PYTHONPATH:../.. python convertertest.py ... FAIL: count tags per movie File tests/convertertest.py, line 73, in test1_count_tags_per_movie['oid']) ... AssertionError: ['oid'] != ['tabl', 'oid'] FAIL: count tags per movie File tests/convertertest.py, line 73, in test1_count_tags_per_movie['oid']) ... AssertionError: ['oid'] != ['tabl', 'oid'] i did print the interesting elements in my Converter.convert_element(), and the result is that a) order is slightly different - which i dont care b) 1 item is not traversed in r3727 e.g. r3626: Column tags.tabl Column tags.oid Column movies.id Column tags.tabl Column tags.oid Column movies.id Column users.id Column userpics.uid Column userpics.state r3627: Column tags.tabl Column tags.oid Column movies.id Column tags.oid Column movies.id Column users.id Column userpics.uid Column userpics.state the 2nd tags.tabl is missing, hence the assertFails ciao svilen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: r3727 / AbstractClauseProcessor problem
On Nov 7, 2007, at 1:20 PM, svilen wrote: ahha. so i am replacing one whole subexpr with somthing, and the original subexpr is not traversed inside. if i comment the stop_on.add(), it attempts to traverse the result subexpr, not the original one. i want the original to be traversed. Something like doing onExit instead of current onEntry. if its too hard, i can probably traverse it twice, once just marking , 2nd time replaceing things? i'll try if youre replacing a subtree, why would you want to then traverse that part of the subtree which was replaced ? can you see why how i have it working now is reasonably straightforward ? perhaps a second call to ACP.traverse() should be called for those elements which were replaced. also r3754 fixes the issue that aliased selects, while being traversed, were not being copied properly. i still need to tweak the rules for aliases (alised tables, for example, remain immutable). --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: r3727 / AbstractClauseProcessor problem
ahha. so i am replacing one whole subexpr with somthing, and the original subexpr is not traversed inside. if i comment the stop_on.add(), it attempts to traverse the result subexpr, not the original one. i want the original to be traversed. Something like doing onExit instead of current onEntry. if its too hard, i can probably traverse it twice, once just marking , 2nd time replaceing things? i'll try if youre replacing a subtree, why would you want to then traverse that part of the subtree which was replaced ? because as i said, the needed work is onExit of the nodes, i.e. all room/building is traversed, now what u want to make out of it? thus i can traverse only once. can you see why how i have it working now is reasonably straightforward ? perhaps a second call to ACP.traverse() should be called for those elements which were replaced. i did change my code to traverse all twice, once doing nothing, and once replaceing; it works. speed there is not an issue. Still, if u can make the onEntry/onExit/both switchable, that would be usable. Maybe not immediately, but i'm sure once day you'll need it. Right now, it is onEntry, and the actual code/choice is not done clearly, it is spread between traverse() and _convert_element() and similar. Has anything changed on plain ClauseVisitor? coz' i have (more important) other code elsewhere, which also breaks at same r3727 - it's today topic of digging. also r3754 fixes the issue that aliased selects, while being traversed, were not being copied properly. i still need to tweak the rules for aliases (alised tables, for example, remain immutable). i dont really understand why u need the ACP being so different to plain visitor; i mean cant they share some skeleton part of traversing, while putting all the choices (visit* vs convert; onentry/onexit; stop/dont) in their own parts. After all, visitor pattern is twofold, a) Guide + b) Visitor; the Guide doing traversing, the Visitor noting things; choice where to go might be in visitor and/or in guide. some times (one extreme) the visitor is just one dumb functor; other cases (other extreme end) the visitor is very sofisticated and even does guiding/traversing. Here it looks more like second case, u have most of both sides put in the Visitor, and only small part (specific visit_* / copy_internals) left to the actual nodes. And to me, the skeleton is still same between ACP and ClauseVisitor. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: r3727 / AbstractClauseProcessor problem
try out r3754. On Nov 7, 2007, at 6:56 PM, Michael Bayer wrote: OK i found some more things that i think is probably screwing you up. will keep you posted. On Nov 7, 2007, at 10:45 AM, svilen wrote: On Wednesday 07 November 2007 16:57:08 Michael Bayer wrote: On Nov 7, 2007, at 2:03 AM, [EMAIL PROTECTED] wrote: - something changed in the traversing (AbstractClauseProcessor - r3727) and it does not find proper things... ACP has been entirely rewritten. if you can provide simple tests in the form that theyre present in test/sql/generative.py and/or test/sql/ selectable.py that would be helpful. I have a feeling its not missing things, its just doing it slightly differently. http://dbcook.svn.sourceforge.net/viewvc/dbcook/trunk/dbcook/misc/aggregator/ (no it does not need dbcook) $ cd dbcook/misc/aggregator/tests $ PYTHONPATH=$PYTHONPATH:../.. python convertertest.py ... FAIL: count tags per movie File tests/convertertest.py, line 73, in test1_count_tags_per_movie['oid']) ... AssertionError: ['oid'] != ['tabl', 'oid'] FAIL: count tags per movie File tests/convertertest.py, line 73, in test1_count_tags_per_movie['oid']) ... AssertionError: ['oid'] != ['tabl', 'oid'] i did print the interesting elements in my Converter.convert_element(), and the result is that a) order is slightly different - which i dont care b) 1 item is not traversed in r3727 e.g. r3626: Column tags.tabl Column tags.oid Column movies.id Column tags.tabl Column tags.oid Column movies.id Column users.id Column userpics.uid Column userpics.state r3627: Column tags.tabl Column tags.oid Column movies.id Column tags.oid Column movies.id Column users.id Column userpics.uid Column userpics.state the 2nd tags.tabl is missing, hence the assertFails ciao svilen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
