Re: Cleaning up Leo's iterators

[Offray Vladimir Luna Cárdenas]

 Hi,

El 06/09/09 14:01, Edward K. Ream escribió:



On Sun, Sep 6, 2009 at 1:12 PM, Ville M. Vainio > wrote:



> 1.  Position generators are the base of the scheme.  For
example, here
> is a partially tested rewrite of c.allNodes_iter, called, much more
> properly, allPositions().  These functions will eventually become

Could we have more pythonic & shorter names for these methods that
don't use camelCase? Something like c.walk() and c.walknodes()


I agree, it would be good to shorten and clarify names. The names of 
some of the new "unique" iterators are way too long.


Let me think about a scheme that will be relatively consistent across 
all iterators.  I would like to get rid of the _iter suffix, but it's 
not clear that can be done cleanly.


As for camelCase, I usually prefer xY to x_y, but not always, and Leo 
uses both without much plan.  That's not likely to change, absent a 
safe script for making the changes.


Referred pep says:

 Function Names

  Function names should be lowercase, with words separated by underscores
  as necessary to improve readability.

  mixedCase is allowed only in contexts where that's already the
  prevailing style (e.g. threading.py), to retain backwards compatibility.


So mixedCase could be used because is already a  writing style in 
practice (I liked more  that this_style also :) )


Cheers,

Offray



Edward

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google 
Groups "leo-editor" group.

To post to this group, send email to leo-editor@googlegroups.com
To unsubscribe from this group, send email to 
leo-editor+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/leo-editor?hl=en

-~--~~~~--~~--~--~---



--
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To post to this group, send email to leo-edi...@googlegroups.com.
To unsubscribe from this group, send email to 
leo-editor+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/leo-editor?hl=en.

Re: Cleaning up Leo's iterators

[Edward K. Ream]

On Mon, Sep 7, 2009 at 2:44 AM, Ville M. Vainio  wrote:

>
> On Mon, Sep 7, 2009 at 3:05 AM, Edward K. Ream 
> wrote:
>
> > The parallel with p.moveToThreadNext is exact.  Clearly,
> > p.moveToFirstChild, p.moveToNext and p.moveToParent are very fast, so
>
> They are still a lot slower than (for nodes).
>
> def walk(node):
>  for chi in par.children:
>yield chi
>if chi.children is not None:
>  for rec_chi in walk(chi): yield rec:chi
>
> Note that there is  no overhead for "self", or function calls (apart
> from recursion), or list manipulation (unlike with p.move It's a
> completely read-only generator.
>

True, but it yield vnodes :-) Usually we want positions.

Edward

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To post to this group, send email to leo-editor@googlegroups.com
To unsubscribe from this group, send email to 
leo-editor+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/leo-editor?hl=en
-~--~~~~--~~--~--~---

Re: Cleaning up Leo's iterators

[Ville M. Vainio]

On Mon, Sep 7, 2009 at 3:05 AM, Edward K. Ream  wrote:

> The parallel with p.moveToThreadNext is exact.  Clearly,
> p.moveToFirstChild, p.moveToNext and p.moveToParent are very fast, so

They are still a lot slower than (for nodes).

def walk(node):
  for chi in par.children:
yield chi
if chi.children is not None:
  for rec_chi in walk(chi): yield rec:chi

Note that there is  no overhead for "self", or function calls (apart
from recursion), or list manipulation (unlike with p.move It's a
completely read-only generator.

-- 
Ville M. Vainio
http://tinyurl.com/vainio

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To post to this group, send email to leo-editor@googlegroups.com
To unsubscribe from this group, send email to 
leo-editor+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/leo-editor?hl=en
-~--~~~~--~~--~--~---

Re: Cleaning up Leo's iterators

[Edward K. Ream]

On Sep 6, 12:06 pm, "Ville M. Vainio"  wrote:

> I think it would be a better idea to bypass positions for this
> generator, because we can implement *much* more efficient generators
> by using nodes alone.

Rather than calling p.threadNext, c.allNodes_iter could be defined
using:

def threadNext_iter(p):
if p:
p = p.copy() # The only copy
while p:
yield p
if p.hasChildren():
p.moveToFirstChild()
elif p.hasNext():
p.moveToNext()
else:
while p:
p.moveToParent()
if p and p.hasNext():
p.moveToNext()
break
raise StopIteration

The parallel with p.moveToThreadNext is exact.  Clearly,
p.moveToFirstChild, p.moveToNext and p.moveToParent are very fast, so
it seems that this is about as good as can be expected.  But this code
is so much cleaner than the present code.

Anyway, it is quite fun to play with various ideas.  It's all good, as
long as we can drop in new versions for old versions without changing
Leo's core.

Edward

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To post to this group, send email to leo-editor@googlegroups.com
To unsubscribe from this group, send email to 
leo-editor+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/leo-editor?hl=en
-~--~~~~--~~--~--~---

Re: Cleaning up Leo's iterators

[Edward K. Ream]

On Sun, Sep 6, 2009 at 1:12 PM, Ville M. Vainio  wrote:

>
> > 1.  Position generators are the base of the scheme.  For example, here
> > is a partially tested rewrite of c.allNodes_iter, called, much more
> > properly, allPositions().  These functions will eventually become
>
> Could we have more pythonic & shorter names for these methods that
> don't use camelCase? Something like c.walk() and c.walknodes()
>

I agree, it would be good to shorten and clarify names. The names of some of
the new "unique" iterators are way too long.

Let me think about a scheme that will be relatively consistent across all
iterators.  I would like to get rid of the _iter suffix, but it's not clear
that can be done cleanly.

As for camelCase, I usually prefer xY to x_y, but not always, and Leo uses
both without much plan.  That's not likely to change, absent a safe script
for making the changes.

Edward

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To post to this group, send email to leo-editor@googlegroups.com
To unsubscribe from this group, send email to 
leo-editor+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/leo-editor?hl=en
-~--~~~~--~~--~--~---

Re: Cleaning up Leo's iterators

[Edward K. Ream]

On Sun, Sep 6, 2009 at 1:36 PM, Ville M. Vainio  wrote:

>
> Don't bother too much about this yet, just implement the
> straightforward version in terms of positions as you suggested and
> I'll provide & benchmark the optimized version later. My gut feeling
> tells me significant speedup can be achieved, but of course I might be
> wrong.
>
> This is a harmless optimization in that no changes outside of this
> function will be needed.
>

Ok then.  That's fine with me.  BTW, Bernhard suggested what looked like
weird generators to me.  Perhaps you are channeling his ideas.

Edward

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To post to this group, send email to leo-editor@googlegroups.com
To unsubscribe from this group, send email to 
leo-editor+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/leo-editor?hl=en
-~--~~~~--~~--~--~---

Re: Cleaning up Leo's iterators

[Ville M. Vainio]

On Sun, Sep 6, 2009 at 9:25 PM, Edward K. Ream  wrote:

>
> Actually, I can be even more emphatic about this statement.
>
> It is not immediately obvious from the code, but it is a fact nonetheless,
> that Leo's present iterators are already highly optimized.  A loop such as:
>
>     for p in c.allNodes_iter():
>     << whatever >>
>
> generates *exactly one* position p, and this position is "moved" by the
> iterator using p.moveToX methods. This is a crucial space optimization.  The

Yeah, a space optimization. I'm thinking of time optimization here.

> comment in the p.moveToX node says:
>
> QQQ
> These routines change self to a new position "in place".
> That is, these methods must _never_ call p.copy().
> QQQ
>
> Furthermore,  p.stack turns out to be essential in computing the next
> position.  To contemplate "optimized" vnode iterators is misguided: they
> would have to simulate p.stack in order to work properly.

I'm thinking of using a recursive generator for this.  OTOH, emulating
the position stack is easy.

Don't bother too much about this yet, just implement the
straightforward version in terms of positions as you suggested and
I'll provide & benchmark the optimized version later. My gut feeling
tells me significant speedup can be achieved, but of course I might be
wrong.

This is a harmless optimization in that no changes outside of this
function will be needed.

-- 
Ville M. Vainio
http://tinyurl.com/vainio

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To post to this group, send email to leo-editor@googlegroups.com
To unsubscribe from this group, send email to 
leo-editor+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/leo-editor?hl=en
-~--~~~~--~~--~--~---

Re: Cleaning up Leo's iterators

[Edward K. Ream]

On Sun, Sep 6, 2009 at 1:16 PM, Edward K. Ream  wrote:

>
> This can't be correct, for several reasons.
>

Actually, I can be even more emphatic about this statement.

It is not immediately obvious from the code, but it is a fact nonetheless,
that Leo's present iterators are already highly optimized.  A loop such as:

for p in c.allNodes_iter():
<< whatever >>

generates *exactly one* position p, and this position is "moved" by the
iterator using p.moveToX methods. This is a crucial space optimization.  The
comment in the p.moveToX node says:

QQQ
These routines change self to a new position "in place".
That is, these methods must _never_ call p.copy().
QQQ

Furthermore,  p.stack turns out to be essential in computing the next
position.  To contemplate "optimized" vnode iterators is misguided: they
would have to simulate p.stack in order to work properly.

Edward

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To post to this group, send email to leo-editor@googlegroups.com
To unsubscribe from this group, send email to 
leo-editor+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/leo-editor?hl=en
-~--~~~~--~~--~--~---

Re: Cleaning up Leo's iterators

[Edward K. Ream]

On Sun, Sep 6, 2009 at 12:06 PM, Ville M. Vainio  wrote:

>
> On Sun, Sep 6, 2009 at 7:53 PM, Edward K. Ream 
> wrote:
>
> > 2. Generators returning (unified)nodes simply use the corresponding
> > position generators.  Thus, all such generators must be members of the
> > commands or position class, *not* the vnode class.
> >
> > def allNodes(c):
> >for p in allPositions(c):
> >yield p.v
> >raise StopIteration
>
> I think it would be a better idea to bypass positions for this
> generator, because we can implement *much* more efficient generators
> by using nodes alone. Also, the node generator could return the whole
> parent stack (because it maintains it anyway).
>

This can't be correct, for several reasons.

Most importantly, iterators are naturally related to positions, not nodes.
It would thus be bad design to have iterators be part of a node class.
Except for some unusual special cases in the read logic (when positions
don't yet exist), Leo (properly) treats position as the essential data to
have, and nodes as derived from positions.  Changing this would greatly (and
wrongly) change Leo's core.  I'm not going to do it.  Instead, the
contemplated revision merely replaces clumsy code by elegant code.

Second, (v)nodes are never enough, by themselves, to generate most kinds of
"next" positions.  Two more data are needed for the next position to be well
defined: the specific parent of the  node and the node's child index. The
p.moveToX methods used by the present (and future) iterators pass such data
to the low-level vnode methods.

Finally, the data structures needed by your suggested node iterators are
available to positions via p.v.  Thus, if need be, we could implement the
same optimizations you contemplate for vnode-based iterators in the position
class.

I have no appetite for further optimizations.  The code is good, and
previous optimizations have been complex.  However, I won't rule them out
entirely, if it turns out that optimizations are much easier to do in the
one-node world.

Edward

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To post to this group, send email to leo-editor@googlegroups.com
To unsubscribe from this group, send email to 
leo-editor+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/leo-editor?hl=en
-~--~~~~--~~--~--~---

Re: Cleaning up Leo's iterators

[Ville M. Vainio]

On Sun, Sep 6, 2009 at 7:53 PM, Edward K. Ream  wrote:

> 1.  Position generators are the base of the scheme.  For example, here
> is a partially tested rewrite of c.allNodes_iter, called, much more
> properly, allPositions().  These functions will eventually become

Could we have more pythonic & shorter names for these methods that
don't use camelCase? Something like c.walk() and c.walknodes()

In general following pep-8 would be nice.

http://www.python.org/dev/peps/pep-0008/

-- 
Ville M. Vainio
http://tinyurl.com/vainio

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To post to this group, send email to leo-editor@googlegroups.com
To unsubscribe from this group, send email to 
leo-editor+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/leo-editor?hl=en
-~--~~~~--~~--~--~---

Re: Cleaning up Leo's iterators

[Ville M. Vainio]

On Sun, Sep 6, 2009 at 7:53 PM, Edward K. Ream  wrote:

> 2. Generators returning (unified)nodes simply use the corresponding
> position generators.  Thus, all such generators must be members of the
> commands or position class, *not* the vnode class.
>
> def allNodes(c):
>    for p in allPositions(c):
>        yield p.v
>    raise StopIteration

I think it would be a better idea to bypass positions for this
generator, because we can implement *much* more efficient generators
by using nodes alone. Also, the node generator could return the whole
parent stack (because it maintains it anyway).

-- 
Ville M. Vainio
http://tinyurl.com/vainio

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To post to this group, send email to leo-editor@googlegroups.com
To unsubscribe from this group, send email to 
leo-editor+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/leo-editor?hl=en
-~--~~~~--~~--~--~---