On Wed, Dec 29, 2021 at 11:13 AM vitalije <[email protected]> wrote:
>
>> Thanks for the link. I wasn't clear enough about the problem I am trying
>> to solve. I want to duplicate the TokenOrderGenerator class in leoAst.py
>> without recursion or generators.
>>
>> I don't think it is possible what you are trying to achieve. To totally
> avoid any kind of recursion, you must know some facts about the AST in
> advance, for example how deep it is.
>
Thanks for this comment. I am quite sure the approach I am using will
work. However, the working code will be ugly, and at least as complex as
the present code in leoAst.py. I agree with Thomas that the final result
would be a simulation of generators. For this reason I am considering
abandoning the project.
However, I should at least explain why an iterative simulation is
feasible. Here is the top-level iterative loop:
def traverse(self, contents):
"""Completely traverse the ast tree for the given contents."""
<< define action_dict >>
self.node = root = ast.parse(contents)
action_list = self.action_dict.get(root.__class__.__name__, []).copy()
self.stack = [StackEntry(None, action_list)] # (parent_node,
action_list)
while self.stack:
if action_list:
action = action_list.pop(0)
action.func(action.args)
else:
entry = self.stack.pop()
self.node = entry.parent
action_list = entry.action_list
Action functions execute in the context of a *specific* ast node. The most
general action function is visit. Here is a slightly simplified version of
the present code:
def visit(self, field):
child = getattr(self.node, field, None)
if not child:
return
if isinstance(child, (list, tuple)):
# Do *not* change self.node.
for z in child:
self.stack.append(StackEntry(self.node,
[Action(self.visit_list, z)]))
return
assert isinstance(child, ast.AST), repr(child)
action_list = self.action_dict.get(child.__class__.__name__, None)
if not action_list:
if action_list is None:
print(f"visit: Missing: {child.__class__.__name__}")
return
self.stack.append(StackEntry(self.node, action_list.copy()))
# *Do* change self.node.
self.node = child
Let us consider the difficulties of handling ast.If nodes. The TOG visitor,
which uses generators, is:
def do_If(self, node):
# Use the next significant token to distinguish between 'if' and 'elif'.
token = self.find_next_significant_token()
yield from self.gen_name(token.value)
yield from self.gen(node.test)
yield from self.gen_op(':')
# Body...
self.level += 1
yield from self.gen(node.body)
self.level -= 1
#
# Else and elif clauses...
if node.orelse:
self.level += 1
token = self.find_next_significant_token()
if token.value == 'else':
yield from self.gen_name('else')
yield from self.gen_op(':')
yield from self.gen(node.orelse)
else:
yield from self.gen(node.orelse)
self.level -= 1
As a prototype, we could consider the following *general action list* for
any ast.If node.
'If': [
Action(visit, 'value',),
Action(visit, 'test'),
Action(visit, 'body'),
Action(visit, 'orelse'),
]
These actions will suffice to visit the subtree of *any* ast.If node
(regardless of complexity) because these actions will be "bound" to an
actual node *during the traversal of the tree*.
However, such *general* actions will not suffice to simulate all the
complications in TOG.do_if. Instead, we need, for almost every kind of ast
node, a list of *node-specific actions*. Something like this;
'If': [
Action(visit_if_value)
Action(visit_if_test)
Action(visit_if_body)
Action(visit_if_else),
]
These action methods can generate the required tests and calls because they
are aware that they are executing in the context of an ast.If node, just as
the TOG.do_If visitor is.
*Summary*
Every action method executes in the context of a single, *specific, *ast
node in the parse tree. The algorithm creates new action methods as it
traverses the tree. Each action method handles exactly one child field of
the parent ast node. There is no doubt in my mind that this scheme is
sound.
General action methods suffice to traverse the tree, but we must have more
node-specific actions to simulate each visitor of the TOG class. In effect,
the node-specific action methods split each TOG visitor into pieces. The
effect is to simulate generators (coroutines) "by hand."
Edward
--
You received this message because you are subscribed to the Google Groups
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/leo-editor/CAMF8tS2v6%3DWFKH%2BUU2EHyXe6AdCSGBN0m%2BU11uLX3ZTvUBxUKg%40mail.gmail.com.
