Hey all,
I've been playing around with the parser module, and based on the documentation I would expect all symbols in a parse tree to be part of the grammar.[1] For example, I find this line in the symbol module docs:
Refer to the file Grammar/Grammar in the Python distribution for the definitions of the names in the context of the language grammar.[2]
However, the program below gives me a human-readable parse tree (also below) that contains symbols that don't seem to be in the grammar, e.g., small_stmt, expr_stmt, factor. Is my expectation wrong?
Thanks.
chad
----------
[1] http://python.org/doc/ref/grammar.txt [2] http://python.org/doc/lib/module-symbol.html
############################ # A PROGRAM # ############################
#!/usr/bin/env python """print out a human-readable parse tree for a simple code block """
import parser, symbol, token from pprint import pprint
def experiment(block):
"""experimenting, like the man said
""" ast = parser.suite(block)
raw = ast.tolist()def interp(x):
"""given an int or a string, return a symbol, token, or other string
"""
if type(x) is type(0):
if x < 256:
return token.tok_name[x]
else:
return symbol.sym_name[x]
return x
def process(raw):
"""recursively convert a raw parse tree into something readable
"""
for node in raw:
if type(node) is type([]):
process(node)
else:
raw[raw.index(node)] = interp(node) process(raw)
pprint(raw)
experiment("""\
x = 3
for y in range(10):
y == x""")
############################ # ITS OUTPUT # ############################
['file_input',
['stmt',
['simple_stmt',
['small_stmt',
['expr_stmt',
['testlist',
['test',
['and_test',
['not_test',
['comparison',
['expr',
['xor_expr',
['and_expr',
['shift_expr',
['arith_expr',
['term',
['factor', ['power', ['atom', ['NAME', 'x']]]]]]]]]]]]]]],
['EQUAL', '='],
['testlist',
['test',
['and_test',
['not_test',
['comparison',
['expr',
['xor_expr',
['and_expr',
['shift_expr',
['arith_expr',
['term',
['factor',
['power', ['atom', ['NUMBER', '3']]]]]]]]]]]]]]]]],
['NEWLINE', '']]],
['stmt',
['compound_stmt',
['for_stmt',
['NAME', 'for'],
['exprlist',
['expr',
['xor_expr',
['and_expr',
['shift_expr',
['arith_expr',
['term', ['factor', ['power', ['atom', ['NAME', 'y']]]]]]]]]]],
['NAME', 'in'],
['testlist',
['test',
['and_test',
['not_test',
['comparison',
['expr',
['xor_expr',
['and_expr',
['shift_expr',
['arith_expr',
['term',
['factor',
['power',
['atom', ['NAME', 'range']],
['trailer',
['LPAR', '('],
['arglist',
['argument',
['test',
['and_test',
['not_test',
['comparison',
['expr',
['xor_expr',
['and_expr',
['shift_expr',
['arith_expr',
['term',
['factor',
['power',
['atom',
['NUMBER',
'10']]]]]]]]]]]]]]]],
['RPAR', ')']]]]]]]]]]]]]]],
['COLON', ':'],
['suite',
['NEWLINE', ''],
['INDENT', ''],
['stmt',
['simple_stmt',
['small_stmt',
['expr_stmt',
['testlist',
['test',
['and_test',
['not_test',
['comparison',
['expr',
['xor_expr',
['and_expr',
['shift_expr',
['arith_expr',
['term',
['factor',
['power', ['atom', ['NAME', 'y']]]]]]]]]],
['comp_op', ['EQEQUAL', '==']],
['expr',
['xor_expr',
['and_expr',
['shift_expr',
['arith_expr',
['term',
['factor',
['power', ['atom', ['NAME', 'x']]]]]]]]]]]]]]]]],
['NEWLINE', '']]],
['DEDENT', '']]]]],
['NEWLINE', ''],
['ENDMARKER', '']]-- http://mail.python.org/mailman/listinfo/python-list
