bootkey wrote:
>
>Thanks for the reply. I wonder why the tokenizer classifies all
>operators simply as OP, instead of the various operators listed in the
>tok_name dictionary.
I imagine it's just an exercise left to the reader. It's not that hard of
an extension.
--
Tim Roberts, [email protected]
On Jun 28, 1:46 am, Tim Roberts wrote:
> Jim wrote:
> >I'm trying to understand the output of the tokenize.generate_tokens()
> >generator. The token types returned seem to be more general than I'd
> >expect. For example, when fed the following line of code:
>
> >def func_a():
> >...
> >It seems
Jim wrote:
>I'm trying to understand the output of the tokenize.generate_tokens()
>generator. The token types returned seem to be more general than I'd
>expect. For example, when fed the following line of code:
>
>def func_a():
>...
>It seems to me that the token '(' should be identified as 'LP
> After installation my program that uses tokenize module,when I run
> myprogram.exe (vgsveki.exe):
>
> AttributeError: 'module' object has no attribute 'untokenize'
Perhaps you use the 2.4 version of tokenize.py here, which did
not have untokenize.
Regards,
Martin
--
http://mail.python.org/mai
