Aivar Annamaa added the comment: (Sorry, I didn't mean to challenge the authority of a core developer. I simply didn't notice that adding a comment reopens the issue. I hope this time I selected correct parameters and this doesn't happen again)
I'm trying to rephrase my concern. Initially I thought there was a mistake in the tokenizer or in the token module. After you pointed out the documentation about token.OP and exact_type, I'm worried about a smaller detail. The documentation only talks about *operators* and *delimiters* having type attribute set to token.OP. According to my understanding (and also the listings at https://docs.python.org/3/reference/lexical_analysis.html#operators), ellipsis is neither operator nor delimiter. Am I right about this? (I understand that the source representation of both ELLIPSIS and DOT tokens contains period *character(s)*, but I don't see why is this relevant when we discuss the properties of *tokens*) Anyway, if ellipsis is neither operator nor delimiter, (and if there is a reason why it is treated similarly with operators and delimiters) then I recommend to update the documentation by replacing > all Operators and Delimiters tokens > are returned using the generic token.OP token type with > all Operators, Delimiters and Ellipsis tokens > are returned using the generic token.OP token type (I understand, that this is not a serious issue. If you prefer not to discuss it further then I'm happy to leave it as it is.) ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue31394> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
