New submission from Jan David Mol <[email protected]>:
The shlex module does not function as expected in the presence of
comments when newlines are not whitespace. An example (attached):
>>> from shlex import shlex
>>>
>>> lexer = shlex("a \n b")
>>> print ",".join(lexer)
a,b
>>>
>>> lexer = shlex("a # comment \n b")
>>> print ",".join(lexer)
a,b
>>>
>>> lexer = shlex("a \n b")
>>> lexer.whitespace=" "
>>> print ",".join(lexer)
a,
,b
>>>
>>> lexer = shlex("a # comment \n b")
>>> lexer.whitespace=" "
>>> print ",".join(lexer)
a,b
Now where did my newline go? The comment ate it! Even though the docs
seem to indicate the newline is not part of the comment itself:
shlex.commenters:
The string of characters that are recognized as comment beginners.
All characters from the comment beginner to end of line are ignored.
Includes just '#' by default.
----------
files: lexertest.py
messages: 93776
nosy: jjdmol2
severity: normal
status: open
title: shlex behaves unexpected if newlines are not whitespace
type: behavior
versions: Python 2.4, Python 2.5, Python 2.6, Python 2.7, Python 3.0, Python
3.1, Python 3.2
Added file: http://bugs.python.org/file15087/lexertest.py
_______________________________________
Python tracker <[email protected]>
<http://bugs.python.org/issue7089>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
