Dan Christian <robo...@users.sourceforge.net> added the comment: > Sure :) That’s why I suggest using dash for quick tests and rely on the work > of other people who did read the POSIX spec. I’ll have to check it too > before committing a patch.
The point of ref_shlex.py is that all shells act the same for common cases and shlex doesn't match any of them. The only real split it that csh based shells do some things differently that sh based shells ('2>' vs '&>'). >> shlex uses a series of character strings to drive it's parsing: whitespace, >> escape, quotes. >> Add another one: control = '();<>|&'. If it is unset (by default?), then >> the behavior is as >> before. > So we would need to add a Shlex subclass to the module to provide the new > behavior. I think I prefer a new argument, because we can just extend the > existing class and functions instead of adding subtly differing duplicates. You don't have to do a subclass (although that might have some advantages). You could do something like: def shlex(s, comments=False, posix=True, control=False): ... if control: if control is True: self.control = '();<>|&' else: self.control = control # let user specify their own control set >> If it is set, then shlex will output any character in control as a separate >> token. > Unless it is part of a quoted segment, right? (See #7611 for 'foo#bar' vs. > 'foo #bar'). Correct, quotes wouldn't change. >> There might be a shell specific script (or maybe it's left to the user) >> that decides that certain tokens can be recombined: > Seems to much complexity. I really prefer if we agree on one command parsing > behavior (POSIX, i.e. dash) and improve shlex to support that. People > wanting zsh rules can write their own subclass. shlex is a pretty simple lexer (as lexers go), and I wouldn't want it to get complicated. It's easier in the current code structure to split everything and then re-join as needed. This also allows you to select sh vs csh joining rules (e.g. '|&' means different things in sh vs csh). Every shell that I've seen follows one of those two flavors for syntax. >> '&&', '||', '|&', '>>', etc. > Wouldn’t it be more correct to consider them different tokens? I don’t have > a format training in CS or programming, so I’m not sure that my definition is > correct at all, but in my mind a token is a unit, and thus & and && are two > different things. Ideally, the final tokens have exact meanings. It easier to write handler code for '&&' than ('&', '&'). This is just a case of whether the parse joins them together or it's done in a second step. The current code doesn't do much look ahead, so it's hard for the lexer to produce things like '&&' directly. -Dan ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue1521950> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com