I help maintain apps on a number of machines and recently a bug surfaced which
affected some of them using the preppy template module; the bug ended up being
the way tokenize worked as in the following
#############################
from StringIO import StringIO
import tokenize, token
L = []
s='i'
tokenize.tokenize(StringIO(s.strip()).readline,lambda *a: L.append(a))
print('L=%s' % repr(L))
#############################
Older working machines
Python 2.7.15rc1 (default, Nov 12 2018, 14:31:15)\n[GCC 7.3.0] on linux2
L=[(1, 'i', (1, 0), (1, 1), 'i'), (0, '', (2, 0), (2, 0), '')]
upgraded machines showing a bug
Python 2.7.15+ (default, Nov 27 2018, 23:36:35)\n[GCC 7.3.0] on linux2
L=[(1, 'i', (1, 0), (1, 1), 'i'), (4, '', (1, 1), (1, 2), ''), (0, '', (2, 0),
(2, 0), '')]
The extra token represents a linefeed; however StringIO('i').readline() doesn't
contain a linefeed.
so the problem is that parsing a simple string 'i' results in two different
parses. On my Arch machines running 2.7.16 I see the original parse.
The fix was simple and involved using more robust code.
My question is that since the package builders feel able to modify and affect
behaviour in such a simple case will they do it randomly elsewhere perhaps
leading to other less obvious changes.
Should I always be using self build python versions?
It seems that ubuntu feels able to provide packages which are rc versions or
have a + indicating they're modified. They'll probably argue that this improves
things and I shouldn't be using such low level code ....... :(
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
