If str becomes unicode for PY 3000, and we then have bytes as out
coding-agnostic
byte data, then I think bytes should have the str translation method, with a
tweak
that I would hope could also be done to str now.
BTW, str.translate will presumably become unicode.translate, so
perhaps unicode.translate should grow a compatible deletechars parameter.
But that's not the tweak. The tweak is to eliminate unavoidable pre-conversion
to unicode
in str(something).translate(u'...', delchars) (and preemptively
bytes(something).translate(u'...', delchars))
E.g. suppose you now want to write:
s_str.translate(table, delch).encode('utf-8')
Note that s_str has no encoding information, and translate is conceptually just
a 1:1 substitution
minus characters in delch. But if we want to do one-chr:one-unichr substitution
by specifying a
256-long table of unicode characters, we cannot. It would be simple to allow
it, and that's the
tweak I would like. It would allow easy custom decodes.
At the moment, if you want to write the above, you have to introduce a phony
latin-1 decoding
and write it as (not typo-proof)
s_str.translate(table, delch).decode('latin-1').encode('utf-8') # use
str.translate
or
s_str.decode('latin-1').translate(mapping).encode('utf-8') # use
unicode.translate also for delch
to avoid exceptions if you have non-ascii in your s_str (even if delch would
have removed them!!)
It seems s_str.translate(table, delchars) wants to convert the s_str to unicode
if table is unicode, and then use unicode.translate (which bombs on delchars!)
instead of just effectively defining str.translate as
def translate(self, table, deletechars=None):
return ''.join((table or isinstance(table,unicode) and uidentity or
sidentity)[ord(x)] for x in self
if not deletechars or x not in deletechars)
# For convenience in just pruning with deletechars, s_str.translate('',
deletechars) deletes without translating,
# and s_str.translate(u'', deletechars) does the same and then maps to
same-ord unicode characters
# given
# sidentity = ''.join(chr(i) for i in xrange(256))
# and
# uidentity = u''.join(unichr(i) for i in xrrange(256)).
IMO, if you want unicode.translate, then it doesn't hurt to write
unicode(s_str).translate and use that.
Let str.translate just use the str ords, so simple custom decodes can be
written without
the annoyance of e.g.,
UnicodeDecodeError: 'ascii' codec can't decode byte 0xf6 in position 3:
ordinal not in range(128)
Can we change this for bytes? And why couldn't we change this for str.translate
now?
Or what am I missing? I certainly would like to miss the above message for
str.translate :-(
BTW This would also allow taking advantage of features of both translates if
desired, e.g. by
s_str.translate(unichartable256,
strdelchrs).translate(uniord_to_ustr_or_uniord_mapping).
(e.g., the latter permits single to multiple-character substitution)
I think at least a tweaked translate method for bytes would be good for py3k,
and I hope we can do it for str.translate now.
It it is just too handy a high speed conversion goodie to forgo IMO.
Regards,
Bengt Richter
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
