Daniel Stutzbach <daniel <at> stutzbachenterprises.com> writes: > > What kind of input are you using for the Text tests? I'm kind of surprised that the conversion to Unicode results in such a dramatic slowdown, if you're feeding it plain text (characters 0x00 through 0x7f).
It's some arbitrary text composed of 95% ASCII characters and 5% non-ASCII. On this specific example, utf8 decodes at around 250 MB/s, latin1 at almost 1 GB/s (on the same machine on which I ran the benchmarks). You can find the test here: http://svn.python.org/view/sandbox/trunk/iobench/ _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
