How are we going to handle 'operator.div' (not available in Py3) at Cython/Compiler/ExprNodes.py ? Just got a failure...
- On Mon, Jul 6, 2009 at 5:16 PM, Stefan Behnel<[email protected]> wrote: > Hi, > > I pushed a series of code fixes to cython-unstable to support running the > compiler in Python 3.1 and to run the test suite as usual, just by calling > runtests.py. Enabling that made me realise that Cython still isn't Py3 > clean, as there are still issues related to byte strings, mostly when > writing them out to the C code stream. A lovely example is that > > "some unicode string formatted with %s" % some_byte_string > > didn't work, as it results in this: > > "some unicode string formatted with b'byte string content'" > > being written. These problems appear in the weirdest and best hidden > places. For example, the "compile/c_directives" test fails because a > (defaulted?) buffer option pops up from the middle of nowhere that is (for > whatever reason) encoded as a byte string and gets passed on as a keyword > argument. In Py3, keyword argument names must be unicode, so you get a > beautiful compiler crash. > > Another problem is string comparisons. Byte strings and unicode strings > never compare equal in Py3, even when they look exactly the same. So we get > a couple of test failures due to unknown or misinterpreted names, and a > couple of failing doctests because the output contains a 'b' prefix at some > point. > > Regarding code generation: given that all generated code gets written as > Unicode strings in Py3, I decided to go the route of fake-decoding byte > strings from ISO-8859-1 to Unicode (which generates character code points > from 0-255 that are identical to the original byte sequence), and then > re-encoding the written C code as ISO-8859-1 on its way into the target > file to make them re-appear as the correct bytes. This is slightly hackish, > but everything else would largely complicate the way we write strings into > the code stream. To work around this, I gave the BytesLiteral class its own > __str__() method that returns a 'decoded' unicode string, so that % > formatting on unicode strings works now. > > I'll keep carving my way through the code to find the places to fix. > However, I would already like to ask the other core developers to install > the official Python 3.1 release and to make a test run on that platform > part of your normal test cycle. It's easy to get things wrong, but now that > it's easy to run the test suite, I hope it'll also become easier to catch > unexpected problems before non-portable code gets committed. > > It would be great if you could run the test suite on your side and skip > through the failures to check for tests and tested code that you are > familiar with. There are definitely tests where it would take me quite a > while to even finger point the place where things go wrong. > > Stefan > _______________________________________________ > Cython-dev mailing list > [email protected] > http://codespeak.net/mailman/listinfo/cython-dev > -- Lisandro Dalcín --------------- Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC) Instituto de Desarrollo Tecnológico para la Industria Química (INTEC) Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET) PTLC - Güemes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 _______________________________________________ Cython-dev mailing list [email protected] http://codespeak.net/mailman/listinfo/cython-dev
