Re: [Zope-dev] whitespace filtering corner case in doctest
On Tue, Aug 18, 2009 at 08:53:49AM +, Reinout van Rees wrote: On 2009-08-17, Marius Gedminas mar...@gedmin.as wrote: On Mon, Aug 17, 2009 at 02:46:46PM +, Reinout van Rees wrote: In some cases, importing readline can result in the escape code ^[[?1034h= to be send (8bit on). According to the gentoo bug report (liked from your blog post), this happens if your termcap/terminfo define smm/rmm codes (meta on/meta off). ... which I didn't do, at least not consiously. Just a pretty normal OSX installation. BTW, regarding your workaround: I'd suggest using TERM=3Ddummy instead of TERM=3Dlinux, as a safer choice. Not that it should matter much in practice. That'd be better, yes. I never do anything tweaking with TERM anyway, so I don't know the options/effects. jladage suggested to do the workaround in buildout, btw: add the following to the part that generates the bin/test script: environment-vars = TERM linux This escape code isn't visible, which leads to hard to find test errors, see http://reinout.vanrees.org/weblog/2009/07/16/invisible-test-diff.html Granted, it are basically corner cases. On the other hand, it does seem to cause problems now and then, according= to my googling. Would it be ok to add it to the whitespace normalization of doctests? Opinions? Well, it's not whitespace, really... My gut feeling is that this fixup doesn't belong in the core doctest code. Ok. If this happened in my project, I'd either 1) make sure I import readline at module level, before any tests are run This doesn't work, surprisingly. It *is* a corner case. It are tests where z3c.testsetup tests that it runs tests correctly. That's a lot of tests in one sentence, which means that the test output comes from separate python processes that run tests. So if I import readline in z3c.testsetup's tests, I still get the output from the test runners that are being tested. Sigh, difficult to explain, such a recusive testing thingy :-) Sounds perfectly clear to me. I once tried to make sure zope.testing's tests covered the test coverage code, and measuring the coverage of the tests testing coverage measurement turned out to be impossible. or 2) add a renormalizer that removes the escape sequence for the test that triggers this And you'd do this per-project and not in core zope.testing, right? Fine. Uli already send me some example code, so I'll do that (and put the code snippet on my weblog so that google can find it). Perhaps it would be nice if doctest's differ escaped ASCII control characters? (You could do that too with a renormalizer.) Can we safely assume that a specific set of control characters needs to be escaped? It sounds a bit dangerous to me. ASCII and defines which characters are printable and which ones are control characters. Maybe \x9b poses a bit of a problem, since it's an escape character for some terminals, but a real character on some legacy 8bit charsets. (It's not allowed in UTF-8 sequences.) Or perhaps you mean it may make it difficult to distinguish a test printing \x1b from a test printing ^ followed by [ (assuming that's the visualisation chosen for control characters). That is an issue. BTW I've no clue why I'm replying to a month-old post. I missed it earlier somehow. Marius Gedminas -- http://pov.lt/ -- Zope 3 consulting and development signature.asc Description: Digital signature ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] whitespace filtering corner case in doctest
On 2009-08-17, Marius Gedminas mar...@gedmin.as wrote: On Mon, Aug 17, 2009 at 02:46:46PM +, Reinout van Rees wrote: In some cases, importing readline can result in the escape code ^[[?1034h= to be send (8bit on). According to the gentoo bug report (liked from your blog post), this happens if your termcap/terminfo define smm/rmm codes (meta on/meta off). ... which I didn't do, at least not consiously. Just a pretty normal OSX installation. BTW, regarding your workaround: I'd suggest using TERM=3Ddummy instead of TERM=3Dlinux, as a safer choice. Not that it should matter much in practice. That'd be better, yes. I never do anything tweaking with TERM anyway, so I don't know the options/effects. jladage suggested to do the workaround in buildout, btw: add the following to the part that generates the bin/test script: environment-vars = TERM linux This escape code isn't visible, which leads to hard to find test errors, see http://reinout.vanrees.org/weblog/2009/07/16/invisible-test-diff.html Granted, it are basically corner cases. On the other hand, it does seem to cause problems now and then, according= to my googling. Would it be ok to add it to the whitespace normalization of doctests? Opinions? Well, it's not whitespace, really... My gut feeling is that this fixup doesn't belong in the core doctest code. Ok. If this happened in my project, I'd either 1) make sure I import readline at module level, before any tests are run This doesn't work, surprisingly. It *is* a corner case. It are tests where z3c.testsetup tests that it runs tests correctly. That's a lot of tests in one sentence, which means that the test output comes from separate python processes that run tests. So if I import readline in z3c.testsetup's tests, I still get the output from the test runners that are being tested. Sigh, difficult to explain, such a recusive testing thingy :-) or 2) add a renormalizer that removes the escape sequence for the test that triggers this And you'd do this per-project and not in core zope.testing, right? Fine. Uli already send me some example code, so I'll do that (and put the code snippet on my weblog so that google can find it). Perhaps it would be nice if doctest's differ escaped ASCII control characters? (You could do that too with a renormalizer.) Can we safely assume that a specific set of control characters needs to be escaped? It sounds a bit dangerous to me. Thanks! Reinout -- Reinout van Rees - rein...@vanrees.org - http://reinout.vanrees.org Software developer at http://www.thehealthagency.com Military engineers build missiles. Civil engineers build targets ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] whitespace filtering corner case in doctest
In some cases, importing readline can result in the escape code ^[[?1034h to be send (8bit on). This escape code isn't visible, which leads to hard to find test errors, see http://reinout.vanrees.org/weblog/2009/07/16/invisible-test-diff.html Granted, it are basically corner cases. On the other hand, it does seem to cause problems now and then, according to my googling. Would it be ok to add it to the whitespace normalization of doctests? Opinions? Reinout -- Reinout van Rees - rein...@vanrees.org - http://reinout.vanrees.org Software developer at http://www.thehealthagency.com Military engineers build missiles. Civil engineers build targets ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] whitespace filtering corner case in doctest
On Mon, Aug 17, 2009 at 02:46:46PM +, Reinout van Rees wrote: In some cases, importing readline can result in the escape code ^[[?1034h to be send (8bit on). According to the gentoo bug report (liked from your blog post), this happens if your termcap/terminfo define smm/rmm codes (meta on/meta off). BTW, regarding your workaround: I'd suggest using TERM=dummy instead of TERM=linux, as a safer choice. Not that it should matter much in practice. This escape code isn't visible, which leads to hard to find test errors, see http://reinout.vanrees.org/weblog/2009/07/16/invisible-test-diff.html Granted, it are basically corner cases. On the other hand, it does seem to cause problems now and then, according to my googling. Would it be ok to add it to the whitespace normalization of doctests? Opinions? Well, it's not whitespace, really... My gut feeling is that this fixup doesn't belong in the core doctest code. If this happened in my project, I'd either 1) make sure I import readline at module level, before any tests are run or 2) add a renormalizer that removes the escape sequence for the test that triggers this Perhaps it would be nice if doctest's differ escaped ASCII control characters? (You could do that too with a renormalizer.) Marius Gedminas -- http://pov.lt/ -- Zope 3 consulting and development signature.asc Description: Digital signature ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
