Re: [Python-Dev] Can Python implementations reject semantically invalid expressions?
> We know that many semantic errors in Python lead to runtime errors, e.g. > 1 + "1". If an implementation rejected them at compile time, would it > still be Python? E.g. if the keyhole optimizer raised SyntaxError (or > some other exception) on seeing this: > > def f(): > return 1 + "1" > > instead of compiling something which can't fail to raise an exception, > would that still be a legal Python implementation? I'd say "no". Any syntactically correct module should start executing, and type errors are only a runtime concept. If you were to reject code at startup more restrictively, you might end up rejecting the standard library, as it contains syntax errors in code that isn't being imported normally (test/badsyntax*). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can Python implementations reject semantically invalid expressions?
On Jul 2, 2010, at 12:28 AM, Steven D'Aprano wrote: > This question was inspired by something asked on #python today. Consider > it a hypothetical, not a serious proposal. > > We know that many semantic errors in Python lead to runtime errors, e.g. > 1 + "1". If an implementation rejected them at compile time, would it > still be Python? E.g. if the keyhole optimizer raised SyntaxError (or > some other exception) on seeing this: > > def f(): >return 1 + "1" > > instead of compiling something which can't fail to raise an exception, > would that still be a legal Python implementation? I'd say "no". Python has defined semantics in this situation: a TypeError is raised. To me, this seems akin to a keyhole optimizer arbitrarily deciding that raise TypeError() should cause the compiler to abort. If this type of expression were common, it would be within the rights of, for example, a Python JIT to generate a fast path through 'f' that wouldn't bother to actually invoke its 'int' type's '__add__' method, since there is no possible way for a Python program to tell the difference, since int.__add__ is immutable. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can Python implementations reject semantically invalid expressions?
Glyph Lefkowitz, 02.07.2010 06:43: On Jul 2, 2010, at 12:28 AM, Steven D'Aprano wrote: This question was inspired by something asked on #python today. Consider it a hypothetical, not a serious proposal. We know that many semantic errors in Python lead to runtime errors, e.g. 1 + "1". If an implementation rejected them at compile time, would it still be Python? E.g. if the keyhole optimizer raised SyntaxError (or some other exception) on seeing this: def f(): return 1 + "1" instead of compiling something which can't fail to raise an exception, would that still be a legal Python implementation? I'd say "no". Python has defined semantics in this situation: a TypeError is raised. So, would it still be Python if it folded 1 + "1" into raise TypeError() at compile time? Stefan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can Python implementations reject semantically invalid expressions?
On Fri, Jul 2, 2010 at 12:31 AM, Stefan Behnel wrote: > Glyph Lefkowitz, 02.07.2010 06:43: >> >> On Jul 2, 2010, at 12:28 AM, Steven D'Aprano wrote: >> >>> This question was inspired by something asked on #python today. Consider >>> it a hypothetical, not a serious proposal. >>> >>> We know that many semantic errors in Python lead to runtime errors, e.g. >>> 1 + "1". If an implementation rejected them at compile time, would it >>> still be Python? E.g. if the keyhole optimizer raised SyntaxError (or >>> some other exception) on seeing this: >>> >>> def f(): >>> return 1 + "1" >>> >>> instead of compiling something which can't fail to raise an exception, >>> would that still be a legal Python implementation? >> >> I'd say "no". Python has defined semantics in this situation: a TypeError >> is raised. > > So, would it still be Python if it folded > > 1 + "1" > > into > > raise TypeError() > > at compile time? > > Stefan > This question has an easy answer - can you possibly tell the difference? Cheers, fijal ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can Python implementations reject semantically invalid expressions?
> This question has an easy answer - can you possibly tell the difference? > Ok, I'm obviously being silly here, but sure you can: >>> dis.dis("raise TypeError()") 0 <114> 26977 3 <115>8293 6 IMPORT_STAR 7 SETUP_EXCEPT25968 (to 25978) 10 <69> 11 <114> 28530 14 <114> 10536 >>> dis.dis("1 + '1'") 0 <49> 1 SLICE+2 2 STORE_SLICE+3 3 SLICE+2 4 <39> 5 <49> 6 <39> That said, I agree with the point you're making -- they have the same semantics, so you should be fine substituting one for the other. Honestly, though, I'd come down on the side of letting the compiler raise an error -- while I understand that it means you have *different* behavior, I think it's *preferable* behavior. -cc ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can Python implementations reject semantically invalid expressions?
Am 02.07.2010 08:55, schrieb Craig Citro: >> This question has an easy answer - can you possibly tell the difference? >> > > Ok, I'm obviously being silly here, but sure you can: The dis module is deliberately (*) not part of the Python language and standard library; it's an implementation detail (as is the func_code attribute, and the code object). So the question really is: can you tell the difference, using only mechanisms not explicitly documented as implementation-specific? Regards, Martin (*) Unfortunately, the documentation fails to mention that, probably because it's too obvious. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can Python implementations reject semantically invalid expressions?
On Fri, Jul 2, 2010 at 7:55 AM, Craig Citro wrote: > > Ok, I'm obviously being silly here, but sure you can: > dis.dis("raise TypeError()") > 0 <114> 26977 > 3 <115> 8293 > 6 IMPORT_STAR > 7 SETUP_EXCEPT 25968 (to 25978) > 10 <69> > 11 <114> 28530 > 14 <114> 10536 dis.dis("1 + '1'") > 0 <49> > 1 SLICE+2 > 2 STORE_SLICE+3 > 3 SLICE+2 > 4 <39> > 5 <49> > 6 <39> Whoa. That's very peculiar looking bytecode. Is dis.dis behaving as it should here? BTW, I think you want 'raise TypeError', not 'raise TypeError()'. Mark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can Python implementations reject semantically invalid expressions?
On Fri, Jul 2, 2010 at 8:22 AM, Mark Dickinson wrote: > On Fri, Jul 2, 2010 at 7:55 AM, Craig Citro wrote: >> > dis.dis("raise TypeError()") >> 0 <114> 26977 >> 3 <115> 8293 >> 6 IMPORT_STAR >> 7 SETUP_EXCEPT 25968 (to 25978) >> 10 <69> >> 11 <114> 28530 >> 14 <114> 10536 > dis.dis("1 + '1'") >> 0 <49> >> 1 SLICE+2 >> 2 STORE_SLICE+3 >> 3 SLICE+2 >> 4 <39> >> 5 <49> >> 6 <39> > > Whoa. That's very peculiar looking bytecode. Is dis.dis behaving as > it should here? Ah. I see. It looks like the string "raise TypeError()" is being interpreted *directly* as Python bytecode, with no intermediate compilation. I don't think this is what you intended. Try: >>> dis.dis(compile("raise TypeError", "", "exec")) 1 0 LOAD_NAME0 (TypeError) 3 RAISE_VARARGS1 6 LOAD_CONST 0 (None) 9 RETURN_VALUE >>> dis.dis(compile("1 + '1'", "", "exec")) 1 0 LOAD_CONST 0 (1) 3 LOAD_CONST 1 ('1') 6 BINARY_ADD 7 POP_TOP 8 LOAD_CONST 2 (None) 11 RETURN_VALUE Mark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can Python implementations reject semantically invalid expressions?
On Fri, Jul 2, 2010 at 1:20 AM, "Martin v. Löwis" wrote: > Am 02.07.2010 08:55, schrieb Craig Citro: >>> This question has an easy answer - can you possibly tell the difference? >>> >> >> Ok, I'm obviously being silly here, but sure you can: > > The dis module is deliberately (*) not part of the Python language and > standard library; it's an implementation detail (as is the func_code > attribute, and the code object). > > So the question really is: can you tell the difference, using only > mechanisms not explicitly documented as implementation-specific? > > Regards, > Martin > > (*) Unfortunately, the documentation fails to mention that, probably > because it's too obvious. > Even more, Jython and IronPython don't have Python bytecode at all and they're considered python implementations. Cheers, fijal ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can Python implementations reject semantically invalid expressions?
On Fri, 2 Jul 2010 04:55:10 pm Craig Citro wrote: > > This question has an easy answer - can you possibly tell the > > difference? > > Ok, I'm obviously being silly here, but sure you can: > >>> dis.dis("raise TypeError()") > > 0 <114> 26977 > 3 <115>8293 > 6 IMPORT_STAR > 7 SETUP_EXCEPT25968 (to 25978) > 10 <69> > 11 <114> 28530 > 14 <114> 10536 Craig, what are you using to get that? When I try it in Python 3.1, I get: TypeError: don't know how to disassemble str objects How do you get that result? -- Steven D'Aprano ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can Python implementations reject semantically invalid expressions?
On Fri, Jul 2, 2010 at 12:25 PM, Steven D'Aprano wrote: > Craig, what are you using to get that? When I try it in Python 3.1, I > get: > > TypeError: don't know how to disassemble str objects > > How do you get that result? As I just discovered (see above), dis.dis is happy to interpret byte strings (i.e., strings in 2.x, bytes in 3.x) as direct representations of Python bytecode. There's also an open feature request[1] to allow text strings as input in py3k, doing an automatic compile before passing the result to dis.dis. Mark [1] http://bugs.python.org/issue6507 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can Python implementations reject semantically invalid expressions?
On Fri, Jul 2, 2010 at 4:55 PM, Craig Citro wrote: > Honestly, though, I'd come down on the side of letting the compiler > raise an error -- while I understand that it means you have > *different* behavior, I think it's *preferable* behavior. But you would be taking a module that will compile and making it uncompilable. The Python code: def f(): return 1 + "1" has fully defined runtime semantics: when f() is called, it will raise TypeError. A module containing this code is still perfectly valid Python (e.g. the Python test suite does that kind of thing a lot in tests of the core interpreter semantics). A Python implementation issuing a SyntaxWarning over this would be fine, but triggering a SyntaxError would not be valid. However, I'd be inclined to leave this kind of check to tools like pychecker and pylint. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can Python implementations reject semantically invalid expressions?
> Whoa. That's very peculiar looking bytecode. Is dis.dis behaving as > it should here? > BTW, I think you want 'raise TypeError', not 'raise TypeError()'. > Yep, that's embarrassing. I was being lazy: I was expecting different bytecodes, and I got it ... so I apparently didn't bother to actually read the bytecodes? It's interesting -- if I'd just had TypeError instead of TypeError(), I would have found this out, because "raise TypeError" is not the bytes representation of a valid sequence of bytecodes. ;) Anyway, here's what I was going for: >>> def foo(): ... return 1+'1' ... >>> def bar(): ... raise TypeError ... >>> dis.dis(foo) 2 0 LOAD_CONST 1 (1) 3 LOAD_CONST 2 ('1') 6 BINARY_ADD 7 RETURN_VALUE >>> dis.dis(bar) 2 0 LOAD_GLOBAL 0 (TypeError) 3 RAISE_VARARGS1 6 LOAD_CONST 0 (None) 9 RETURN_VALUE That said, I totally concede Martin's point -- this is an implementation-specific thing. It happens that all the major Python implementations compile to some VM, and I'd bet that these two compile to different bytecodes on any of them, but that doesn't preclude another implementation from making a different choice there. -cc ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can Python implementations reject semantically invalid expressions?
On Fri, Jul 2, 2010 at 3:44 PM, Craig Citro wrote: >> Whoa. That's very peculiar looking bytecode. Is dis.dis behaving as >> it should here? >> BTW, I think you want 'raise TypeError', not 'raise TypeError()'. >> > > Yep, that's embarrassing. I was being lazy: I was expecting different > bytecodes, and I got it ... so I apparently didn't bother to actually > read the bytecodes? It's interesting -- if I'd just had TypeError > instead of TypeError(), I would have found this out, because "raise > TypeError" is not the bytes representation of a valid sequence of > bytecodes. ;) Ah; interesting. Well clearly that's exactly what I meant when I said 'raise TypeError' was better than 'raise TypeError()'. ;-) (Actually, I confess to being similarly embarrassed: I have no idea *what* I was thinking there, since 'raise TypeError()' is just as valid, if a little less idiomatic.) > That said, I totally concede Martin's point -- this is an > implementation-specific thing. It happens that all the major Python > implementations compile to some VM, and I'd bet that these two compile > to different bytecodes on any of them, but that doesn't preclude > another implementation from making a different choice there. Agreed. Mark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can Python implementations reject semantically invalid expressions?
> But you would be taking a module that will compile and making it uncompilable. > You're absolutely right, and since I definitely *don't* think that the program "raise TypeError" should cause a CompileError, you could say it's safer to have a simple rule like "vaild syntax => will compile" -- it's probably a slippery slope once you start deciding which bits of semantics raise CompileErrors and which don't. However, in this particular case, here's a question: *why* would someone write "return 1 + '1'"? Did they do it *knowing* what would happen, or because they just didn't realize it was just an error? * If they knew what it was going to do, then I'd say shame on them -- they should have just raised a TypeError instead, and anyone who comes along to read or maintain that code would thank them for the change. My impression is that we generally try to discourage people from writing "tricky" code with Python, and this would do exactly that. * If, on the other hand, it's an accident, then I think it's a service to the user to let them know as soon as possible. Here I'm thinking both of someone new to Python, or even a seasoned pro who makes a few "quick fixes" before sending some code to someone, and doesn't happen to test that code path before handing it off. Either way, I personally prefer the CompileError -- it helps me catch a stupid mistake if I've made one, and it prevents other people from writing code I find less clear. My real motive, though, is that I'd like to have more freedom for Python implementations, *especially* things that let you make more decisions at compile-time. (This is no doubt influenced by the fact that I've spent a lot of time thinking about Cython lately.) In this case, I see it as a win-win -- it gives more freedom to the folks writing the implementation, and (personally) I find it more pleasing as a user. Again, I don't think this *particular* case allows us to do something awesome behind the scenes with Cython -- but the community starting to consider changes of this ilk *would* be a big win, I think. -cc ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can Python implementations reject semantically invalid expressions?
Am 02.07.2010 18:51, schrieb Craig Citro: >> But you would be taking a module that will compile and making it >> uncompilable. >> > > You're absolutely right, and since I definitely *don't* think that the > program "raise TypeError" should cause a CompileError, you could say > it's safer to have a simple rule like "vaild syntax => will compile" > -- it's probably a slippery slope once you start deciding which bits > of semantics raise CompileErrors and which don't. > > However, in this particular case, here's a question: *why* would > someone write "return 1 + '1'"? Did they do it *knowing* what would > happen, or because they just didn't realize it was just an error? > > * If they knew what it was going to do, then I'd say shame on them -- > they should have just raised a TypeError instead, and anyone who comes > along to read or maintain that code would thank them for the change. "1/0" is much faster to type than "raise SomeError" and serves the same purpose sometimes for debugging purposes. Let's not forget that not all code is written for eternity :) Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can Python implementations reject semantically invalid expressions?
On Sat, Jul 3, 2010 at 2:51 AM, Craig Citro wrote: >> But you would be taking a module that will compile and making it >> uncompilable. >> > > You're absolutely right, and since I definitely *don't* think that the > program "raise TypeError" should cause a CompileError, you could say > it's safer to have a simple rule like "vaild syntax => will compile" > -- it's probably a slippery slope once you start deciding which bits > of semantics raise CompileErrors and which don't. > > However, in this particular case, here's a question: *why* would > someone write "return 1 + '1'"? Did they do it *knowing* what would > happen, or because they just didn't realize it was just an error? To test that adding a string to an integer raises TypeError at runtime. That is, something along the lines of: with self.assertRaises(TypeError): 1 + "1" If an end user is doing it rather than an implementation's own test suite... well, I have no idea why anyone else would want to do that :) > My real motive, though, is that I'd like to have more freedom for > Python implementations, *especially* things that let you make more > decisions at compile-time. (This is no doubt influenced by the fact > that I've spent a lot of time thinking about Cython lately.) In this > case, I see it as a win-win -- it gives more freedom to the folks > writing the implementation, and (personally) I find it more pleasing > as a user. Again, I don't think this *particular* case allows us to do > something awesome behind the scenes with Cython -- but the community > starting to consider changes of this ilk *would* be a big win, I > think. It's definitely something of a grey area. The existing test suite would likely fail if "obviously insane" operations between literals started throwing SyntaxError, but it would be possible to classify some of those tests as implementation specific. However, an implementation that did that would need to completely fork any affected parts of the test suite, since the implementation specific test decorators won't help protect against failures to compile the code. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can Python implementations reject semantically invalid expressions?
> "1/0" is much faster to type than "raise SomeError" and serves the same > purpose sometimes for debugging purposes. Let's not forget that not > all code is written for eternity :) > Doesn't "raise" do the same thing for just two extra characters? I agree that not all code lives forever -- but I bet we all have stories about "debugging code" living on a lot longer than it should have. ;) -cc ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can Python implementations reject semantically invalid expressions?
On Sat, Jul 3, 2010 at 3:13 AM, Craig Citro wrote: >> "1/0" is much faster to type than "raise SomeError" and serves the same >> purpose sometimes for debugging purposes. Let's not forget that not >> all code is written for eternity :) >> > > Doesn't "raise" do the same thing for just two extra characters? No, raise on its own is only valid in an exception handler. Writing "1/0" is at least somewhat common as an idiom for forcing a ZeroDivisionError in examples and in test harnesses (I know I have used it for both of those things many times). Given the diverse range of uses Python is put to, moving things from runtime to compile time can definitely have significant unexpected consequences (hence why many of us would be hesitant to consider an implementation that made such changes to be an actual Python implementation). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can Python implementations reject semantically invalid expressions?
> To test that adding a string to an integer raises TypeError at > runtime. That is, something along the lines of: > > with self.assertRaises(TypeError): > 1 + "1" > Well, this would just mean the test suite would have to change -- that test would become something like with self.assertRaises(TypeError): import operator operator.add(1, "1") Of course, checking that the literal syntax now raises a compile error could be ugly: with self.assertRaises(CompileError): eval('1 + "1"') ... or it could move to test_parser. ;) > If an end user is doing it rather than an implementation's own test > suite... well, I have no idea why anyone else would want to do that :) > Exactly -- and if it's a clear win for users, I don't think "it makes test-writing harder but not impossible" should really be a counter-argument. Of course, "there's lots of existing code it would break" is a very good counter-argument ... maybe Py4k. ;) > It's definitely something of a grey area. The existing test suite > would likely fail if "obviously insane" operations between literals > started throwing SyntaxError, but it would be possible to classify > some of those tests as implementation specific. However, an > implementation that did that would need to completely fork any > affected parts of the test suite, since the implementation specific > test decorators won't help protect against failures to compile the > code. > Well, I think there's some momentum towards splitting some of the tests into Python-standard and implementation-specific things, so that they can get shared between implementations, right? As long as clear lines are drawn, isn't it just a question of which bucket these tests go into? -cc ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can Python implementations reject semantically invalid expressions?
On Fri, Jul 2, 2010 at 10:28 AM, Nick Coghlan wrote: > On Sat, Jul 3, 2010 at 3:13 AM, Craig Citro wrote: >>> "1/0" is much faster to type than "raise SomeError" and serves the same >>> purpose sometimes for debugging purposes. Let's not forget that not >>> all code is written for eternity :) >>> >> >> Doesn't "raise" do the same thing for just two extra characters? > > No, raise on its own is only valid in an exception handler. Writing > "1/0" is at least somewhat common as an idiom for forcing a > ZeroDivisionError in examples and in test harnesses (I know I have > used it for both of those things many times). > > Given the diverse range of uses Python is put to, moving things from > runtime to compile time can definitely have significant unexpected > consequences (hence why many of us would be hesitant to consider an > implementation that made such changes to be an actual Python > implementation). +1 on not changing this. For one, this will most likely break a large amount of 3rd party and stdlib software -- there are tons of statements like this that are practically unreachable or intentional. Second, I don't think it's going to make the kind of difference the OP is thinking of. Since Python is totally dynamic, and doesn't have macros, the only cases that would be caught would be things you are unlikely to type by accident -- like 1/0 or 1+"1". In other languages that have this behavior, there is usually a benefit where the arguments involved are *variables* whose type is known to the compiler, so it will catch things like (simple C example) #define FOO 0 main() { printf("%d\n", 1/FOO); } However the equivalent Python FOO = 0 def main(): print 1/FOO cannot be rejected at compile time because there is insufficient evidence that the value of FOO won't be changed before main() is called. I even reject the substitution of "raise ZeroDivisionError" for "1/0" since (a) nobody cares about such an optimization, and (b) it would break introspection and invalidate tests. (We have a long history of constant propagation in expressions causing subtle bugs. This could be worse.) -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can Python implementations reject semantically invalid expressions?
2010/7/2 Guido van Rossum > On Fri, Jul 2, 2010 at 10:28 AM, Nick Coghlan wrote: > > On Sat, Jul 3, 2010 at 3:13 AM, Craig Citro > wrote: > >>> "1/0" is much faster to type than "raise SomeError" and serves the same > >>> purpose sometimes for debugging purposes. Let's not forget that not > >>> all code is written for eternity :) > >>> > >> > >> Doesn't "raise" do the same thing for just two extra characters? > > > > No, raise on its own is only valid in an exception handler. Writing > > "1/0" is at least somewhat common as an idiom for forcing a > > ZeroDivisionError in examples and in test harnesses (I know I have > > used it for both of those things many times). > > > > Given the diverse range of uses Python is put to, moving things from > > runtime to compile time can definitely have significant unexpected > > consequences (hence why many of us would be hesitant to consider an > > implementation that made such changes to be an actual Python > > implementation). > > +1 on not changing this. > > For one, this will most likely break a large amount of 3rd party and > stdlib software -- there are tons of statements like this that are > practically unreachable or intentional. > > Second, I don't think it's going to make the kind of difference the OP > is thinking of. Since Python is totally dynamic, and doesn't have > macros, the only cases that would be caught would be things you are > unlikely to type by accident -- like 1/0 or 1+"1". In other languages > that have this behavior, there is usually a benefit where the > arguments involved are *variables* whose type is known to the > compiler, so it will catch things like (simple C example) > > #define FOO 0 > main() { > printf("%d\n", 1/FOO); > } > > However the equivalent Python > > FOO = 0 > def main(): > print 1/FOO > > cannot be rejected at compile time because there is insufficient > evidence that the value of FOO won't be changed before main() is > called. > > I even reject the substitution of "raise ZeroDivisionError" for "1/0" > since (a) nobody cares about such an optimization, and (b) it would > break introspection and invalidate tests. (We have a long history of > constant propagation in expressions causing subtle bugs. This could be > worse.) > > -- > --Guido van Rossum (python.org/~guido) > from __future__ import compile_checks Cesare Di Mauro ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can Python implementations reject semantically invalid expressions?
On 7/2/2010 12:43 AM, Glyph Lefkowitz wrote: def f(): return 1 + "1" instead of compiling something which can't fail to raise an exception, would that still be a legal Python implementation? I'd say "no". Python has defined semantics in this situation: a TypeError is raised. The manuals are rather inconsistent about defining the exception semantics. Some parts define the exception returned. Other, equivalent parts, do not. I should start a separate thread on this when I find the examples I once had. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can Python implementations reject semantically invalid expressions?
Wow! I didn't expect anywhere near this amount of interest. Thanks to all who responded. One small comment follows: On Sat, 3 Jul 2010 03:44:05 am Guido van Rossum wrote: > On Fri, Jul 2, 2010 at 10:28 AM, Nick Coghlan wrote: > > Given the diverse range of uses Python is put to, moving things > > from runtime to compile time can definitely have significant > > unexpected consequences (hence why many of us would be hesitant to > > consider an implementation that made such changes to be an actual > > Python implementation). > > +1 on not changing this. > > For one, this will most likely break a large amount of 3rd party and > stdlib software -- there are tons of statements like this that are > practically unreachable or intentional. > > Second, I don't think it's going to make the kind of difference the > OP is thinking of. As I said in my initial post, this was a hypothetical, not a serious suggestion for a change, so I'm not pushing for it. I'm not even sure I would vote for change -- somebody asked on #python whether implementations were free to reject semantically impossible code at compile-time, my first instinct was to say Yes, and then I thought about it a bit more and thought "maybe not" and decided to ask here. I'm glad I did, because I've learned a lot. *If* such a change was made, I think it would have to be controlled by a command-line switch or environmental variable, like -O, and documented as potentially changing the behaviour of the program. But given how few accidental errors are likely to be caught by this, I doubt it would be of any real benefit. Thanks to all who answered! -- Steven D'Aprano ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can Python implementations reject semantically invalid expressions?
Stefan Behnel wrote: So, would it still be Python if it folded 1 + "1" into raise TypeError() at compile time? It would have to be raise TypeError("Exactly the message that would have been produced at run time") That might be acceptable, but then you have to ask, is it really worth performing this optimisation? The overhead of raising and handling the exception is likely to completely swamp that of executing the original code. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can Python implementations reject semantically invalid expressions?
Craig Citro wrote: Ok, I'm obviously being silly here, but sure you can: dis.dis("raise TypeError()") If producing different bytecode were considered a reason against performing an optimisation, then no code optimisations would be permissible at all! -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can Python implementations reject semantically invalid expressions?
Steven D'Aprano wrote: if the keyhole optimizer raised SyntaxError (or some other exception) on seeing this: def f(): return 1 + "1" That might break code that was deliberately trying to raise an exception. Sometimes you see things like try: 1/0 except Exception, e: ... Usually this kind of thing is only done in test code or illustrative snippets, but even so, it should work as expected. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can Python implementations reject semantically invalid expressions?
On Sat, 3 Jul 2010 11:39:07 am Greg Ewing wrote: > Stefan Behnel wrote: > > So, would it still be Python if it folded > > > > 1 + "1" > > > > into > > > > raise TypeError() > > > > at compile time? > > It would have to be > > raise TypeError("Exactly the message that would have been > produced at run time") Python doesn't make any guarantees about the message that exceptions display, so I don't think you need to match the message, just the exception. Anyone testing for specific exception messages is living in a state of sin and shouldn't complain when their code stops working. An implementation might choose to raise TypeError('is this the right place for an argument?') on alternate Tuesdays, and it would still meet the promises made by the language. -- Steven D'Aprano ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can Python implementations reject semantically invalid expressions?
Craig Citro wrote: However, in this particular case, here's a question: *why* would someone write "return 1 + '1'"? They might not intend to execute the code at all -- e.g. they may want to pass the compiled code to dis() to find out what bytecode gets generated. Having it refuse to compile would be annoying in that case. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can Python implementations reject semantically invalid expressions?
Steven D'Aprano, 03.07.2010 06:35: On Sat, 3 Jul 2010 11:39:07 am Greg Ewing wrote: Stefan Behnel wrote: So, would it still be Python if it folded 1 + "1" into raise TypeError() at compile time? It would have to be raise TypeError("Exactly the message that would have been produced at run time") Python doesn't make any guarantees about the message that exceptions display, so I don't think you need to match the message, just the exception. Anyone testing for specific exception messages is living in a state of sin and shouldn't complain when their code stops working. Yep, the Cython project keeps running into this all over the place. When we reimplement CPython functionality for performance reasons, we make sure we match the behaviour exactly, which usually means that we try to match the exceptions and their messages as well. Since we use doctests for our test suite, this means that we need to special case some Python versions in the test suite to make sure we test our code as good as possible without letting the tests break across CPython versions. This can be really tricky at times. The general trend seems to be that modified exception messages become more exact and telling over time, so we tend to follow Python 3.x in our own choice of error messages. Stefan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can Python implementations reject semantically invalid expressions?
On Fri, Jul 2, 2010 at 10:35 PM, Steven D'Aprano wrote: > On Sat, 3 Jul 2010 11:39:07 am Greg Ewing wrote: >> Stefan Behnel wrote: >> > So, would it still be Python if it folded >> > >> > 1 + "1" >> > >> > into >> > >> > raise TypeError() >> > >> > at compile time? >> >> It would have to be >> >> raise TypeError("Exactly the message that would have been >> produced at run time") > > > Python doesn't make any guarantees about the message that exceptions > display, so I don't think you need to match the message, just the > exception. Anyone testing for specific exception messages is living in > a state of sin and shouldn't complain when their code stops working. An > implementation might choose to raise TypeError('is this the right place > for an argument?') on alternate Tuesdays, and it would still meet the > promises made by the language. > In theory it does not, in practice people rely on it. Besides, I would not be overly happy if: def f(): return 1 + "1" and def g(): a = 1 return a + "1" raise different messages ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com